JP2007257299A - Vehicle external recognition system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To properly control a vehicle state in accordance with a recognition result of a plurality of traffic signals and stop signs existing in the traveling direction of the vehicle. <P>SOLUTION: A location determining part 23 determines a detection target existing at a closest position in front of one's own vehicle on the basis of a road structure recognized by a road structure recognizing part 21 and a detection target (at least traffic signal or stop sign) detected by the vehicle external recognizing part 22. The location determining part 23 determines that the largest detection target on image data among a plurality of detection targets is a detection target existing at the closest position in front of the one's own vehicle when a road shape ahead in the traveling direction of the one's own vehicle is a straight-ahead shape, and determines that a detection target existing at relatively opposite side in a circulation direction on the image data among a plurality of detection targets is a detection target existing at the closest position in front of the one's own vehicle when the road shape ahead in the traveling direction of the one's own vehicle is a curved shape. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a vehicle exterior recognition device.

Conventionally, for example, image processing such as binarization processing is performed on an image obtained by shooting with a black and white camera, an edge corresponding to the contrast of pixels constituting the image data after this image processing is extracted, and the image data A vehicle is known that determines that a stop sign has been shot when a trapezoidal edge and a triangular edge are present at the upper and lower portions (see, for example, Patent Document 1).
Conventionally, for example, color information of red R, blue B, and green G of an image obtained by shooting with a color camera is converted into color attributes of hue H, saturation S, and brightness I, and the image after this conversion processing is performed. If there is a figure in the data that is equivalent to a triangle-shaped keynote whose area ratio of the red area is greater than or equal to a predetermined value, if the red area of this figure is 50% or more, it is a stop sign and less than 50% If it is then, the apparatus which judges that it is a slow sign is known (for example, refer patent document 2).
Conventionally, for example, a notification device for recognizing characters constituting various traffic signs by pattern matching from an image obtained by, for example, shooting by a camera and notifying a vehicle occupant of the recognition result (for example, (See Patent Document 3).
Japanese Patent Laid-Open No. 9-180090 Japanese Patent Laid-Open No. 9-185703 Japanese Patent Publication No. 4-5239

By the way, in the above-described prior art, only a traffic sign is detected from image data obtained by photographing with a vehicle-mounted camera. For example, depending on the installation position of the traffic sign, the shooting direction of the vehicle-mounted camera, etc. A traffic sign may be detected on the same image. For this reason, in the above conventional technology, for example, when controlling the running state of the vehicle according to the detected traffic sign, it is appropriately determined which of the detected traffic signs is to be executed. It is hoped that.
The present invention has been made in view of the above circumstances, and provides an out-of-vehicle recognition device capable of appropriately controlling the state of a vehicle according to the recognition results of a plurality of traffic lights and a stop sign present in the traveling direction of the vehicle. The purpose is to do.

  In order to solve the above-described problems and achieve the object, an outside recognition device according to the present invention according to claim 1 is provided in a vehicle and recognizes an existence of an intersection in the traveling direction of the vehicle (for example, The road structure recognition unit 21) in the embodiment, the photographing means for photographing a predetermined area in the traveling direction of the vehicle (for example, the external camera 13 in the embodiment), and the image photographed by the photographing means A vehicle outside recognition device comprising vehicle outside recognition means (for example, vehicle outside recognition unit 22 in the embodiment) for recognizing at least one of the traffic signal and the stop sign on the basis of the intersection. Road structure recognizing means for recognizing the road structure including the road structure recognizing unit (for example, also serving as the road structure recognizing unit 21 in the embodiment), and at least a plurality of the traffic lights or images in the image photographed by the photographing means In the case where the stop sign is present, determination means for determining the order in which the plurality of the traffic lights or the stop signs are present based on the road structure recognized by the road structure recognition means (for example, in the embodiment) The existence position determination unit 23) is provided.

  According to the above outside recognition device, when there are a plurality of traffic lights or stop signs in the image taken by the photographing means, a plurality of traffic lights or stop signs based on the road structure recognized by the road structure recognition means. By determining the order in which the vehicles exist, that is, the order of arrangement along the traveling direction of the vehicle, it is possible to perform appropriate traveling control on the vehicle.

  Furthermore, in the vehicle exterior recognition apparatus according to the second aspect of the present invention, the determination means includes any one of the plurality of at least the traffic lights or the stop signs present in an image photographed by the photographing means. It is characterized by determining whether the traffic light or the said stop sign exists in the position nearest to a vehicle.

  According to the above-described vehicle recognition device, it is possible to perform appropriate traveling control on the vehicle by detecting a traffic light or a stop sign present at a position closest to the vehicle.

  Furthermore, in the vehicle exterior recognition apparatus according to the third aspect of the present invention, the road structure recognition means recognizes at least one of a road curve, a gradient, and an altitude as the road structure.

  According to the above vehicle exterior recognition device, it is possible to accurately determine the order of presence of a plurality of traffic lights or stop signs, that is, the arrangement order along the traveling direction of the vehicle, based on at least one of a road curve, a slope, and an altitude. it can.

  Furthermore, in the vehicle exterior recognition apparatus according to the fourth aspect of the present invention, the determination unit includes the plurality of at least the traffic lights or the stop sign on the image based on the road structure recognized by the road structure recognition unit. It is characterized by estimating the existence position of.

  According to the above-described vehicle outside recognition device, a plurality of traffic lights or traffic lights are estimated based on the road structure, and a plurality of traffic lights are determined based on the estimation result and appropriate recognition processing for the images. Alternatively, the detection accuracy of the stop sign can be improved.

  Further, in the vehicle exterior recognition apparatus according to the present invention as set forth in claim 5, the determination means sets the presence position on the image as a presence area having a predetermined size.

  According to the above-described vehicle outside recognition device, a plurality of traffic lights or temporary stop signs are set on the image for each of a plurality of traffic lights or temporary stop signs. The detection accuracy of the traffic light or the stop sign can be improved.

  Furthermore, in the outside recognition apparatus of the present invention according to claim 6, the determination means recognizes at least the size of the traffic light or the stop sign or a predetermined portion, and the plurality of the plurality of signals within the same existence area. The presence order is determined based on the magnitude when at least the traffic light or the stop sign is present.

  According to the above vehicle exterior recognition device, by determining the order of presence based on the presence area for each of a plurality of traffic lights or stop signs and the size of each of the plurality of traffic lights or stop signs, for example, a plurality of existence areas Even when there are overlapping regions between them, the determination accuracy can be improved.

  Further, in the vehicle exterior recognition device according to the present invention as set forth in claim 7, the determination means increases the size of the existence area as at least the position where the traffic light or the stop sign is present is close to the vehicle. It is characterized by setting to change to a trend.

  According to the above vehicle exterior recognition device, the plurality of traffic lights or temporary stop signs change in size on the image according to the relative position with respect to the vehicle. A plurality of traffic lights or stop signs can be appropriately detected by changing the size of the signal to an increasing tendency.

  Furthermore, in the vehicle exterior recognition apparatus according to the present invention described in claim 8, the determination means estimates the presence position on the image based on at least an installation height of the traffic light or the stop sign. It is said.

  According to the above-described vehicle outside recognition device, by setting the installation height of the traffic light or the installation height of the stop sign in advance, on the image for each of the plurality of traffic lights or stop signs based on the road structure. The estimation accuracy when estimating the presence position can be improved.

As described above, according to the outside recognition apparatus of the present invention, when there are a plurality of traffic lights or stop signs in an image obtained by photographing by the photographing means, the arrangement order along the traveling direction of the vehicle is changed. By determining, the traveling state of the vehicle can be appropriately controlled according to the determination result.
Furthermore, according to the vehicle exterior recognition apparatus of the second aspect of the present invention, it is possible to appropriately control the traveling state of the vehicle in accordance with the traffic light or the stop sign present at the position closest to the vehicle.
Furthermore, according to the third aspect of the present invention, the order of arrangement of a plurality of traffic lights or stop signs along the traveling direction of the vehicle is accurately determined based on at least one of a curve, a slope, and an altitude of the road. Can be judged well.

Furthermore, according to the outside recognition apparatus of the present invention as set forth in claim 4, a plurality of traffic lights or stop signs are obtained by the presence position on the image for each of the plurality of traffic lights or stop signs and appropriate recognition processing for the images. Detection accuracy can be improved.
Furthermore, according to the outside recognition apparatus of the present invention as set forth in claim 5, a plurality of traffic lights or stop signs are obtained by a presence area on an image for each of a plurality of traffic lights or stop signs and an appropriate recognition process for the image. Detection accuracy can be improved.

Furthermore, according to the outside recognition apparatus of the present invention described in claim 6, it is possible to improve the determination accuracy of the presence order based on the existence area and the size for each of the plurality of traffic lights or the stop signs.
Furthermore, according to the outside recognition apparatus of this invention of Claim 7, a some traffic signal or a stop sign can be detected appropriately.
Furthermore, according to the outside recognition apparatus of the present invention described in claim 8, the presence position on the image for each of the plurality of traffic lights or stop signs is determined by the installation height of the traffic lights or the installation height of the stop signs. The estimation accuracy at the time of estimation can be improved.

Hereinafter, an outside recognition device according to an embodiment of the present invention will be described with reference to the accompanying drawings.
As shown in FIG. 1, for example, the vehicle control device 10 including the vehicle exterior recognition device 1 according to the present embodiment includes a navigation device 11, a map data storage device 12, an external camera 13, a host vehicle state sensor 14, A braking device 15, an alarm device 16, and a processing device 17 are provided.

  The navigation device 11 uses a positioning signal such as a GPS (Global Positioning System) signal for measuring the position of the vehicle using an artificial satellite, for example, or the current position of the host vehicle based on each detection result such as a gyro sensor and an acceleration sensor. And the traveling direction is detected, and based on the detection result, map matching is performed on the map data stored in the map data storage device 12, and the display position of the current position of the vehicle on the display screen is controlled. The map display on the display screen is controlled with respect to the detected current position of the host vehicle or an appropriate vehicle position input by the operator via various switches, a keyboard, and the like.

In addition to map display data for displaying a map on the display screen of the navigation device 11 and road coordinate data required for map matching processing based on the current position of the host vehicle, the map data storage device 12 Data necessary for processing such as route search and route guidance, for example, each data of a node that is a point consisting of latitude and longitude at a predetermined position such as an intersection and a link that is a line connecting each node as map data In addition, various information is added to the nodes and links.
For example, as shown in FIG. 2, when an intersection exists on the road ahead of the traveling direction of the host vehicle A, a plurality of nodes N1,..., N4 and links L1, L2 connecting the nodes N1,. , L3 are set. For example, for the node N2 indicating the position of the intersection, in addition to the latitude, longitude, and altitude, intersection information such as the presence / absence of a traffic light, the intersection angle and shape of a plurality of roads, and the like are added. L3 is added with a road type (for example, national road, prefectural road, city road, etc.), road width information (for example, width data), and road structure information (for example, the number of lanes).

  The external camera 13 is configured to include a camera such as a CCD camera or a CMOS camera that can shoot in the visible light region and an image processing unit. For example, the camera disposed in the passenger compartment moves in the traveling direction of the host vehicle through the front window. The outside of a predetermined detection range ahead (for example, a region ahead of a predetermined distance according to the traveling state of the host vehicle, the environment of the host vehicle, etc., and depending on the road shape such as the gradient and curvature of the host vehicle's driving path) A region around a position where a road sign is estimated to exist is photographed at predetermined time intervals. Then, the image processing unit performs predetermined image processing such as filtering on the image obtained by photographing with the camera, generates image data including pixels of a two-dimensional array, and outputs the image data to the processing device 17.

  The own vehicle state sensor 14 is a vehicle speed sensor that detects the speed (vehicle speed) of the own vehicle, a yaw angle (a rotation angle around the vertical axis of the vehicle center of gravity), or a yaw rate (up and down A yaw rate sensor that detects the rotational angular velocity around the direction axis, and a steering angle sensor that detects the steering angle (direction and size of the steering angle input by the driver) and the actual steering angle (steering angle) according to the steering angle In addition, each sensor is configured to include a direction indicator and each sensor for detecting the on / off state of the brake.

The braking device 15 is, for example, a brake control device or the like, and controls the brake fluid pressure or the like, for example, according to a control signal input from the processing device 17 so as to apply a braking force to the host vehicle.
The alarm device 16 includes, for example, a tactile transmission device, a visual transmission device, and an auditory transmission device.
The tactile transmission device is, for example, a seat belt device, a steering control device, or the like. In response to a control signal input from the processing device 17, for example, a predetermined tension is generated on the seat belt so that the occupant of the host vehicle is tactile. Applying a perceptible tightening force to the vehicle or generating vibrations (steering vibrations) that can be perceived tactilely by the driver of the host vehicle on the steering wheel. To recognize.
The visual transmission device is, for example, a display device or the like, and according to a control signal input from the processing device 17, for example, by displaying predetermined alarm information on the display device or blinking a predetermined alarm light, Let the occupant recognize the necessity of stopping the vehicle at the intersection.
The auditory transmission device is, for example, a speaker or the like, and outputs a predetermined warning sound or voice in response to a control signal input from the processing device 17, thereby confirming the necessity of stopping the vehicle at the intersection. To recognize.

  The processing device 17 includes, for example, a road structure recognition unit 21, a vehicle outside recognition unit 22, a presence position determination unit 23, and a travel control unit 24.

The road structure recognition unit 21 is based on the current position and traveling direction of the host vehicle detected by the navigation device 11 and the map data stored in the map data storage device 12, and the road structure ahead of the traveling direction of the host vehicle (for example, Recognize the existence of intersections, road shapes, curves, slopes, elevations, etc.).
The “curve” mentioned here does not mean only a so-called curve shape, but also includes a crank shape and the like. When the vehicle is used as a reference, a plurality of intersections are arranged in the horizontal (horizontal) direction. It also includes a state in which a plurality of intersections do not exist in a straight line with respect to the own vehicle, such as being present in the traveling direction with the position shifted.
The vehicle outside recognition unit 22 detects at least a traffic light or a stop sign from image data output from the external camera 13 by a recognition process targeting at least a traffic light or a stop sign ahead of the traveling direction of the host vehicle.

  Based on the road structure recognized by the road structure recognition unit 21 and the detection target (that is, at least a traffic light or a stop sign) detected by the vehicle outside recognition unit 22, the presence position determination unit 23 is the closest position to the host vehicle. In the case where a detection target existing in the vehicle or a plurality of detection targets is detected, the presence order of these detection targets with respect to the host vehicle is determined.

  Then, the traveling control unit 24 sets a traveling support level for the traveling state of the host vehicle based on the determination result by the presence position determining unit 23, and activates the braking device 15 and the alarm device 16 according to the traveling support level. For example, the traveling control unit 24 stops when the host vehicle reaches a predetermined alarm position before the temporary stop sign in a state where the driver's intention to decelerate based on the output of the host vehicle state sensor 14 is not detected. When the vehicle reaches a predetermined deceleration position in a state where the speed of the host vehicle is equal to or higher than the predetermined speed, the host vehicle is automatically decelerated.

  The vehicle exterior recognition device 1 according to the present embodiment has the above-described configuration. Next, the operation of the vehicle exterior recognition device 1, in particular, a traffic light or a stop sign that is present at the closest position to the host vehicle by the presence position determination unit 23. A process for determining the above will be described.

First, a determination process based on a change in road shape in a horizontal plane will be described below.
For example, in step S01 shown in FIG. 3, at least a traffic light or a stop sign is displayed on the image data output from the external camera 13 by a recognition process targeting a traffic light or a stop sign ahead of the traveling direction of the host vehicle. To detect.

In step S02, it is determined whether or not a plurality of detection targets (that is, a plurality of at least traffic lights or stop signs) are detected.
When this determination result is “YES”, for example, when two stop signs M1 and M2 are detected as shown in FIG. 4B by the recognition process for the image data P1 shown in FIG. It progresses to step S04 mentioned later.
On the other hand, if this determination is “NO”, the flow proceeds to step S 03, and in this step S 03, the detected single traffic light or the stop sign is the detection target present closest to the host vehicle. And the series of processing ends.

Further, in step S04, based on the current position and traveling direction of the host vehicle detected by the navigation device 11 and the map data stored in the map data storage device 12, the road structure ahead of the traveling direction of the host vehicle, for example, Recognize road shapes such as curves.
For example, in a road structure as shown in FIG. 4 (a), as shown in FIG. 4 (c), a straight path R1, a first intersection C1, and a right-handed curved road are sequentially provided in front of the traveling direction of the host vehicle A. R2, the second intersection C2, and the straight path R3 are recognized.

In step S05, it is determined whether the road shape ahead of the traveling direction of the host vehicle is a straight traveling shape.
If this determination is “NO”, the flow proceeds to step S 07 described later.
On the other hand, if this determination is “YES”, the flow proceeds to step S 06, and in this step S 06, the detection target having the maximum size on the image data among the plurality of detection targets is the largest for the host vehicle. It determines with it being the detection target which exists in a near position, and complete | finishes a series of processes.

Moreover, in step S07, it is determined whether the road shape ahead of the own vehicle is a left curve shape.
When the determination result is “YES”, the process proceeds to step S08, and in this step S08, the detection target that is relatively on the right side of the image data among the plurality of detection targets is the most relative to the host vehicle. It determines with it being the detection target which exists in a near position, and complete | finishes a series of processes.
On the other hand, if this determination is “NO”, that is, if the road shape ahead of the host vehicle is a right curve as shown in FIG. 4C, the process proceeds to step S09, and in this step S09. Detects a detection target (for example, the temporary stop sign M1 shown in FIG. 4D) present on the relatively left side of the image data among a plurality of detection targets at the closest position to the host vehicle. It determines with it being object, and complete | finishes a series of processes.

Next, the determination process based on the change in the road shape in the vertical direction will be described.
For example, in step S11 shown in FIG. 5, at least a traffic light or a stop sign is displayed on the image data output from the external camera 13 by a recognition process for a traffic light or a stop sign ahead of the traveling direction of the host vehicle. To detect.

In step S12, it is determined whether or not a plurality of detection targets (that is, a plurality of at least traffic lights or stop signs) are detected.
When the determination result is “YES”, as shown in FIG. 6B or FIG. 7B, for example, by the recognition processing for the image data P2 and P3 shown in FIG. 6A or FIG. 7A. When two temporary stop signs M1 and M2 are detected, the process proceeds to step S14 described later.
On the other hand, if this determination is “NO”, the flow proceeds to step S 13, and in this step S 13, the detected single traffic light or the stop sign is the detection target present closest to the host vehicle. And the series of processing ends.

In step S14, based on the current position and the traveling direction of the host vehicle detected by the navigation device 11 and the map data stored in the map data storage device 12, the road structure ahead of the traveling direction of the host vehicle, for example, Recognize road shapes such as slope and elevation.
For example, in a road structure as shown in FIG. 6 (a), as shown in FIGS. 6 (c) and 6 (d), a straight road R1 with a downward slope and a first intersection sequentially in front of the traveling direction of the host vehicle A. C1, the up-straight straight path R2, the second intersection C2, and the up-gradient straight path R3 are recognized.
Further, for example, in a road structure as shown in FIG. 7A, a flat straight road R1 and a first straight road R1 are sequentially provided in front of the traveling direction of the host vehicle A as shown in FIGS. 7C and 7D. An intersection C1, a flat straight path R2, a second intersection C2, and a flat straight path R3 are recognized.

In step S15, it is determined whether or not the road shape ahead of the traveling direction of the host vehicle is an upward gradient.
If this determination is “NO”, the flow proceeds to step S 17 described later.
On the other hand, when the determination result is “YES”, for example, as shown in FIGS. 6C and 6D, when the road shape after the first intersection C1 is an uphill in front of the traveling direction of the host vehicle, Proceeding to step S16, in this step S16, the detection target present on the lower side in the vertical direction relative to the image data among the plurality of detection targets is the detection target existing closest to the host vehicle. And the series of processing ends.

Moreover, in step S17, it is determined whether the road shape ahead of the traveling direction of the host vehicle is a downward slope.
If this determination is “YES”, the flow proceeds to step S 18, and in this step S 18, a detection target that is relatively above the image data in the vertical direction among the plurality of detection targets is assigned to the host vehicle. On the other hand, it is determined that the detection target is present at the nearest position, and the series of processes is terminated.
On the other hand, if this determination is “NO”, that is, if the road shape ahead of the host vehicle is substantially flat as shown in FIGS. 7C and 7D, the process proceeds to step S19. In step S19, for example, as in step S06 described above, the detection target having the maximum size on the image data (for example, the stop sign M1 shown in FIG. 7E) among the plurality of detection targets is displayed on the host vehicle. It is determined that the detection target is present at the foremost position, or among the plurality of detection targets, the detection target is present at the farthest distance from the vanishing point VP on the image data (for example, FIG. ) Is determined to be a detection target that is closest to the host vehicle, and the series of processes is terminated.

Note that the gradient of the road ahead of the traveling direction of the host vehicle is calculated based on, for example, the current position of the host vehicle and each intersection position information stored in the map data.
For example, as shown in FIG. 8, in the state where the host vehicle A that has passed through the first intersection S1 travels toward the second intersection S2 and the third intersection S3 existing forward in the traveling direction, each of the latitude, longitude, and altitude Depending on the position information of the intersections S1 (x1, y1, z1), S2 (x2, y2, z2), S3 (x3, y3, z3) and the current position AP of the host vehicle consisting of at least the latitude and longitude, that is, the latitude and longitude Based on the current position AP (xv, yv) or the current position AP (xv, yv, zv) composed of latitude, longitude and altitude, the relative angles θ1, θ2 between the host vehicle and the intersections S2, S3 can be calculated. .

For example, with respect to the current position AP (xv, yv, zv) by three-dimensional positioning, the elevation difference h2, h3 between the own vehicle and each of the intersections S2, S3 and the two-dimensional surface are calculated by the following formula (1). The distances d2 and d3 between the own vehicle and each of the intersections S2 and S3 are calculated.
And relative angle (theta) 1 and (theta) 2 of the own vehicle and each intersection S2, S3 are calculated by following Numerical formula (2).

For example, for the current position AP (xv, yv) by two-dimensional positioning, the distance d1 between the own vehicle and each intersection S1 on the two-dimensional surface and the first intersection S1 according to the following formula (3): And a distance d21 between the second intersection S2 and the second intersection S2.
Then, since the altitude zv of the host vehicle is calculated by the following formula (4), the relative angle θ1, between the host vehicle and each of the intersections S2 and S3 is thereafter calculated based on the formulas (1) and (2). θ2 is calculated.

  As described above, according to the vehicle exterior recognition device 1 according to the present embodiment, when there are a plurality of detection targets (that is, at least traffic lights or stop signs) in the image data captured by the external camera 13, the road Based on the road structure recognized by the structure recognition unit 21, the order of the presence of a plurality of traffic lights or stop signs, or the traffic light or stop sign present at the nearest position relative to the host vehicle is determined to determine the traffic lights or stop signs. Appropriate traveling control according to the vehicle can be performed.

  In the above-described embodiment, the presence position determination unit 23 is based on the road structure and the detection target (that is, at least a traffic light or a stop sign), or the detection target that is present at the closest position to the host vehicle, or The order of existence of a plurality of detection targets with respect to the host vehicle is determined. However, the present invention is not limited to this. Further, based on the road structure, the presence position of the detection target on the image data is estimated as an existence region having a predetermined size. Then, the position of the detection target may be determined based on the determination result as to whether or not each detection target detected by the vehicle exterior recognition unit 22 exists in the existence region.

  In the modification of the above-described embodiment, first, for example, in step S21 shown in FIG. 9, for the image data output from the external camera 13, the traffic light or the stop sign ahead in the traveling direction of the host vehicle is targeted. At least a traffic light or a stop sign is detected by the recognition processing performed.

In step S22, it is determined whether or not a plurality of detection targets (that is, a plurality of at least traffic lights or stop signs) are detected.
When the determination result is “YES”, as shown in FIG. 10B or FIG. 11B, for example, by the recognition processing for the image data P2 and P3 shown in FIG. 10A or FIG. If two temporary stop signs M1, M2 are detected, the process proceeds to step S24 described later.
On the other hand, if this determination is “NO”, the flow proceeds to step S 23, and in this step S 23, the detected single traffic light or the stop sign is the detection target present closest to the host vehicle. And the series of processing ends.

Further, in step S24, based on the current position and traveling direction of the host vehicle detected by the navigation device 11 and the map data stored in the map data storage device 12, the road structure ahead of the traveling direction of the host vehicle, for example, Recognize road shapes such as curves, slopes, and elevations.
Thus, for example, in a road structure as shown in FIG. 10 (a), as shown in FIG. 10 (c), the straight road R1, the first intersection C1, and the The direction bending path R2, the second intersection C2, and the straight path R3 are recognized.
Further, for example, in a road structure as shown in FIG. 11 (a), as shown in FIGS. The first intersection C1, the uphill straight path R2, the second intersection C2, and the uphill straight path R3 are recognized.

Furthermore, in step S24, based on the recognized road structure, the position on the image data when the detection target consisting of at least a traffic light or a stop sign is photographed by the external camera 13 at, for example, an intersection is estimated. Presence areas (for example, the respective existence areas AR1, AR2 shown in FIG. 10 (d) and the existence areas AR1, AR2 shown in FIG. 11 (e)) having a predetermined size around the position are set. To do.
Note that the size of each existence region is set so as to change in an increasing trend as the relative distance of each detection target to the host vehicle decreases, for example.

Then, in step S25, the number of detection targets detected in each existence area is acquired. And in step S26, a plurality of detection areas within the existence area corresponding to the intersection existing at the closest position to the host vehicle are obtained. It is determined whether a detection target has been detected.
If this determination is “NO”, the flow proceeds to step S 23 described above.
On the other hand, if this determination is “YES”, the flow proceeds to step S27.

Then, in step S27, it is determined whether or not a single detection target is detected only in the existence area corresponding to the intersection existing at the closest position to the host vehicle.
When this determination result is “YES”, for example, as shown in FIG. 10 (e) or FIG. 11 (f), when only each stop sign M1, M2 is detected in each existence region R1, R2, or For example, as shown in FIG. 12A, even if each of the existence areas AR1 and AR2 has an overlapping area with each other, a single stop sign M1 is provided only in the foreground existence area AR1. If detected, the process proceeds to step S23 described above.
On the other hand, when this determination result is “NO”, for example, as shown in FIG. 12B, when a plurality of stop signs M1, M2 are detected in the overlapping areas of the respective existence areas AR1, AR2, or For example, as shown in FIG. 12C, even if each of the existence areas AR1 and AR2 has an overlapping area, a plurality of stop signs M1 are provided only in the foreground existence area R1. , M2 is detected, the process proceeds to step S28.

  In step S28, a plurality of detection targets (for example, a plurality of temporary stop signs M1 shown in FIG. 12C) detected only in the existence area corresponding to the intersection existing at the closest position to the host vehicle. , M2), it is determined that the detection target having the maximum size is the detection target existing closest to the host vehicle, and the series of processing ends.

In the following, processing for estimating the presence position of the detection target on the image data as an existence area having a predetermined size based on the road structure will be described.
For example, in step S31 shown in FIG. 13, the distance from the host vehicle to the intersection, the relative azimuth of the intersection, and the elevation of the intersection are calculated based on the road structure based on the map data stored in the map data storage device 12.
Next, in step S32, based on the distance from the own vehicle to the intersection, the relative orientation of the intersection, and the elevation of the intersection, the distance from the own vehicle to the detection target (that is, at least the traffic light or the stop sign) and the relative detection target. Calculate the azimuth and elevation of the detection target.

Next, in step S33, the location of the detection target on the image data is estimated based on the distance from the host vehicle to the detection target, the relative orientation of the detection target, and the elevation of the detection target.
Note that the location of the detection target on the image data is estimated according to, for example, an installation position according to legal requirements, an installation position set appropriately in advance, etc. The lower end position (for example, 4.5 m, etc.), the predetermined lower end position (for example, 1.8 m, for example) for the installation position of the stop sign, for example, or the installation position of the traffic light in advance, for example. Within a predetermined height range (for example, 4 to 7 m, etc.) or a predetermined height range (for example, 1.5 to 3 m, etc.) appropriately set with respect to the installation position of the stop sign in advance. Estimated accordingly.
Next, in step S34, an existing area having a predetermined size is set based on the estimated existing position.

  In step S31 to step S34 described above, for example, when the road shape ahead of the own vehicle is flat, the object to be detected from the own vehicle based on the distance from the own vehicle to the intersection and the relative direction of the intersection. And the relative orientation of the detection target may be calculated, and the location of the detection target on the image data may be estimated based on only the distance and the relative orientation.

It is a block diagram of the vehicle control apparatus which comprises the external vehicle recognition apparatus which concerns on embodiment of this invention. It is a figure which shows an example of the node and link which comprise map data. It is a flowchart which shows the process which determines the traffic signal or stop sign which exists in the nearest position with respect to the own vehicle. FIG. 4A is a diagram illustrating an example of image data output from the external camera, and FIG. 4B is a diagram illustrating an example of a detection target detected on the image data, and FIG. FIG. 4 is a diagram illustrating an example of a road structure detected in front of the traveling direction of the host vehicle, and FIG. 4D is a diagram illustrating an example of a detection target that is present at the closest position to the host vehicle. It is a flowchart which shows the process which determines the traffic signal or stop sign which exists in the nearest position with respect to the own vehicle. FIG. 6A is a diagram illustrating an example of image data output from an external camera, and FIG. 6B is a diagram illustrating an example of a detection target detected on the image data, and FIG. FIG. 6 is a diagram illustrating an example of a road structure detected in front of the traveling direction of the host vehicle, and FIG. 6D is a diagram illustrating an example of a road structure detected in front of the traveling direction of the host vehicle. e) is a figure which shows an example of the detection target which exists in the nearest position with respect to the own vehicle. FIG. 7A is a diagram illustrating an example of image data output from the external camera, and FIG. 7B is a diagram illustrating an example of a detection target detected on the image data, and FIG. FIG. 7 is a diagram illustrating an example of a road structure detected in front of the traveling direction of the host vehicle, and FIG. 7D is a diagram illustrating an example of a road structure detected in front of the traveling direction of the host vehicle. e) is a figure which shows an example of the detection target which exists in the nearest position with respect to the own vehicle. It is a figure which shows an example of the relative position of present position PA of the own vehicle, and each intersection S1, S2, S3. It is a flowchart which shows the process which determines the traffic signal or stop sign which exists in the nearest position with respect to the own vehicle which concerns on the modification of embodiment of this invention. FIG. 10A is a diagram illustrating an example of image data output from the external camera, and FIG. 10B is a diagram illustrating an example of a detection target detected on the image data, and FIG. FIG. 10 is a diagram showing an example of a road structure detected in front of the traveling direction of the host vehicle, and FIG. 10D is a diagram showing an example of an existing area of a detection target estimated on image data. e) is a figure which shows an example of the detection target which exists in the nearest position with respect to the own vehicle. FIG. 11A is a diagram illustrating an example of image data output from the external camera, and FIG. 11B is a diagram illustrating an example of a detection target detected on the image data, and FIG. FIG. 11 is a diagram illustrating an example of a road structure detected in front of the traveling direction of the host vehicle, and FIG. 11D is a diagram illustrating an example of a road structure detected in front of the traveling direction of the host vehicle. FIG. 11E is a diagram illustrating an example of the existence region of the detection target estimated on the image data, and FIG. 11F is a diagram illustrating an example of the detection target existing at the closest position to the host vehicle. 12A to 12C are diagrams illustrating an example of the detection target existence area estimated on the image data. It is a flowchart which shows the process which estimates the presence position of the detection target on image data as a presence area | region which has a predetermined magnitude | size based on a road structure.

Explanation of symbols

1 Outside recognition device 10 Vehicle control device 13 Outside camera (photographing means)
21 Road structure recognition unit (intersection recognition means, road structure recognition means)
22 Outside recognition unit (outside recognition means)
23 Existence position determination unit (determination means)

Claims (8)

An intersection recognition means provided on the vehicle for recognizing the presence of an intersection in the traveling direction of the vehicle;
Photographing means for photographing a predetermined area in the traveling direction of the vehicle;
A vehicle exterior recognition device comprising vehicle exterior recognition means for recognizing at least one of the traffic signal at the intersection and a stop sign based on an image photographed by the photography means,
Road structure recognition means for recognizing the road structure including the intersection in the traveling direction of the vehicle;
When there are a plurality of at least the traffic lights or the stop signs in the image photographed by the photographing means, the plurality of at least the traffic lights or the pauses based on the road structure recognized by the road structure recognition means. A vehicle exterior recognition apparatus comprising: a determination unit that determines the presence order of the signs. The determination means is at least one of the plurality of traffic lights or the stop sign present in the image photographed by the photographing means at a position closest to the vehicle. The outside recognition device according to claim 1, wherein the determination is made as to whether the The vehicle exterior recognition device according to claim 1, wherein the road structure recognition unit recognizes at least one of a curve, a gradient, and an altitude of the road as the road structure. The said determination means estimates the presence position on the said image of at least the said traffic lights or the said stop sign based on the said road structure recognized by the said road structure recognition means. Item 4. The vehicle outside recognition device according to any one of Items 3 to 4. The vehicle outside recognition device according to claim 4, wherein the determination unit sets the presence position on the image as a presence area having a predetermined size. The determination means recognizes at least the size of the traffic light or the stop sign as a whole or a predetermined portion, and when the plurality of at least the traffic light or the stop sign exist in the same existence area, The outside recognition apparatus according to claim 5, wherein the presence order is determined based on a size. 6. The determination means is set so that the size of the presence area changes in an increasing tendency when at least the position where the traffic light or the stop sign is present approaches the vehicle. Or the recognition apparatus outside a vehicle of Claim 6. The said determination means estimates the said presence position on the said image based on the installation height of the said traffic light or the said stop sign at least, The Claim 1 characterized by the above-mentioned. Outside vehicle recognition device.

Images