JP6329442B2 - Outside environment recognition device - Google Patents

Description

  The present invention relates to a vehicle environment recognition apparatus that identifies a specific object existing around a host vehicle.

  Conventionally, a specific object such as a vehicle positioned in front of the host vehicle is specified, and a collision with a preceding vehicle is avoided (collision avoidance control), or the distance between the preceding vehicle and the preceding vehicle is controlled to be kept at a safe distance ( (Cruise Control) technology is known. Such forward monitoring technology of the own vehicle is expected to be effective in avoiding and reducing contact accidents with specific objects such as preceding vehicles and pedestrians.

  As a specific object specifying technique, for example, a distance image generated by stereo image processing is used, and image blocks whose three-dimensional positions are adjacent to each other are grouped, and the horizontal and vertical outermost ends of the grouped image blocks are grouped. A technique for specifying a rectangular region surrounding the image as a target image (image of a target object) that can be a control target is disclosed (for example, Patent Document 1).

JP 2013-203337 A

  A vehicle or a person positioned in front of the host vehicle uses, for example, two images captured by two imaging devices arranged in the vehicle with different optical axes, and a three-dimensional position (three-dimensional position) by pattern matching. ) Is identified and used for collision avoidance control and cruise control. However, for example, when the road surface is wet due to the influence of rain, light from a light source such as a traffic light or a streetlight arranged on the road may be reflected to form a light band extending in the vertical direction of the screen. . Then, there is a possibility that such an optical band that does not actually exist in front of the host vehicle as a three-dimensional object becomes a control target of the above-described collision avoidance control or cruise control.

  However, since the light band is an image reflected on the road surface, when the three-dimensional position is obtained, the light band exists vertically below the road surface. Therefore, it is conceivable that the target image (here, the light band) positioned vertically below the road surface is a special image that does not actually exist as a specific object and is excluded from the control target for collision avoidance control and cruise control.

  However, in the situation where the road surface is wet as described above, if there are a plurality of light sources having the same light emission mode, such as traffic lights, the light band reflected on the road surface also appears with the same color value and size. There is a case. At this time, there is no problem if the same optical bands are matched between the two images acquired from the two imaging devices, but the different optical bands are mismatched, so that the optical band is more than the position of the original optical band. It may be misrecognized as being located in front of the host vehicle. In that case, it may be determined that the light band is not located vertically below the road surface due to perspective, and as described above, it can be excluded from the target of collision avoidance control and cruise control as a special image. Disappear.

  In view of such a problem, the present invention provides an external environment recognition device capable of improving the accuracy of specifying a specific object by appropriately specifying a light band reflected on a road surface as a special image. It is aimed.

  In order to solve the above-described problem, the vehicle environment recognition apparatus according to the present invention compares a reference image captured by two imaging devices at different positions and an image acquisition unit that acquires a comparison image, and pattern matching. A comparison block correlated with an arbitrary reference block in the reference image is extracted from a predetermined range of the image, and an image processing unit that generates disparity information of the reference block; and 3 of the target part in the reference image using the disparity information A three-dimensional position deriving unit for deriving a three-dimensional position, a grouping unit for grouping target portions whose difference in three-dimensional position is within a predetermined range, and making a target image vertical in a reference image There is a target image showing a light source above, and an assumed mismatch distance that is a difference between the disparity information of the arbitrary target image and the assumed parallax in a predetermined horizontal direction starting from the arbitrary target image and the light source To the partial shift position, when the target image and a light source having a correlation with any subject image and the light source is present, characterized in that it comprises a special image determining section for determining a special image of any subject images, a.

  The special image determination unit further determines that the target image is a special image if the ratio of the edge direction perpendicular to the edge extension direction in the target image included in the predetermined direction range that is regarded as the horizontal direction is equal to or greater than a predetermined threshold. You may judge.

  The special image determination unit may further determine that the target image is a special image if the environment outside the vehicle is an environment where the road surface reflects the light source.

  The special image determination unit may determine that the target image is a special image if the target image itself does not include a light source.

  In order to solve the above-described problem, another external environment recognition device of the present invention includes a reference image captured by two imaging devices at different positions and an image acquisition unit that acquires a comparison image, and pattern matching. A comparison block correlated with an arbitrary reference block in the reference image is extracted from a predetermined range of the comparison image, and an image processing unit that generates disparity information, and a three-dimensional target region in the reference image using the disparity information A three-dimensional position deriving unit for deriving a position, a grouping unit for grouping target parts whose difference in three-dimensional position is within a predetermined range, and setting a target image as a starting point in a reference image When a target image correlated with an arbitrary target image exists at a position shifted by an assumed mismatch distance that is a difference between the parallax information of the arbitrary target image and the assumed parallax in a predetermined horizontal direction, And special image determining section for determining a target image at will a special image, characterized in that it comprises a.

  A specific object specifying unit that excludes the target image determined by the special image determination unit to be a special image from the one or more target images grouped by the grouping unit, and specifies a specific object from the remaining target images; You may prepare.

  According to the present invention, it is possible to improve the accuracy of specifying a specific object by appropriately specifying the light band reflected on the road surface as a special image.

It is the block diagram which showed the connection relation of the environment recognition system. It is the functional block diagram which showed the schematic function of the external environment recognition apparatus. It is a flowchart which shows the flow of a vehicle exterior environment recognition process. It is explanatory drawing for demonstrating an example of pattern matching. It is explanatory drawing for demonstrating determination of a special image. It is explanatory drawing for demonstrating the other example of pattern matching. It is explanatory drawing for demonstrating determination of a special image. It is explanatory drawing for demonstrating a color image and a distance image. It is a flowchart which shows the flow of a special image determination process. It is explanatory drawing for demonstrating edge direction determination processing. It is explanatory drawing for demonstrating a light source position determination process, a light source mismatch determination process, and a target image mismatch determination process.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The dimensions, materials, and other specific numerical values shown in the embodiment are merely examples for facilitating understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted, and elements not directly related to the present invention are not illustrated. To do.

(Environment recognition system 100)
FIG. 1 is a block diagram showing a connection relationship of the environment recognition system 100. The environment recognition system 100 includes an imaging device 110, a vehicle exterior environment recognition device 120, and a vehicle control device (ECU: Engine Control Unit) 130 provided in the host vehicle 1.

  The imaging device 110 includes an imaging element such as a CCD (Charge-Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor). The imaging device 110 captures an environment corresponding to the front of the host vehicle 1, Alternatively, a luminance image (monochrome image) with luminance (Y) can be generated. Here, the color value is from a YUV format color space consisting of one luminance (Y) and two color differences (U, V), and three hues (R (red), G (green), B (blue)). A numerical value group represented by any one of the RGB color space or the HSB color space consisting of hue (H), saturation (S), and brightness (B). In this embodiment, an example using a color image will be described. However, a luminance image can be used because it is sufficient if it has at least luminance (Y).

  In addition, the imaging devices 110 are arranged in a substantially horizontal direction so that the optical axes of the two imaging devices 110 are substantially parallel on the traveling direction side of the host vehicle 1. The imaging device 110 continuously generates a color image obtained by capturing an object existing in the detection area in front of the host vehicle 1, for example, every 1/60 second frame (60 fps). Here, the object to be recognized and the specific object to be specified in association with the object are not only objects that exist three-dimensionally independently such as vehicles, pedestrians, bicycles, traffic lights, roads, guardrails, buildings, but also tail lamps. Also included are items that can be specified as part of specific items such as lighting parts of turn signals, turn signals, and traffic lights. One of the objects of the present embodiment is to specify a target image (object image) included in a color image as a specific object. Each functional unit in the following embodiments performs each process for each frame triggered by such acquisition (update) of a color image.

  The vehicle exterior environment recognition device 120 acquires color images from each of the two imaging devices 110, derives parallax using so-called pattern matching, and associates the derived parallax information (corresponding to a relative distance described later) with the color image. Generate a distance image. The color image and the distance image will be described in detail later. In addition, the vehicle environment recognition apparatus 120 uses the color value based on the color image and the relative distance z to the host vehicle 1 based on the distance image as a control target for the target image in the detection area in front of the host vehicle 1. A specific object or a special image that does not exist as a specific object is specified. This special image will be described in detail later.

  Further, when the outside environment recognition device 120 identifies a specific object, the specific object and the host vehicle 1 may collide with each other by deriving the relative speed of the specific object while tracking the specific object (for example, a preceding vehicle). Judgment is made whether or not the property is high. Here, when it is determined that the possibility of the collision is high, the outside environment recognition device 120 displays a warning (notification) to the driver through the display 122 installed in front of the driver, and the vehicle control device. Information indicating that is output to 130.

  The vehicle control device 130 receives a driver's operation input through the steering wheel 132, the accelerator pedal 134, and the brake pedal 136, and controls the host vehicle 1 by transmitting it to the steering mechanism 142, the drive mechanism 144, and the brake mechanism 146. In addition, the vehicle control device 130 controls the drive mechanism 144 and the braking mechanism 146 in accordance with instructions from the outside environment recognition device 120.

  Hereinafter, the configuration of the outside environment recognition device 120 will be described in detail. Here, a procedure for specifying a target image as a special image, which is characteristic of the present embodiment, will be described in detail, and a description of a configuration unrelated to the characteristics of the present embodiment will be omitted.

(Vehicle environment recognition device 120)
FIG. 2 is a functional block diagram showing a schematic function of the outside environment recognition device 120. As shown in FIG. 2, the vehicle exterior environment recognition device 120 includes an I / F unit 160, a data holding unit 162, and a central control unit 164.

  The I / F unit 160 is an interface for performing bidirectional information exchange with the imaging device 110 and the vehicle control device 130. The data holding unit 162 includes a RAM, a flash memory, an HDD, and the like. The data holding unit 162 holds various information necessary for the processing of each function unit described below, and temporarily holds a color image received from the imaging device 110. To do.

  The central control unit 164 is configured by a semiconductor integrated circuit including a central processing unit (CPU), a ROM storing programs, a RAM as a work area, and the like, and through the system bus 166, an I / F unit 160, a data holding unit 162 and the like are controlled. In the present embodiment, the central control unit 164 also functions as the image acquisition unit 170, the image processing unit 172, the three-dimensional position derivation unit 174, the grouping unit 176, the special image determination unit 178, and the specific object specifying unit 180. .

(External vehicle environment recognition processing)
FIG. 3 is a flowchart showing the flow of the external environment recognition process. In the outside environment recognition processing, the image acquisition unit 170 acquires an image from the imaging device 110 (S200), the image processing unit 172 processes the acquired image (S202), and the three-dimensional position derivation unit 174 extracts the image from the image. The three-dimensional position is derived (S204), the grouping unit 176 generates a grouped target image based on the three-dimensional position (S206), and the special image determination unit 178 determines that the grouped target image is special. It is determined whether or not the image is an image (S208), and the specific object specifying unit 180 specifies a specific object (S210). Hereinafter, each process is explained in full detail.

(Image acquisition process S200)
The image acquisition unit 170 acquires two color images captured by the two imaging devices 110. Here, of the two color images, one color image acquired from the imaging device 110 located on the right side toward the front of the host vehicle 1, which serves as a reference for pattern matching described later, is used as a reference image, and such a reference image is used. The other color image acquired from the imaging device 110 located on the left side toward the front of the host vehicle 1 is referred to as a comparison image.

(Image processing S202)
First, the image processing unit 172 extracts a reference block arbitrarily extracted from the reference image (for example, an array of 4 horizontal pixels × 4 vertical pixels). Then, the image processing unit 172 uses so-called pattern matching to extract a comparison block correlated with an arbitrary reference block from a predetermined range of the comparison image, and the positional relationship between the arbitrary reference block and the comparison block with respect to the image To generate disparity information (disparity) of the reference block. Here, “horizontal” indicates the horizontal direction of the screen, “vertical” indicates the vertical direction of the screen, and the predetermined range indicates a range where parallax can occur in the horizontal direction, for example, horizontal 128 pixels × vertical 4 pixels.

  As this pattern matching, it is conceivable to compare the luminance (Y color difference signal) between two color images in units of blocks indicating arbitrary image positions. For example, SAD (Sum of Absolute Difference) that takes a luminance difference, SSD (Sum of Squared Intensity Difference) that uses the difference squared, or NCC that takes the similarity of a variance value obtained by subtracting an average value from the luminance of each pixel There are methods such as (Normalized Cross Correlation). The image processing unit 172 performs such block-based parallax derivation processing for all blocks displayed in the detection region (for example, horizontal 600 pixels × vertical 180 pixels). Here, the block is assumed to be horizontal 4 pixels × vertical 4 pixels, but the number of pixels in the block can be arbitrarily set. Hereinafter, a block that is a unit for deriving such parallax information is referred to as a parallax block.

  FIG. 4 is an explanatory diagram for explaining an example of pattern matching. Here, it is assumed that the image acquisition unit 170 acquires the reference image 212a illustrated in FIG. 4A and the comparison image 212b illustrated in FIG. In the reference image 212a, the road surface is wet due to the rain, and the light of the light sources (blue light) 214a and 216a of any two traffic lights arranged on the road is reflected by the road surface, and the respective light sources 214a, It is assumed that optical bands 218a and 220a forming bands extending in the vertical direction of the screen are generated with respect to 216a. Such light bands 218a and 220a appear remarkably when the brightness of the environment outside the vehicle is low (for example, at night). Similarly, in the comparative image 212b, light from the light sources 214b and 216b of the same two traffic lights is reflected on the road surface, and light bands 218b and 220b forming bands in the vertical direction of the screen with respect to the respective light sources 214b and 216b. Suppose that has occurred.

  The image processing unit 172 extracts, for example, a reference block that is a partial parallax block of the optical band 218a from the reference image 212a, and is a parallax block correlated with the reference block from a predetermined range of the comparison image 212b. Search for a comparison block. Then, in FIG. 4B, a comparison is made such that the vertical position is equal to the reference block of the reference image 212a that is part of the same optical band 218b at a position separated by parallax (assumed parallax described later) indicated by an arrow. A block is extracted. Similarly, the image processing unit 172 extracts some reference blocks of the optical band 220a from the reference image 212a, and searches for a comparison block correlated with the reference block from a predetermined range of the comparison image 212b. To do. Then, in FIG. 4B, a comparison block that is a part of the same optical band 220b and has the same vertical position as the reference block of the reference image 212a is extracted at positions separated by parallax indicated by arrows.

  In this way, the image processing unit 172 generates parallax information between the optical band 218a of the correlated reference image 212a and the optical band 218b of the comparative image 212b, and the grouping unit 176 described later uses the reference image in FIG. The target image 230 shown in 212a is specified. Similarly, the image processing unit 172 generates parallax information between the optical band 220a of the correlated reference image 212a and the optical band 220b of the comparative image 212b, and the grouping unit 176 described below uses FIG. The target image 232 shown in the reference image 212a is identified.

  However, such target images 230, 232 of the optical band are special images that do not actually exist as specific objects, and if such special images are used as specific objects for collision avoidance control or cruise control, There is a risk of affecting the stability of vehicle travel. Therefore, in the special image determination unit 178 described later, the target images 230 and 232 by the optical bands 218a, 218b, 220a, and 220b are set as special images, and are excluded from the control targets of the collision avoidance control and the cruise control.

  FIG. 5 is an explanatory diagram for explaining the determination of the special image. FIG. 5 shows the positional relationship between the relative distance and height of the target image 230 of the light source 214a and the light band 218a with respect to the host vehicle 1. Since the target image 230 of the light band 218a is obtained by reflecting the light of the light source 214a on the road surface, it appears so as to extend from the light source 214a as shown by a thick line in FIG. However, the object (object image) that actually exists is always located vertically above the road surface, whereas the object image 230 of the light band 218a is assumed to be located vertically below the road surface if it actually exists. Be recognized.

  Therefore, when the target image 230 is located vertically below the road surface, the special image determination unit 178 described later determines that the target image 230 is a special image and excludes it from the control target for collision avoidance control and cruise control.

  FIG. 6 is an explanatory diagram for explaining another example of pattern matching. As described above, the target image 230 is specified by the optical band 218a of the correlated reference image 212a and the optical band 218b of the comparative image 212b, and the optical band 220a of the correlated reference image 212a and the optical band 220b of the comparative image 212b. If the target image 232 is specified by the above, it is determined that both of the target images 230 and 232 exist vertically below the road surface, and are finally excluded from the control targets of the collision avoidance control and the cruise control. However, in a situation where the road surface is wet, as shown in FIG. 6A, when there are a plurality of light sources 214a and 216a having the same light emission mode, such as traffic lights, the light bands 218a and 218b reflected on the road surface, 220a and 220b may appear with similar color values and sizes.

  At this time, as described with reference to FIG. 4, with respect to the reference image 212a and the comparison image 212b acquired from the two imaging devices 110, there is a problem if the same optical bands 218a and 218b or the optical bands 220a and 220b are matched. There are cases where different optical bands, for example, the optical band 218a of the reference image 212a and the optical band 220b of the comparative image 212b are mismatched. Then, as shown in FIG. 6A and FIG. 6B, a mismatch amount is added to the assumed parallax that is the original parallax (the parallax between the optical band 218a of the reference image 212a and the optical band 218b of the comparative image 212b). Of the estimated mismatch distance (the distance between the optical band 218b of the comparative image 212b and the optical band 220b of the comparative image 212b) is added, and the parallax information is derived. Then, as shown in FIG. 6C, there is a case where the target image 234 in the optical band 218a is erroneously recognized as being positioned on the near side as viewed from the own vehicle 1 from the position of the target image 230 in the original optical band 218a. is there.

  FIG. 7 is an explanatory diagram for explaining the determination of the special image. FIG. 7 shows the positional relationship between the relative distance and the height of the target image 234 of the light source 214a and the light band 218a with respect to the host vehicle 1. Since the target image 234 of the optical band 218a is originally the light of the light source 214a reflected on the road surface, the parallax information is assumed to be parallax, and appears to extend from the light source 214a as shown by a thick broken line in FIG. Should do. However, as described with reference to FIG. 6, when the optical band 218a of the reference image 212a and the optical band 220b of the comparative image 212b are mismatched, the disparity information becomes larger than the assumed disparity by the assumed mismatch distance, The object image 234 in the optical band 218a is misrecognized as being located on the near side as viewed from the own vehicle 1 with respect to the position of the target image 230 in the original optical band 218a, and as shown by a bold solid line in FIG. From the relationship, it may be recognized that it is located vertically above the road surface. In this case, it is determined that the target image 234 (light band) is not positioned vertically below the road surface, and as described above, it cannot be excluded from the control target for collision avoidance control and cruise control as a special image.

  Therefore, in the present embodiment, the image processing unit 172 once derives parallax information by pattern matching regardless of the presence or absence of mismatching, and generates a distance image to be described later. Then, the target image 234 once grouped by the grouping unit 176 described later is determined under other conditions, so that the target image 234 is specified as a special image and excluded from the control target for collision avoidance control and cruise control. . In this way, it is possible to avoid the specific object of the target image 234 of the optical band 218a due to mismatching and improve the specific accuracy of the specific object.

  FIG. 8 is an explanatory diagram for explaining the color image 212 and the distance image 240. For example, it is assumed that two color images 212 are generated for the detection region 242 through the two imaging devices 110. However, only the color image (reference image 212a) acquired from the imaging device 110 located on the right side is shown here for convenience of explanation. In the present embodiment, the image processing unit 172 obtains the parallax for each block from the color image 212 (the reference image 212a and the comparison image 212b), and forms the distance image 240 as illustrated in FIG. Each parallax block in the distance image 240 is associated with the parallax of the parallax block. Here, for convenience of explanation, the parallax block from which the parallax is derived is represented by black dots.

(Three-dimensional position derivation process S204)
Returning to FIG. 3, the three-dimensional position deriving unit 174 uses the so-called stereo method to calculate disparity information for each disparity block in the detection region 242 based on the distance image 240 generated by the image processing unit 172. It is converted into a three-dimensional position in the real space including the horizontal distance x, the height y, and the relative distance z. Here, the stereo method is a method of deriving a relative distance z with respect to the imaging device 110 of the target part from the parallax in the distance image 240 of the target part (a pixel or a block composed of a plurality of pixels) by using a triangulation method. is there. At this time, the three-dimensional position deriving unit 174 obtains the relative distance z of the target part, and based on the detected distance on the distance image 240 between the point on the road surface at the same relative distance z as the target part and the target part. The height y of the target part from the road surface is derived. Then, the derived three-dimensional position is associated with the distance image 240 again. Since various known techniques can be applied to the relative distance z deriving process and the three-dimensional position specifying process, the description thereof is omitted here.

(Grouping process S206)
The grouping unit 176 groups target parts having a three-dimensional position difference within a predetermined range to obtain a target image. Specifically, the grouping unit 176 compares the target portions in the distance image 240 in which the difference in the horizontal distance x, the difference in the height y, and the difference in the relative distance z are within a predetermined range (for example, 0.1 m). Are grouped on the assumption that they correspond to the same specific object. Thus, a target image that is a virtual target region group is generated. The above range is represented by a distance in real space, and can be set to an arbitrary value by a manufacturer or a passenger. Further, the grouping unit 176 also sets the difference in the horizontal distance x, the difference in the height y, and the difference in the relative distance z within a predetermined range with respect to the target part newly added by the grouping. A certain target part is further grouped. As a result, all target parts that can be assumed to be the same specific object are grouped. Here, it is assumed that the target image 234 of the optical band 218a in which the optical band 218a of the reference image 212a and the optical band 220b of the comparative image 212b are mismatched as illustrated in FIG. 6C is generated.

Here, the horizontal distance x difference, the height y difference, and the relative distance z difference are determined independently, and only when all are included in a predetermined range, the same group is used. You can also. For example, the root mean square of the difference in horizontal distance x, the difference in height y, and the difference in relative distance z ((difference in horizontal distance x) ² + (difference in height y) ² + (difference in relative distance z) ² ) Are included in the predetermined range, the same group may be used. With this calculation, an accurate distance between the target parts in the real space can be derived, so that the grouping accuracy can be improved.

(Special image determination processing S208)
FIG. 9 is a flowchart showing the flow of the special image determination process S208. The special image determination unit 178 determines that a target image satisfying a predetermined condition among the target images is a special image. Specifically, the road surface determination process (S208-1) is satisfied, or the road surface state determination process (S208-2), the light source presence / absence determination process (S208-3), and the edge direction determination process (S208-). 4) If all the conditions of the light source position determination process (S208-5), the light source mismatch determination process (S208-6), and the target image mismatch determination process (S208-7) are satisfied, the target image is determined as a special image ( S208-8). Then, the special image determination unit 178 determines whether or not the processing from step S208-1 to step S208-7 has been completed for all target images grouped by the grouping unit 176 (S208-9). If not completed (NO in S208-9), the next target image is prepared (S208-10), and the processing from the road surface determination process (S208-1) is repeated. If completed (YES in S208-9), the special image determination process S208 is terminated.

  Here, in addition to the conditions of the road surface lower determination process (S208-1), the road surface state determination process (S208-2), the light source presence / absence determination process (S208-3), the edge direction determination process (S208-4), the light source An example in which the target image is determined to be a special image when all six conditions of the position determination process (S208-5), the light source mismatch determination process (S208-6), and the target image mismatch determination process (S208-7) are satisfied. However, the target image may be determined as a special image when one or more conditions selected from these are satisfied, or when other conditions are satisfied in addition to such conditions. Hereinafter, road surface down determination processing (S208-1), road surface state determination processing (S208-2), light source presence / absence determination processing (S208-3), edge direction determination processing (S208-4), and light source position determination processing (S208). -5), the seven conditions of the light source mismatch determination process (S208-6) and the target image mismatch determination process (S208-7) will be described in that order.

(Road surface determination processing S208-1)
The special image determination unit 178 first acquires the three-dimensional position of the target image, and determines whether or not the target image is positioned vertically below the road surface. As a result, when it is located vertically below the road surface, the target image is determined as a special image (S208-8), and when it is located vertically above the road surface, the process proceeds to road surface state determination processing S208-2.

(Road surface state determination processing S208-2)
Then, the special image determination unit 178 determines whether the environment outside the vehicle is an environment where the road surface reflects the light source. As a result, if the road surface reflects the light source, the process proceeds to the light source presence / absence determination process S208-3. If the road surface does not reflect the light source, the process proceeds to the completion determination process S208-9. Such a determination can be made by using various existing techniques such as a rainfall state by a rain sensor and a color value of the road surface by the outside environment recognition device 120.

(Light source presence / absence determination processing S208-3)
Subsequently, the special image determination unit 178 determines whether or not the target image itself includes a light source. As a result, if the target image itself does not include a light source, the process proceeds to edge direction determination processing S208-4. If the target image itself includes a light source, the process proceeds to completion determination processing S208-9.

(Edge direction determination processing S208-4)
Next, the special image determination unit 178 determines whether or not the ratio of the target image in which the edge direction perpendicular to the edge extension direction is included in a predetermined direction range that is regarded as the horizontal direction is equal to or greater than a predetermined threshold value. As a result, if it is equal to or greater than the predetermined threshold, the process proceeds to the light source position determination process S208-5. Hereinafter, this process will be described in detail.

  FIG. 10 is an explanatory diagram for explaining the edge direction determination processing S208-4. The special image determination unit 178 first derives the edge directions of all the blocks (2 horizontal pixels × 2 vertical pixels) of the target image 234. Here, the edge direction indicates a direction perpendicular to the extending direction of the edge.

  As shown in FIG. 10A, when an arbitrary block 250 of the target image 234 is enlarged, the luminance distribution as shown in FIG. 10B is obtained. Also, the luminance range is 0 to 255, and in FIG. 10B, it is assumed that the white fill is luminance “200” and the black fill is luminance “0”. Here, in FIG. 10B of the block 250, the luminance of the upper left pixel is A, the luminance of the upper right pixel is B, the luminance of the lower left pixel is C, the luminance of the lower right pixel is D, and the horizontal direction of the edge direction The component is defined as (B + D) − (A + C), and the vertical component in the edge direction is defined as (A + B) − (C + D).

  Then, the horizontal component in the edge direction of the block 250 shown in FIG. 10B is (B + D) − (A + C) = (0 + 0) − (200 + 200) = − 400, and the vertical component in the edge direction is (A + B). )-(C + D) = (200 + 0)-(200 + 0) = 0. Therefore, the horizontal direction component is “−400” and the vertical direction component is “0”, and the edge direction is indicated by a horizontal leftward (negative) arrow as shown in FIG. However, as shown in FIG. 10D, the horizontal component is positive in the screen right direction, and the vertical component is positive in the screen upward direction.

  As described above, the configuration in which the other half area is subtracted from the half area in the block can remove the luminance offset and noise included in the entire block, and can appropriately extract the edge. . In addition, since the edge direction can be derived by simple calculation with only addition and subtraction, the calculation load can be reduced.

  The object of the present embodiment is to accumulate the edge directions derived in this way and derive the ratio. However, if the values derived from the horizontal direction component and the vertical direction component are used as they are and the edge direction is simply used, there are infinite variations in the edge direction. Then, the edge direction range that can be regarded as the same for the infinite variation must be set.

  Therefore, in this embodiment, both the horizontal direction component and the vertical direction component are defined by unit length, and the variation in the edge direction is simplified. That is, both the horizontal direction component and the vertical direction component are assumed to be either -1, 0, or +1. Then, the edge directions can be limited to eight directions each having an angle of 45 degrees as shown in FIG. For example, in the example of FIG. 10B described above, the edge direction is the direction “6” in FIG. By doing so, the calculation load of the special image determination unit 178 can be greatly reduced. However, the edge direction deriving unit is not limited to such a case, and various existing deriving units that can determine the appearance mode of the edge can be applied.

  The special image determination unit 178 derives such edge directions for all the blocks of the target image 234, and accumulates the number of the edge directions. As described above, the target image 234 is represented by a band reflected from the light source 214a, and its direction is close to the vertical direction. Accordingly, in the vicinity of the left edge of the optical band 218a in the horizontal direction, the directions “1” to “3” in FIG. 10E are easily derived as edge directions, and in the vicinity of the right edge of the optical band 218a in the horizontal direction. The directions “5” to “7” in FIG. 10E are easily derived as the edge directions.

  Therefore, if the ratio of the edge direction in the target image 234 included in the predetermined direction range is equal to or greater than the predetermined threshold, the special image determination unit 178, for example, “1” to “3” and “ 5 ”to“ 7 ”direction (direction range is 22.5 ° to 157.5 °, −22.5 ° to −157.5 ° from the vertical upper side with the clockwise direction being positive) is 80% or more. Judge whether there is.

(Light source position determination processing S208-5)
FIG. 11 is an explanatory diagram for explaining the light source position determination processing S208-5, the light source mismatch determination processing S208-6, and the target image mismatch determination processing S208-7. 11A shows an enlarged view of the reference image 212a shown in FIG. 6C, and FIGS. 11B to 11D show enlarged views of portions surrounded by circles in FIG. 11A, respectively. .

  Subsequently, the special image determination unit 178 determines whether or not there is a target image indicating a light source vertically above the target image 234 in the reference image 212a. As a result, as shown in FIG. 11B, if there is a target image indicating the light source 214a vertically above the target image 234, the process proceeds to the light source mismatch determination processing S208-6, and the light source is positioned vertically above the target image 234. If there is no target image indicating 214a, the process proceeds to the completion determination process S208-9. Here, when it is assumed that the target image 234 is reflected on the road surface, it is determined whether or not the light source 214a that is the reflection source is present. Further, if there is a target image indicating the light source 214a vertically above the target image 234, the special image determination unit 178 holds the distance between the target image 234 and the light source 214a for the target image mismatch determination processing S208-7 described later. To do.

(Light source mismatch determination processing S208-6)
Next, the special image determination unit 178 acquires the parallax information of the target image 234 and derives an assumed mismatch distance by subtracting the assumed parallax from the parallax information. Here, as described with reference to FIG. 6, the assumed mismatch distance is derived from the disparity information of the target image 234 in the optical band 218a as well as the disparity for the mismatch (assumed mismatch distance). This is because the assumed parallax, which is the original parallax, is included. Therefore, light source mismatch determination processing S208-6 and target image mismatch determination processing S208-7 described later are performed using the assumed mismatch distance obtained by subtracting the assumed parallax from the parallax information of the target image 234. Further, as the assumed parallax, a value that is the original parallax of the optical band 218a is referred to. For example, the relative distance z of the light band 218a is substantially equal to the light source 214a. Therefore, here, the parallax of the light source 214a is used as the assumed parallax.

  The reason for using the light source 214a is as follows. In an environment where the road surface is wet, the light band 218a has a band shape, but its width varies depending on the vertical position, and there may be a portion that does not form a band at all (width = 0). Therefore, there is a risk of matching with a similar adjacent optical band by skipping a portion of the optical band that should originally be matched. In contrast, the light source 214a is stable in luminance and edge, and is unlikely to cause mismatching. Therefore, the parallax of the light source 214a can be used as the assumed parallax. However, it is sufficient for the assumed parallax to be an estimated value of the original parallax of the optical band 218a, and not only the parallax of the light source 214a but also various probable values can be referred to.

  Then, the special image determination unit 178 starts from the light source 214a vertically above the target image 234 specified in the light source position determination processing S208-5 in the reference image 212a, and is equivalent to the assumed mismatch distance of the target image 234 in a predetermined horizontal direction. It is determined whether another light source 216a correlated with the light source 214a exists at the shifted position. As a result, as shown in FIG. 11C, if another light source 216a exists, the process proceeds to the target image mismatch determination process S208-7. If no other light source 216a exists, the completion determination process S208- The processing is moved to 9.

  The erroneous pattern matching described above, that is, the mismatch between the light band 218a of the reference image 212a and the light band 220b of the comparison image 212b is shifted by the assumed mismatch distance of the target image 234 in the predetermined horizontal direction of the light source 214a as the reflection source. This is because the light source 216a having the same light emission mode exists at the position. Therefore, here, if the light source 216a exists at a position shifted from the light source 214a by the assumed mismatch distance of the target image 234 in the predetermined horizontal direction, it can be determined that there is a possibility of mismatch. The predetermined horizontal direction is, for example, the horizontal right direction when the reference image 212a is a color image captured by the right imaging device 110, and the reference image 212a is a color image captured by the left imaging device 110. If there is, it is in the horizontal left direction.

(Target image mismatch determination processing S208-7)
Subsequently, the special image determination unit 178 acquires the distance between the target image 234 held in the light source position determination process S208-5 and the light source 214a, and the light source specified in the light source mismatch determination process S208-6 in the reference image 212a. It is determined whether or not another light band 220a correlated with the target image 234 exists at a position shifted vertically from the other light source 216a correlated with the target image 234 by the distance between the target image 234 and the light source 214a. As a result, as shown in FIG. 11D, if another light band 220a exists, that is, road surface state determination processing (S208-2), light source presence / absence determination processing (S208-3), edge direction determination processing. (S208-4), the light source position determination process (S208-5), the light source mismatch determination process (S208-6), and the target image mismatch determination process (S208-7) all satisfy the conditions, and the target image 234 is obtained. The special image is determined (S208-8). If there is no other optical band 220a, the process proceeds to the completion determination process S208-9.

  The erroneous pattern matching described above, that is, the mismatch between the optical band 218a of the reference image 212a and the optical band 220b of the comparative image 212b, is shifted to the position shifted by the assumed mismatch distance of the target image 234 in the predetermined horizontal direction of the target image 234. This results from the presence of the light band 220a having the same color value and size. Therefore, here, light is emitted from the light source 216a as a starting point to a position vertically shifted by a distance between the target image 234 and the light source 214a (a position shifted by an assumed mismatch distance of the target image 234 in a predetermined horizontal direction of the target image 234). If the band 220a exists, it can be determined that there is a possibility of mismatching.

  In this special image determination processing S208, since the presence / absence of mismatching is determined based only on the positional relationship between the light source and the light band in the reference image 212a without using the comparison image 212b, the processing load can be greatly reduced. it can.

  Further, since the search position of the light source 216a and the optical band 220a with respect to the light source 214a and the target image 234 can be grasped in advance as being the estimated mismatch distance of the target image 234, it is not necessary to search a wide range unnecessarily, and the processing load is further increased. Mitigation can be achieved.

  Furthermore, since a plurality of images such as the light source 214a and the light source 216a and the target image 234 and the light band 220a are comprehensively determined here, the correlation can be obtained with high accuracy, and the special image can be specified. The accuracy can be increased.

(Specific object identification process S210)
The specific object specifying unit 180 excludes the target image determined by the special image determining unit 178 as a special image from one or more target images grouped by the grouping unit 176. The specific object specifying unit 180 specifies a specific object such as a vehicle, a person, or a bicycle based on one or more parameters such as the size of the remaining target image. Since various existing techniques can be applied as a technique for identifying a specific object from the target image, detailed description thereof is omitted here.

  Here, by excluding a special image that does not exist in front of the host vehicle 1, the accuracy of specifying a specific object can be improved, and the special image can be prevented from becoming a control target for collision avoidance control or cruise control. Stability can be improved.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to this embodiment. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Is done.

  For example, in the above-described embodiment, the correlation between the light source 214a and the light source 216a, and both the target image 234 and the light band 220a is comprehensively determined. As long as the correlation between the target image 234 and the light band 220a can be determined, it is possible to exclude a special image that does not exist in front of the host vehicle 1 and improve the accuracy of specifying a specific object. Therefore, the special image can be excluded without the light sources 214a and 216a. In this case, the parallax of the light source 214a cannot be used as the assumed parallax, but the parallax of the specific object other than the special image, for example, a lattice structure arranged at equal intervals, which makes the relative distance z equal to the special image is assumed. It will be used as.

  It should be noted that each step of the vehicle environment recognition processing in the present specification does not necessarily have to be processed in time series in the order described in the flowchart, and may include processing in parallel or by a subroutine.

  INDUSTRIAL APPLICABILITY The present invention can be used for an external vehicle environment recognition device that identifies a target image existing around the host vehicle.

DESCRIPTION OF SYMBOLS 1 Self-vehicle 100 Environment recognition system 110 Imaging device 120 Outside-vehicle environment recognition device 170 Image acquisition part 172 Image processing part 174 Three-dimensional position derivation part 176 Grouping part 178 Special image determination part 180 Specific object identification part 212a Reference image 212b Comparison image 214a , 214b, 216a, 216b Light source 218a, 218b, 220a, 220b Optical band 234 Object image

Claims (6)

An image acquisition unit that acquires a reference image captured by two image capturing devices at different positions and a comparison image;
An image processing unit that extracts a comparison block correlated with an arbitrary reference block in the reference image from a predetermined range of the comparison image by pattern matching, and generates disparity information of the reference block;
A three-dimensional position deriving unit for deriving a three-dimensional position of a target part in the reference image using the parallax information;
A grouping unit for grouping target portions in which a difference in the three-dimensional position is within a predetermined range to be a target image;
In the reference image, there is a target image indicating a light source vertically above an arbitrary target image, and the parallax information of the arbitrary target image is assumed in a predetermined horizontal direction from the arbitrary target image and the light source. Special image determination in which, when there is a target image and a light source correlated with the arbitrary target image and the light source at a position shifted by an assumed mismatch distance that is a difference from the parallax, the arbitrary target image is determined as a special image And
A vehicle exterior environment recognition device comprising:   The special image determination unit further determines the target image if the ratio of the edge direction perpendicular to the edge extension direction in the target image included in the predetermined direction range considered as the horizontal direction is equal to or greater than a predetermined threshold. The external environment recognition device according to claim 1, wherein the recognition device determines that the image is a special image.   3. The vehicle exterior environment recognition according to claim 1, wherein the special image determination unit further determines that the target image is a special image if the vehicle exterior environment is an environment in which a road surface reflects a light source. apparatus.   The outside of the vehicle according to any one of claims 1 to 3, wherein the special image determination unit further determines the target image as a special image if the target image itself does not include a light source. Environment recognition device. An image acquisition unit that acquires a reference image captured by two image capturing devices at different positions and a comparison image;
An image processing unit that extracts a comparison block correlated with an arbitrary reference block in the reference image from a predetermined range of the comparison image by pattern matching, and generates disparity information;
A three-dimensional position deriving unit for deriving a three-dimensional position of a target part in the reference image using the parallax information;
A grouping unit for grouping target portions in which a difference in the three-dimensional position is within a predetermined range to be a target image;
In the reference image, starting from an arbitrary target image, the arbitrary target image is shifted to a position in the horizontal predetermined direction by an assumed mismatch distance that is a difference between the parallax information of the arbitrary target image and the assumed parallax. A special image determination unit that determines that the arbitrary target image is a special image,
A vehicle exterior environment recognition device comprising:   Of the one or more target images grouped by the grouping unit, the target image determined by the special image determination unit to be the special image is excluded, and a specific object specification is specified from the remaining target images The outside environment recognition device according to claim 1, further comprising a unit.

Images