JP2014044521A - Device and method for recognizing vehicle exterior environment - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve precision of detection of a moving body even when an own vehicle turns.SOLUTION: A device 120 for recognizing a vehicle exterior environment includes: an image data acquisition part which acquires image data; an image segmentation part 160 which segments a partial image; a flow vector derivation part 162 which derives a flow vector; a moving state acquisition part 164 which acquires a moving state; a correction flow vector generation part 166 which assumes two stationary bodies at two distances different from each other from a wide-angle lens, estimates flow vectors of the two stationary bodies based upon the moving state, and generates one correction flow vector; a flow vector subtraction part 168 which subtracts the correction flow vector from the derived flow vectors; and a grouping part 170 which groups flow vectors showing a moving body among the subtracted flow vectors based upon a criterion line 204 in the horizontal center of the image data.

Description

  The present invention relates to a vehicle environment recognition apparatus and a vehicle environment recognition method for recognizing an environment outside a host vehicle.

  Conventionally, there has been a technology for mounting a wide-angle (wide-angle) camera in front of the host vehicle and displaying the environmental image outside the host vehicle captured by the wide-angle camera on the display in the vehicle interior to assist the driver with visual confirmation. Are known. In such a technique, for example, it is detected that the host vehicle is temporarily stopped through the control information in the vehicle, and the display of the monitor device is automatically switched to the environmental image captured by the wide-angle camera. ing.

  In order to detect an object such as a vehicle or an obstacle located in front of the host vehicle from an environmental image through such a wide-angle camera, an image processing technique such as an optical flow has been studied. Here, the optical flow is represented by a two-dimensional velocity vector field on the image, that is, a velocity field accompanying movement of a moving object on the image. In such a technique, for example, a point that can be recognized as the same part is set as a feature point between two images consecutive in the time direction imaged in a predetermined cycle, and the movement of the feature point is a vector (hereinafter referred to as a feature point). , Referred to as a flow vector). Then, by calculating this flow vector in all areas in the captured image, for example, information such as the position and moving direction of the moving object in the image is detected based only on the presence or absence of the flow vector when the host vehicle is stopped. can do.

  In the optical flow, the approach of a moving object can be suitably detected when the host vehicle is stopped as described above, but a technique for detecting a moving object while traveling is also being studied. For example, while traveling, flow vectors are generated from environmental images captured with a wide-angle camera, each is classified into a plurality of groups, and the apparent flow vector generated by the movement of the host vehicle is subtracted to flow toward the center of the image. A technique for specifying a vector group as a moving object is known (for example, Patent Document 1).

  However, in the optical flow, when the host vehicle turns, a flow vector in the direction toward the center is also generated in the stationary object in the imaging range, and the stationary object may be erroneously determined as a moving object. Therefore, a technique for avoiding erroneous determination is disclosed by not determining the moving object in the area corresponding to the inside of the turn when the host vehicle turns (for example, Patent Document 2).

Japanese Patent No. 3239521 Japanese Patent No. 4193740

  As described above, in the technique of Patent Document 1, when the host vehicle turns, a stationary object may be erroneously determined as a moving object. Further, in the technique of Patent Document 2, erroneous determination can be avoided by excluding such a region as a detection target in advance, but if a moving object exists in that region, the moving object is detected. This could not be done and could increase the risk of collision.

  Moreover, in the technique of Patent Document 1, in order to appropriately determine the moving object, the size and direction of the flow vector generated in the stationary object due to the movement of the host vehicle are derived. Not only the moving speed and angular speed of the host vehicle but also the distance from the host vehicle to a stationary object is required. Such a distance to a stationary object can be derived by performing a calculation based on a three-point positioning method using a stereo camera or using a laser radar, but the cost for providing such a device increases. In addition, there is a problem that the calculation becomes complicated and the processing load increases.

  In view of such a problem, the present invention aims to improve the detection accuracy of a moving object even when the host vehicle turns without providing a special device for calculating the distance between the host vehicle and a stationary object. It is an object of the present invention to provide a vehicle exterior environment recognition device and a vehicle exterior environment recognition method.

  In order to solve the above-described problems, an external environment recognition apparatus according to the present invention includes an image data acquisition unit that acquires image data obtained by imaging the external environment through a wide-angle lens, and a position and size that are determined in advance from an image based on the image data. An image cutout unit that cuts out the partial image, a flow vector derivation unit that derives a flow vector at an arbitrary part of the partial image, a movement state acquisition unit that acquires a movement state of the wide-angle lens, and two different types from the wide-angle lens Assuming two stationary objects at a distance, the flow vectors of the two stationary objects are estimated based on the moving state, and a correction flow vector generation unit that generates one correction flow vector, and a correction flow from the derived flow vector Based on the flow vector subtraction unit that subtracts the vector and the judgment reference line at the center in the horizontal direction of the image data, Characterized in that it and a grouping unit for grouping the flow vector indicating the moving object out of the vector.

  The corrected flow vector generation unit corrects a flow vector having a relatively large scalar amount toward the determination reference line or a small scalar amount toward the determination reference line out of the two estimated flow vectors. It is good.

  Of the two estimated flow vectors, the corrected flow vector generation unit uses the longest rightward flow or the shortest leftward flow vector as the corrected flow vector in the left area of the determination reference line, and the left longest or right direction in the right area of the determination reference line. The shortest flow vector may be used as the correction flow vector.

  The movement state acquisition unit may acquire the movement speed and angular velocity of the wide-angle lens, and the correction flow vector generation unit may estimate the flow vector based on the movement speed and angular velocity.

  In order to solve the above-described problems, an external environment recognition method of the present invention acquires image data obtained by capturing an external environment through a wide-angle lens, and cuts out a partial image having a predetermined position and size from an image based on the image data. Deriving the flow vector at an arbitrary part of the partial image, obtaining the moving state of the wide-angle lens, assuming two stationary objects at two different distances from the wide-angle lens, and based on the moving state, two stationary objects The flow vector is estimated, a corrected flow vector is generated, the corrected flow vector is subtracted from the derived flow vector, and the subtracted flow vector is moved based on the determination reference line at the horizontal center of the image data. It is characterized by grouping flow vectors indicating objects.

  According to the present invention, even if the host vehicle turns, even if the host vehicle turns, by avoiding an increase in cost and processing load by not providing a special device for calculating the distance between the host vehicle and a stationary object. The detection accuracy can be improved.

It is the block diagram which showed the connection relation of the environment recognition system. It is explanatory drawing for demonstrating the image produced | generated with an imaging device. It is explanatory drawing for demonstrating a partial image. It is explanatory drawing for demonstrating the determination process of the moving object by a flow vector. It is explanatory drawing for demonstrating the determination process of the moving object by a flow vector. It is the functional block diagram which showed the schematic function of the external environment recognition apparatus. It is explanatory drawing for demonstrating the process of a flow vector derivation | leading-out part. It is explanatory drawing for demonstrating the process of a flow vector derivation | leading-out part. It is explanatory drawing for demonstrating the influence of the moving speed when the own vehicle goes straight. It is explanatory drawing for demonstrating the influence of angular velocity when the own vehicle turns. It is explanatory drawing which showed transition of the flow vector. It is explanatory drawing which showed transition of the flow vector. It is explanatory drawing which showed the effect of the correction | amendment flow vector. It is explanatory drawing which showed transition of the flow vector. It is explanatory drawing which showed transition of the flow vector. It is the flowchart which showed the flow of the whole process of the environment recognition method outside a vehicle. It is a flowchart for demonstrating an image cutting-out process. It is a flowchart for demonstrating a flow vector derivation process. It is a flowchart for demonstrating a correction | amendment flow vector generation process. It is a flowchart for demonstrating a flow vector subtraction process. It is a flowchart for demonstrating a grouping process.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The dimensions, materials, and other specific numerical values shown in the embodiment are merely examples for facilitating understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted, and elements not directly related to the present invention are not illustrated. To do.

(Environment recognition system 100)
FIG. 1 is a block diagram showing a connection relationship of the environment recognition system 100. The environment recognition system 100 includes an imaging device 110, a vehicle exterior environment recognition device 120, and a vehicle control device (ECU: Engine Control Unit) 130 provided in the host vehicle 1.

  The imaging device 110 is configured to include a wide-angle (wide-angle) lens and an imaging device such as a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS), and in front of the host vehicle 1 from the horizontal. It is provided slightly downward. The imaging device 110 can capture an environment corresponding to the front of the host vehicle 1 and generate a color image or a monochrome image composed of three hues (R (red), G (green), and B (blue)). . Here, the wide-angle lens refers to a lens having a relatively wide angle of view (viewing angle) compared to a standard lens (for example, 120 degrees or more) or a relatively short focal length. For example, a fish-eye lens is used as a wide-angle lens, A hemispherical space centered on the imaging device 110 can be imaged. Note that the projection method of the imaging apparatus 110 according to the present embodiment uses a three-dimensional projection, but various existing projection methods such as a perspective projection can be employed. Here, an example in which a fish-eye lens is used as the wide-angle lens will be described, but various wide-angle lenses such as a conical (axicon) prism can also be used. The reason why the angle of view of the lens is set to 120 degrees or more is to appropriately grasp the situation of the orthogonal road when the host vehicle 1 stops on the stop line on the cross road or the T-shaped road.

  FIG. 2 is an explanatory diagram for explaining an image generated by the imaging apparatus 110. As can be understood with reference to FIG. 2, an image 200 (hereinafter referred to as “environment image”) 200 generated by the imaging device 110 has a monotonically increasing angle with respect to the distance from the center of the image through a circumferential fisheye lens. (3D projection method). Accordingly, it is possible to acquire a wide range of environment images 200 at a time while maintaining the resolution of the image in the traveling direction of the host vehicle 1.

  In addition, the imaging device 110 continuously generates image data obtained by converting the environment image 200 into data, for example, every 1/10 second (10 fps). Each functional unit in the following embodiment performs each process triggered by such update of image data. Therefore, each functional unit repeats the process in units of frames.

  The outside environment recognition device 120 acquires image data indicating the environment image 200 from the imaging device 110, and derives a flow vector based on the optical flow using so-called pattern matching. Since the flow vector is easily specified at the edge portion of the image (the portion where the difference in brightness between adjacent pixels is large), the block from which the flow vector is derived often has an edge in the environment image 200 as well. However, the vehicle environment recognition apparatus 120 derives a flow vector using a partial image obtained by cutting out only a necessary portion in the environmental image 200.

  FIG. 3 is an explanatory diagram for explaining the partial image 202. As described above, in the environment image 200 shown in FIG. 2, an image with a wide angle of view can be acquired at a time, so that the moving object approaches not only in front of the host vehicle 1 but also in either the left or right direction (horizontal direction). Can also be specified. However, in the environment image 200, since the distance from the center of the image and the angle are approximately proportional, the image is distorted as the distance from the center increases. For example, in the area close to the four corners in the environment image 200 of FIG. 2, an object that actually extends horizontally is displayed curved in the environment image 200, and its end portion also extends in the vertical direction. Visible to. Here, “horizontal” indicates the horizontal direction of the captured environment image 200 and corresponds to the horizontal direction in real space. “Vertical” indicates the vertical direction of the captured environment image 200 and corresponds to the vertical direction in real space.

  Therefore, the outside environment recognition apparatus 120 includes a region indicated by a broken line in FIG. 2 where there is a moving object to be specified in the traffic environment in the environment image 200 shown in FIG. A partial image 202 shown in FIG. 3 is cut out by combining the central region and the left and right regions in the horizontal direction. Then, a flow vector based on the optical flow is derived for the partial image 202.

  Subsequently, the vehicle environment recognition apparatus 120 groups the approximate flow vectors to identify the moving object, and determines the type of the moving object based on the size of the moving object and its speed (the magnitude of the flow vector). To do. Here, the moving object is an object that moves independently, such as a vehicle, a bicycle, or a pedestrian. In addition, stationary objects corresponding to moving objects indicate objects such as buildings, roads, guardrails, and background spaces. Further, the outside environment recognition apparatus 120 includes a speed sensor 124 that detects the moving speed of the host vehicle 1 and an angular speed sensor 126 that detects an angular speed centered on the vertical axis when the host vehicle 1 turns as shown in FIG. The flow vector of the stationary object is estimated through

  For example, the outside environment recognition device 120 recognizes a moving object moving at high speed in the horizontal direction as a preceding vehicle. Moreover, the vehicle exterior environment recognition apparatus 120 can determine whether or not there is a high possibility of collision with the identified moving object. Here, when it is determined that the possibility of a collision is high, the vehicle exterior environment recognition device 120 notifies the driver of the fact through the display 122 and outputs information indicating the fact to the vehicle control device 130. . The processing of the outside environment recognition device 120 will be described in detail later.

  The vehicle control device 130 avoids a collision with a moving object specified by the outside environment recognition device 120. Specifically, the vehicle control device 130 acquires the current traveling state of the host vehicle 1 through the steering angle sensor 132 that detects the steering angle, the speed sensor 124 described above, and the like, and controls the actuator 134 to determine the preceding vehicle. Keep the distance between cars safe. Here, the actuator 134 is a vehicle control actuator used for controlling a brake, a throttle valve, a steering angle, and the like. Further, when a collision with a moving object is assumed, the vehicle control device 130 controls the actuator 134 to automatically brake the host vehicle 1. Such a vehicle control device 130 can also be formed integrally with the outside environment recognition device 120.

(Moving object judgment process by flow vector)
4 and 5 are explanatory diagrams for explaining the moving object determination processing based on the optical flow flow vector according to the present embodiment. Here, the features of the present embodiment will be described in comparison with conventional problems.

  If the flow vector is derived with respect to the partial image 202 shown in FIG. 3 when the host vehicle 1 is traveling straight without turning as shown in FIG. It is indicated by a plurality of arrows as in (b). The length of the arrow indicates the size of the flow vector, and the direction of the arrow indicates the direction of the flow vector. The number of flow vectors is essentially the same as the number of blocks that are derived units, but here, the number of flow vectors is limited to facilitate understanding.

  Since the flow vector is generated by the movement of the feature point in the image, as can be understood with reference to FIG. 4, as the vehicle 1 moves, the flow vector is applied not only to the moving object 2 but also to the stationary object 3. Occurs. In this embodiment, since a wide-angle lens is used, it can be seen that the flow vector of the stationary object 3 increases in size as it shifts to the side as compared with the center of the partial image 202. .

  The part (boundary) on the partial image 202 where the horizontal direction of the flow vector of the stationary object 3 is reversed left and right is on the line segment indicated by the dashed line extending vertically as shown in FIG. Appear. In the present embodiment, such a line segment is set as a determination reference line 204 used as a determination reference for the moving object 2. The determination reference line 204 is as shown in FIG. 4B when the imaging device 110 is positioned at the front end of the host vehicle 1 and at the center in the vehicle width direction and the imaging direction is set equal to the traveling direction. The partial image 202 is positioned at the center with respect to the horizontal direction. However, if the position and imaging direction of the imaging device 110 are different, the determination reference line 204 is not necessarily positioned at the center with respect to the horizontal direction of the partial image 202. In any case, a boundary line where the horizontal direction of the flow vector of the stationary object 3 generated when the host vehicle 1 goes straight may be set as the determination reference line 204.

  At this time, if the partial image 202 shown in FIG. 4B is divided into a left region and a right region based on the determination reference line 204, the flow vector of the stationary object 3 is generated in a direction away from the determination reference line 204. On the other hand, it can be seen that the flow vector of the moving object 2 approaching is generated in a direction approaching the determination reference line 204.

  Therefore, the horizontal region is divided into two in accordance with the judgment reference line 204, and the flow vector facing the right direction in the left region or the flow vector facing the left direction in the right region is detected, and these are detected as the moving object 2. Can be identified.

  However, as shown in FIG. 5A, when the host vehicle 1 travels with a turn, the flow vector is expressed as shown in FIG. 5B. Here, the flow vector of the stationary object 3 is generated in the direction approaching the determination reference line 204 even though the stationary object 3 in the region 206 indicated by the one-dot chain line exists in the right region of the determination reference line 204. This is because when the host vehicle 1 goes straight, the position where the horizontal direction of the flow vector is reversed on the left and right coincides with the vanishing point ahead (infinity), but when the host vehicle 1 turns, the host vehicle 1 moves. This is due to the fact that the reverse position changes in the horizontal direction depending on the velocity and angular velocity and the distance to the stationary object 3.

  Then, the flow vector in the area 206 indicated by the alternate long and short dash line is erroneously determined as the moving object 2 regardless of whether it is the moving object 2 or the stationary object 3. In this case, it is possible not to perform the determination of the moving object 2 in the area corresponding to the inside of the turn, that is, the right area, but the moving object 2 existing in the area cannot be detected. May increase risk.

  Therefore, in order to appropriately determine the moving object 2, first, the flow vector of the stationary object 3 is correctly estimated, and the flow vector is set as a corrected flow vector, and is subtracted from the flow vector detected on the partial image 202. By doing so, the flow vector generated in the stationary object 3 can be canceled (the length of the flow vector is 0), and only the flow vector of the moving object 2 can be left.

  However, in order to accurately estimate the magnitude and direction of the flow vector generated in the stationary object 3 due to the movement of the own vehicle 1, not only the moving speed and angular velocity of the own vehicle 1 but also the distance from the own vehicle 1 to the stationary object 3 Is also required. Such a distance to the stationary object 3 can be derived by performing a calculation based on a three-point positioning method using a stereo camera or using a laser radar. However, the cost for providing such a device increases. In addition, the calculation becomes complicated and the processing load increases.

  Therefore, in the present embodiment, the distance to the stationary object 3 is not deliberately derived, and assuming that the maximum value and the minimum value of the distance to the stationary object 3 are all, the stationary objects 3 are all from the minimum value to the maximum value. Suppose that it exists in any position. Then, the fluctuation range of the flow vector that can occur in the stationary object 3 due to the turning of the host vehicle 1 is estimated from the moving speed and the angular velocity of the host vehicle 1, and the correction flow vector is appropriately selected and detected on the partial image 202. Subtract from the flow vector. By doing so, at least the flow vector of the stationary object 3 is not generated in the direction approaching the determination reference line 204 (error detection can be avoided), and all the flow vectors generated in the direction approaching the determination reference line 204 are all moving objects. 2 can be specified. In this embodiment, even if it is a case where the own vehicle 1 turns in this way, it aims at detecting the moving object 2 appropriately and aiming at the improvement of the detection accuracy.

  Hereinafter, the vehicle environment recognition apparatus 120 for realizing such processing will be specifically described, and then the vehicle environment recognition method will be described in detail.

(Vehicle environment recognition device 120)
FIG. 6 is a functional block diagram showing a schematic function of the outside environment recognition device 120. As shown in FIG. 6, the vehicle exterior environment recognition apparatus 120 includes an I / F unit 150, a data holding unit 152, and a central control unit 154.

  The I / F unit (image data acquisition unit) 150 is an interface for performing bidirectional information exchange with the imaging device 110 and the vehicle control device 130, and for example, acquires image data generated by the imaging device 110. . Further, the I / F unit 150 acquires the moving speed and acceleration from the speed sensor 124 and the acceleration sensor 126. The data holding unit 152 includes a RAM, a flash memory, an HDD, and the like, and holds various pieces of information necessary for processing of each functional unit described below. In the present embodiment, the image data is temporarily held until at least the next frame for pattern matching in the flow vector derivation process. Here, the image data in the previous frame stored in the data holding unit 152 is referred to as “previous image data”, and the image data in the current frame is referred to as “current image data”. In addition, a correction table, which will be described later, in which the moving speed and angular speed are associated with the correction flow vector can be held in advance.

  The central control unit 154 is configured by a semiconductor integrated circuit including a central processing unit (CPU), a ROM storing a program and the like, a RAM as a work area, and the like, and through the system bus 156, the I / F unit 150 and the data holding unit. 152 is controlled. In the present embodiment, the central control unit 154 includes an image cutout unit 160, a flow vector derivation unit 162, a movement state acquisition unit 164, a corrected flow vector generation unit 166, a flow vector subtraction unit 168, a grouping unit 170, a movement It also functions as an object determination unit 172, a collision determination unit 174, a notification unit 176, and an image holding unit 178.

  The image cutout unit 160 is a partial image as shown in FIG. 3 having a predetermined position and size from the environmental image 200 as shown in FIG. 2 based on the current image data acquired by the I / F unit 150. 202 is cut out.

  The flow vector deriving unit 162 derives a flow vector at an arbitrary part of the partial image 202 cut out by the image cutout unit 160 through so-called pattern matching.

  7 and 8 are explanatory diagrams for explaining the processing of the flow vector deriving unit 162. FIG. The flow vector deriving unit 162 is a block arbitrarily extracted from the partial image 202 shown in FIG. 7A based on the previous image data stored in the data holding unit 152 (here, an array of horizontal 8 pixels × vertical 8 pixels). A block 210b corresponding to (configured) 210a (high correlation value) is detected from the partial image 202 shown in FIG. 7B based on the current image data (pattern matching). At this time, the movement trajectory indicated by the arrow in FIG. 7B from the block 210a corresponding to the previous image data to the block 210b corresponding to the current image data becomes the flow vector in the optical flow.

  As such pattern matching, it is conceivable to compare luminance values in units of blocks indicating arbitrary image positions between previous image data and current image data. For example, the SAD (Sum of Absolute Difference) that takes the difference in luminance value, the SSD (Sum of Squared intensity Difference) that uses the difference squared, and the similarity of the variance value obtained by subtracting the average value from the luminance value of each pixel. There are methods such as NCC (Normalized Cross Correlation). The vehicle exterior environment recognition apparatus 120 performs such block-based flow vector derivation processing for all blocks displayed in the partial image 202 (for example, 640 pixels × 128 pixels).

  The number of pixels in the block can be arbitrarily set. Here, the block is assumed to be 8 horizontal pixels × 8 vertical pixels.

  In this way, the flow vector deriving unit 162 can derive a flow vector as shown in FIG. 8 for the moving object 2 corresponding to, for example, a vehicle in the partial image 202 corresponding to the previous image data.

  The movement state acquisition unit 164 acquires the movement state of the imaging device 110 having a wide-angle lens. Although the moving state of the imaging device 110 may be directly measured, in the present embodiment, the imaging device 110 is fixed, and the moving state of the host vehicle 1 that moves on the same movement trajectory as the imaging device 110 is measured. 110 is used as the moving state. Moreover, since the moving state can be expressed by the moving speed in the straight direction of the host vehicle 1 and the angular speed in the turning direction, the moving speed and the angular speed of the host vehicle 1 are acquired here as the moving state. However, the means for specifying the movement state is not limited to this, and can be specified based on various parameters such as the movement speed and the steering angle.

  The corrected flow vector generation unit 166 assumes two stationary objects 3 at two different distances from the wide-angle lens, and calculates the flow vectors of the two stationary objects 3 based on the moving state (here, moving speed and angular velocity). presume.

  Here, first, the reason for assuming two stationary objects 3 at two distances will be described, and the operation when a correction flow vector is generated from the flow vectors of the two stationary objects 3 will be described in detail.

(Reason for assuming two stationary objects 3 at two distances)
The flow vector of the stationary object 3 is affected by the moving speed at which the host vehicle 1 goes straight and the angular speed at which the vehicle turns.

  FIG. 9 is an explanatory diagram for explaining the influence of the moving speed when the host vehicle 1 goes straight, and FIG. 10 is an explanatory diagram for explaining the influence of the angular velocity when the host vehicle 1 turns. .

  When the host vehicle 1 is traveling straight at the moving speed A, the stationary object 3a that is relatively far from the host vehicle 1 and the stationary object 3b that is relatively close to the host vehicle 1 are at the relative speed A. Therefore, the relative movement at the relative speed A is projected onto the partial image 202 and becomes a flow vector. As shown in FIG. 9, although the stationary object 3 a and the stationary object 3 b have the same relative speed A, the distance is different, so that the length of the flow vector is different, and the position is relatively far from the host vehicle 1. The flow vector of the stationary object 3a is shorter than the flow vector of the stationary object 3b located relatively close to the host vehicle 1.

  In addition, the flow vector is directed so that the stationary object 3 is determined from the determination reference line 204 in the partial image 202 as shown in FIG. 9 when the imaging device 110 is positioned at the front end of the host vehicle 1 and at the center in the vehicle width direction. When it is on the left side, it turns to the left, and when the stationary object 3 is on the right side of the determination reference line 204, it turns to the right.

  On the other hand, as shown in FIG. 10, when the host vehicle 1 is turning at an angular velocity B, the stationary object 3 a that is relatively far from the host vehicle 1 and the stationary object 3 b that is relatively close to the host vehicle 1 are Thus, the vehicle is rotating relative to the host vehicle 1 at the relative angular velocity B, and the relative rotation of the relative angular velocity B is projected onto the partial image 202 to become a flow vector. Therefore, the stationary object 3a and the stationary object 3b generate the same flow vector with the same relative angular velocity B.

  Referring to FIGS. 9 and 10, the flow vector of the stationary object 3 due to the influence of the moving speed A at which the host vehicle 1 travels straight can be derived based on the moving speed A of the host vehicle 1 and the distance from the stationary object 3. Further, the flow vector of the stationary object 3 due to the influence of the angular velocity B at which the host vehicle 1 turns can be derived based on the angular velocity B of the host vehicle 1. However, as described above, the moving speed A and the angular speed B of the host vehicle 1 can be easily acquired by the moving state acquisition unit 164, but the distance to the stationary object 3 cannot be easily derived.

  Therefore, in this embodiment, the concept of the distance to the stationary object 3 is handled as follows. That is, it does not mention whether the stationary object 3 is between the assumed maximum distance and the minimum distance. Therefore, even if the stationary object 3 exists between the assumed maximum distance and the minimum distance, the influence of the movement of the host vehicle 1 on the flow vector of the stationary object 3, in particular, the determination reference line 204 Appearance of the flow vector of the stationary stationary object 3 is suppressed. Therefore, although there is a possibility that the moving object 2 having a low moving speed is not detected, it is possible to reliably avoid erroneous detection of the stationary object 3 as the moving object 2.

  11, 12, 14 and 15 are explanatory diagrams showing the transition of the flow vector, and FIG. 13 is an explanatory diagram showing the effect of the correction flow vector. First, the corrected flow vector generation unit 166 assumes two stationary objects 3 when the stationary object 3 is at the shortest distance assumed and when the stationary object 3 is at the longest distance assumed, and the moving state of the host vehicle 1 is assumed. Based on (moving speed and angular velocity), flow vectors of two stationary objects 3 are derived. For example, when the moving speed of the host vehicle 1 is 7 km / h and the angular speed is +15 deg / s (right turn), the correction flow vector generation unit 166 calculates in spatial coordinates based on the moving condition. Then, the flow vectors of the long-distance stationary object 3a and the short-distance stationary object 3b show transitions as shown in FIG.

  In FIG. 11, the horizontal position of the entire partial image 202 at the vertical position corresponding to the boundary between the ground and the stationary object 3 in the partial image 202 is taken as the horizontal axis, and the flow vector of each part is taken as the vertical axis. Here, the length of the flow vector is indicated by the size of the vertical axis, and the direction of the flow vector is indicated by positive in the right direction and negative in the left direction. The stationary object 3 performs the calculation as a planar wall that is orthogonal to the direction of the host vehicle 1 before turning and is at a predetermined distance in front of the vehicle. However, the definition of the stationary object 3 is not limited to this case, and left and right wall surfaces parallel to the direction of the host vehicle 1 may be added, or a circular wall that is equidistant from the imaging device 110 may be used.

  As can be understood with reference to FIG. 11, the long-distance stationary object 3a is less affected by the moving speed at which the host vehicle 1 travels straight, and is almost affected by the angular speed at which the host vehicle 1 turns. Negative (leftward) fixed value. On the other hand, the short-distance stationary object 3b, like the long-distance stationary object 3a, is influenced by the angular velocity at which the host vehicle 1 turns, and the flow vector is negatively offset. In response, a positive flow vector is obtained on the relatively right side in the horizontal direction, and a negative flow vector is obtained on the relatively left side in the horizontal direction. This is because, as described above, since the wide-angle lens is used, the magnitude of the flow vector of the stationary object 3 increases as it shifts to the side compared to the center of the partial image 202. .

  However, the flow vector of the short-distance stationary object 3b also converges to the same flow vector as that of the long-distance stationary object 3a at the left and right ends in the horizontal direction. As described above, this is because the stationary object 3 is assumed to be a planar wall that is orthogonal to the direction of the host vehicle 1 before turning and is at a predetermined distance in front of the vehicle. This is because the angle formed with the optical axis direction of the imaging device 110 is increased, the distance to the stationary object 3 is increased, and the flow vector is equal to the stationary object 3a at a long distance. Therefore, at the left and right ends in the horizontal direction, the influence of the moving speed at which the host vehicle 1 goes straight is also reduced by the short-distance stationary object 3b, and is almost affected by the angular speed at which the host vehicle 1 turns. In addition, when assuming that the stationary object 3 is not a flat wall but a spherical surface that is equidistant from the imaging device 110, the long-distance stationary object 3a and the short-distance stationary object 3b are the same at the left and right ends in the horizontal direction. It does not converge to the flow vector.

  In the above description, the flow vector of the long-distance stationary object 3a has been described as having a negative fixed value. However, the actual flow vector has a concave shape in the negative direction at the end of the horizontal position. This is because the moving distance of the stationary object 3 accompanying the turning is longer at the end of the partial image 202 than at the center due to the characteristics of using the stereoscopic projection by the wide-angle lens. This is because the vector is enlarged.

  Here, when observing FIG. 11 as a whole, when the stationary object 3 exists at a relatively short distance, a flow vector of the stationary object 3b at a short distance is generated, and the horizontal direction of the flow vector of the stationary object 3 is left and right. A part (boundary) that is reversed at is generated at a position indicated by a broken line in FIG. If the angular velocity in turning is greater than 15 deg / s, the reversal part shifts to the right in the horizontal direction. On the other hand, if the angular velocity is less than 15 deg / s, it shifts to the left in the horizontal direction and approaches the central determination reference line 204.

  Since all the stationary objects 3 that can exist in the partial image 202 exist between the assumed shortest distance to the longest distance, here, the short-distance stationary object 3b and the long-distance stationary object 3a. By considering only, all the stationary objects 3 in between are considered.

(Operation when a corrected flow vector is generated from the flow vectors of two stationary objects)
Next, the relationship between the stationary object 3 and the moving object 2 will be described. Referring to FIG. 12 (a) focusing only on the right region of FIG. 11, since the actual distance of the stationary object 3 is not mentioned here, the stationary object 3 is indicated by hatching in FIG. 12 (a). It exists anywhere in the area. The stationary object 3 existing in the hatched area becomes a flow vector in the left direction except for a part in the right area of the determination reference line 204, and the stationary object 3 is determined as the moving object 2. In order to avoid such erroneous detection, if all of the right side area is excluded as a detection target in advance, for example, the actually existing moving object 2a cannot be detected.

  Therefore, the maximum amount of the flow vector of the stationary object 3 that may be erroneously detected, that is, the flow vector having a relatively large scalar amount toward the determination reference line 204 is set as a correction flow vector, and the flow vector of FIG. Thus, the entire flow vector is offset in the right direction accordingly. By doing so, at least the flow vector of the stationary object 3 does not occur in a direction approaching the determination reference line 204.

  In addition, the moving object 2a having a size equal to or larger than the maximum amount of the flow vector of the stationary object 3 does not change the direction of the flow vector only by the offset of the correction flow vector (since the left direction is maintained), FIG. As shown in (b), it is grasped as a flow vector in the left direction and specified as the moving object 2a.

  For example, in the example of FIG. 13, as indicated by the alternate long and short dash line in FIG. 13A before the offset of the correction flow vector, in the right region of the determination reference line 204, although it is a stationary object 3, In FIG. 13B after the offset of the correction flow vector is performed, they also become the flow vector in the right direction, and the flow vector in the left direction moves to the left truly. Only the moving body 2a is present. In this way, it is possible to avoid erroneously detecting at least the stationary object 3 as the moving object 2 and to extract all the flow vectors in the left direction as the moving object 2.

  Further, referring to FIG. 14 (a) focusing only on the left region of FIG. 11, regardless of whether the object is a long-distance stationary object 3a or a short-distance stationary object 3b, the stationary object 3 is at least as shown in FIG. In the area indicated by hatching in (a), the flow vector in the left direction has a certain length in the left area of the determination reference line 204. Such movement of the host vehicle 1 affects not only the stationary object 3 but also the moving object 2. For example, even if the moving object 2b originally has a flow vector in the right direction, it is offset by the flow vector in the left direction by the movement of the host vehicle 1, and the moving object 2b also becomes a flow vector in the left direction. The moving object 2 will not be determined.

  Therefore, the minimum amount of the flow vector affected by the stationary object 3, that is, the flow vector having a relatively small scalar amount in the direction opposite to the determination reference line 204 (here, the flow vector of the long-distance stationary object 3a). As a corrected flow vector, as shown in FIG. 14B, the entire flow vector is offset to the right by that amount.

  Here, if the flow vector of the short distance stationary object 3b is adopted, the stationary object 3 between the short distance and the long distance is offset by the flow vector of the short distance stationary object 3b. The stationary object 3 between the short distance and the long distance becomes a flow vector in the right direction and is erroneously detected as the moving object 2. Therefore, the flow vector of the long distance stationary object 3a is adopted as described above.

  By doing so, it is possible to eliminate the offset of the flow vector in the left direction added to the entire left region, and to minimize the influence of the movement of the host vehicle 1.

  In this way, the moving object 2b whose flow vector has been inverted due to the movement of the host vehicle 1 also receives the minimum offset and becomes the right flow vector. As shown in FIG. It is grasped as a flow vector and specified as a moving object 2b.

  For example, in the example of FIG. 13, in FIG. 13A before the offset of the correction flow vector, the flow vector of the moving object 2b is in the left direction in the region on the left side of the determination reference line 204, and is specified as the moving object 2b. In FIG. 13B after the correction flow vector is offset, the flow vector of the moving object 2b is in the right direction, and it is specified as the moving body 2b that is truly moving in the right direction. Can do.

  As described above, the corrected flow vector generation unit 166 estimates the flow vectors of the two distances of the short distance and the long distance, and the scalar amount toward the determination reference line 204 is relatively larger among the two estimated flow vectors. A flow vector that is large or has a small scalar amount in the opposite direction to the determination reference line 204, that is, a longest rightward flow vector or a shortest leftward flow vector in the left region of the determination reference line 204, and a right flow region of the determination reference line 204 The longest leftward flow vector or the shortest rightward flow vector is set as a corrected flow vector.

  The correction flow vector derived in this way can be expressed as shown by a solid line in FIG. 15A when the left region and the right region are made continuous. Here, the flow vector of the stationary object 3 falls within the range indicated by hatching. When the flow vector of the entire partial image 202 is offset with such a corrected flow vector, the result is as shown in FIG. At this time, in the right region, the flow vectors of all the stationary objects 3 can be set to the rightward flow vectors, and it is possible to avoid erroneous detection of the stationary object 3 as the moving object 2 and to have a certain movement speed. The moving object 2 can be detected. Further, in the left region, the moving object that is offset by the range in which the flow vectors of all the stationary objects 3 are not directed rightward, that is, the range in which the stationary object 3 is not erroneously detected as the moving object 2 and is buried by the movement of the host vehicle 1. 2 flow vectors can be appropriately detected.

  However, since the correction flow vector is uniquely derived from the moving speed and the angular velocity of the host vehicle 1, it can be extracted from the correction table according to the moving speed and the angular velocity without calculating each time. In this case, a correction flow vector is calculated in advance based on the moving speed and the angular velocity, a correction table in which the moving speed, the angular velocity, and the correction flow vector are associated with each other is generated and held in the data holding unit 152. In such a correction table, correction flow vectors are prepared with a resolution of a movement speed of 1 km interval (range 1 to 15 km / h) and an angular velocity of 1 deg interval (range -30 to 30 deg / s). When the correction flow vector is derived with a resolution higher than the resolution, linear interpolation can be performed using a plurality of correction flow vectors. It is also possible to select the correction flow vector so that the stationary object 3 is not erroneously detected as the moving object 2 by subtracting the correction flow vector. Then, the correction flow vector generation unit 166 extracts a correction flow vector by referring to the correction table of the data holding unit 152 according to the movement speed and the angular velocity acquired by the movement state acquisition unit 164.

  The flow vector subtracting unit 168 subtracts the corrected flow vector generated by the corrected flow vector generating unit 166 from the flow vector derived by the flow vector deriving unit 162. In this way, the offset corresponding to the correction flow vector is performed, the stationary object 3 is not erroneously detected as the moving object 2, and the detection accuracy of the moving object 2 is improved.

  The grouping unit 170 divides the region into a left region and a right region based on the determination reference line 204 derived by the flow vector deriving unit 162, and the flow directed to the right in the left region from the flow vector subtracted by the flow vector subtracting unit 168. A vector or a flow vector directed to the left in the right region is extracted. Then, among the extracted flow vectors, the flow vectors of adjacent blocks are sequentially compared, and the direction of the flow vector and the size thereof are approximated (for example, the direction is within ± 45 degrees and the difference in size is within 16 pixels. Group the blocks. Then, the grouped block group is specified as the moving object 2. For example, since the flow vectors shown in FIG. 8 are directed to the left in the right region and are close to each other, the target blocks can be grouped and specified as the moving object 2.

  The moving object determination unit 172 determines the position, the traveling direction, and the position of the moving object 2 constituting the block group according to the ground contact position, size, and speed of the block group that the grouping unit 170 has grouped. Determine the speed. For example, the block group approximated by the flow vector shown in FIG. 8 is determined to be an object that moves in a direction orthogonal to the host vehicle 1.

  The collision determination unit 174 determines whether or not the own vehicle 1 and the moving object 2 are highly likely to collide according to the traveling direction and speed of the own vehicle 1 and the position, traveling direction and speed of the identified moving object 2. To determine. Since various existing techniques can be applied to the collision determination, detailed description thereof is omitted here.

  When the collision determination unit 174 determines that the possibility of a collision is high, the notification unit 176 notifies the driver to that effect through a buzzer (voice output means) and a warning light (display 122). In addition, information indicating that is output to the vehicle control device 130 through the I / F unit 150.

  The image holding unit 178 overwrites the previous image data of the data holding unit 152 with the current image data. Thus, in the next frame, the current image data can be used as the previous image data.

(External vehicle environment recognition method)
Hereinafter, specific processing of the vehicle exterior environment recognition device 120 will be described based on the flowcharts of FIGS. FIG. 16 shows an overall flow relating to the interrupt processing when image data picked up through the wide-angle lens is transmitted from the image pickup apparatus 110, and FIGS. 17 to 21 show individual subroutines therein. . In addition, here, pixels or blocks in the image are listed as the processing target parts.

  When the interruption by the vehicle exterior environment recognition method occurs, the I / F unit 150 acquires the current image data transmitted from the imaging device 110 (S300). The image cutout unit 160 then selects a partial image 202 (for example, horizontal 640 pixels × for example) having a predetermined position from the environmental image 200 (for example, horizontal 640 pixels × vertical 480 pixels) based on the acquired current image data. 128 vertical pixels: horizontal 80 blocks × vertical 16 blocks) are cut out (S302), and the flow vector deriving unit 162 derives a plurality of flow vectors in the partial image 202 (S304).

  Subsequently, the movement state acquisition unit 164 acquires the movement speed and angular velocity of the host vehicle 1 at that time (S306), and the correction flow vector generation unit 166 differs from the wide-angle lens based on the movement speed and angular velocity. The flow vectors of two stationary objects 3 at two distances (assumed shortest distance and longest distance) are estimated, and one corrected flow vector is generated (S308). The flow vector subtraction unit 168 subtracts the corrected flow vector from the derived flow vector (S310). The grouping unit 170 groups the flow vectors indicating the moving object 2 based on the determination reference line 204 (S312). Then, the moving object determination unit 172 determines the position, traveling direction, and speed of the moving object 2 constituting the block group according to the size of the block group grouped by the grouping unit 170 and the speed of the block group. Is determined (S314).

  The collision determination unit 174 determines whether or not the own vehicle 1 and the moving object 2 are likely to collide according to the traveling direction and speed of the own vehicle 1 and the position, traveling direction, and speed of the identified moving object 2. Determination is made (S316). As a result, if it is determined that there is a high possibility of collision with the moving object 2 (YES in S316), the notification unit 176 notifies the driver that the possibility of collision is high and notifies the vehicle control device 130 to that effect. Is output (S318). If it is not determined that the possibility of collision with the moving object 2 is high (NO in S316), the process proceeds to step S320.

  Finally, the image holding unit 178 overwrites the previous image data of the data holding unit 152 with the current image data in order to use the current image data as the previous image data in the next frame (S320). Hereinafter, the image cutout process S302, the flow vector derivation process S304, the corrected flow vector generation process S308, the flow vector subtraction process S310, and the grouping process S312 will be specifically described.

(Image cutting process S302)
Referring to FIG. 17, the image cutout unit 160 initializes a vertical variable j for specifying a pixel (substitutes “0”) (S400). Subsequently, the image cutout unit 160 adds “1” to the vertical variable j and initializes the horizontal variable i (substitutes “0”) (S402). Next, the image cutout unit 160 adds “1” to the horizontal variable i (S404).

  The image cutout unit 160 extracts the luminance value of the pixel (i, j + 100) shifted from the environment image 200 by a predetermined offset amount (for example, 100 pixels) in the vertical direction, and extracts the value as the pixel ( The luminance value of i, j) is set (S406).

  Subsequently, the image cutout unit 160 determines whether or not the horizontal variable i is not less than 640 which is the maximum value of the horizontal pixels in the partial image 202 (S408), and if the horizontal variable i is less than the maximum value (S408). NO), the processing from step S404 is repeated. If the horizontal variable i is equal to or greater than the maximum value (YES in S408), the image cutout unit 160 determines whether the vertical variable j is equal to or greater than 128, which is the maximum value of the vertical pixels in the partial image 202. (S410). If the vertical variable j is less than the maximum value (NO in S410), the processing from step S402 is repeated. If the vertical variable j is equal to or greater than the maximum value (YES in S410), the image cutout process S302 is terminated. In this way, as shown in FIG. 3, the partial image 202 can be cut out from the position offset in the vertical direction of the environment image 200.

(Flow vector derivation process S304)
Referring to FIG. 18, the flow vector deriving unit 162 initializes (substitutes “0”) a vertical variable j for specifying a block (S450). Subsequently, the flow vector deriving unit 162 adds “1” to the vertical variable j and initializes the horizontal variable i (substitutes “0”) (S452). Next, the flow vector deriving unit 162 adds “1” to the horizontal variable i (S454). Here, the block is composed of 8 horizontal pixels × 8 vertical pixels.

  The flow vector deriving unit 162 derives a correlation value between the block (i, j) of the partial image 202 corresponding to the previous image data and the block of the partial image 202 corresponding to the current image data (S456). Specifically, the flow vector deriving unit 162 extracts the block (i, j) of the partial image 202 corresponding to the previous image data, and corresponds to the position of the block (i, j) of the previous image data. A maximum range in which the moving object 2 can move within one frame from the position of the partial image 202 corresponding to the data is set. Then, the flow vector deriving unit 162 obtains the correlation value of both blocks while shifting the block of the previous image data pixel by pixel in the right direction within the maximum range, and further performs such processing pixel by pixel on the screen. Repeat while shifting in the direction. However, the area to be compared when obtaining the correlation value may match the block, or may not match the block, for example, may include a block or include a part of the block. This process is repeated until correlation values are derived for all the blocks within the maximum range in the partial image 202 of the current image data (NO in S458). When the correlation values are derived for all the blocks within the maximum range (YES in S458), the flow vector deriving unit 162 determines that the current image data block has the largest correlation value with the previous image data block (i, j). (I, j) is extracted, and a flow vector (size and direction between the blocks) is derived from the positional relationship between both blocks (S460).

  Subsequently, the flow vector deriving unit 162 determines whether or not the horizontal variable i is 80 or more which is the maximum value of the horizontal block of the partial image 202 (S462), and if the horizontal variable i is less than the maximum value (S462). NO), the processing from step S454 is repeated. If the horizontal variable i is equal to or greater than the maximum value (YES in S462), the flow vector deriving unit 162 determines whether the vertical variable j is equal to or greater than 16 which is the maximum value of the vertical block of the partial image 202 ( S464). If vertical variable j is less than the maximum value (NO in S464), the processing from step S452 is repeated. If the vertical variable j is equal to or greater than the maximum value (YES in S464), the flow vector derivation process S304 is terminated. In this way, the flow vector information can be added to the partial image 202 as shown in FIG.

(Correction flow vector generation processing S308)
Referring to FIG. 19, the corrected flow vector generation unit 166 first assumes a planar wall that is orthogonal to the direction of the host vehicle 1 before turning and is at a predetermined long distance (for example, 40000 m) in front of the vehicle. A flow vector is derived for a portion where the wall surface and the ground are in contact (the block is a wall and the block immediately below in the vertical direction is the ground) (S500). Such a long-distance flow vector can be represented by a vector sequence far (80) of 80 blocks in the horizontal direction independent of the vertical direction. Next, the correction flow vector generation unit 166 assumes a planar wall that is orthogonal to the direction of the host vehicle 1 before the turn and is at a predetermined short distance (for example, 1 m) in front of the vehicle, and the wall surface and the ground are A flow vector of the part in contact is derived (S502). Such a short-distance flow vector can be represented by a vector sequence near (80) of 80 blocks in the horizontal direction independent of the vertical direction.

  The corrected flow vector generation unit 166 initializes (assigns “0”) the vertical variable j for specifying the block (S504). Subsequently, the corrected flow vector generation unit 166 adds “1” to the vertical variable j and initializes the horizontal variable i (substitutes “0”) (S506). Next, the corrected flow vector generation unit 166 adds “1” to the horizontal variable i (S508). Again, the block is composed of 8 horizontal pixels × 8 vertical pixels.

  The corrected flow vector generation unit 166 determines whether or not the horizontal variable i is 40 or less, that is, whether or not the block is in the left region of the determination reference line 204 (S510). If the horizontal variable i is 40 or less (YES in S510), it is determined whether or not the long-distance flow vector far (i) is larger than the short-distance flow vector near (i) (S512). When the long-distance flow vector far (i) is larger than the short-distance flow vector near (i) (YES in S512), the long-distance flow vector far (i) is substituted into the corrected flow vector tgt (i, j). (S514). When the long-distance flow vector far (i) is smaller than the short-distance flow vector near (i) (NO in S512), the short-distance flow vector near (i) is substituted into the corrected flow vector tgt (i, j). (S516). Here, the longest rightward flow vector or the shortest leftward flow vector in the left region is used as the correction flow vector.

  If the horizontal variable i is larger than 40 (NO in S510), it is determined whether the long-distance flow vector far (i) is larger than the short-distance flow vector near (i) (S518). When the long-distance flow vector far (i) is larger than the short-distance flow vector near (i) (YES in S518), the short-distance flow vector near (i) is substituted into the corrected flow vector tgt (i, j). (S520). When the long-distance flow vector far (i) is smaller than the short-distance flow vector near (i) (NO in S518), the long-distance flow vector far (i) is substituted into the corrected flow vector tgt (i, j). (S522). Here, the longest leftward flow direction or the shortest rightward flow vector in the right region is used as the correction flow vector.

  Subsequently, the corrected flow vector generation unit 166 determines whether or not the horizontal variable i is 80 or more which is the maximum value of the horizontal block of the partial image 202 (S524), and if the horizontal variable i is less than the maximum value (S524). NO in S524), the process from step S508 is repeated. If the horizontal variable i is equal to or greater than the maximum value (YES in S524), the corrected flow vector generation unit 166 determines whether the vertical variable j is equal to or greater than 16 that is the maximum value of the vertical block of the partial image 202. (S526). If vertical variable j is less than the maximum value (NO in S526), the processing from step S506 is repeated. If the vertical variable j is equal to or greater than the maximum value (YES in S526), the correction flow vector generation process S308 ends. In the correction flow vector generation processing S308, the same correction flow vector is set in the vertical direction of the partial image 202. In this way, a corrected flow vector is generated as shown in FIG.

(Flow vector subtraction processing S310)
Referring to FIG. 20, the flow vector subtraction unit 168 initializes a vertical variable j for specifying a block (substitutes “0”) (S550). Subsequently, the flow vector subtraction unit 168 adds “1” to the vertical variable j and initializes the horizontal variable i (substitutes “0”) (S552). Next, the flow vector subtraction unit 168 adds “1” to the horizontal variable i (S554). Here, the block is composed of 8 horizontal pixels × 8 vertical pixels.

  The flow vector subtraction unit 168 subtracts the corrected flow vector tgt (i, j) from the flow vector of the block (i, j) of the partial image 202 to update the flow vector of the block (i, j) (S556). .

  Subsequently, the flow vector subtraction unit 168 determines whether or not the horizontal variable i is 80 or more which is the maximum value of the horizontal block of the partial image 202 (S558), and if the horizontal variable i is less than the maximum value (S558). NO), the processing from step S554 is repeated. If the horizontal variable i is equal to or greater than the maximum value (YES in S558), the flow vector subtraction unit 168 determines whether or not the vertical variable j is equal to or greater than 16 that is the maximum value of the vertical block of the partial image 202 ( S560). If vertical variable j is less than the maximum value (NO in S560), the processing from step S552 is repeated. If the vertical variable j is equal to or greater than the maximum value (YES in S560), the flow vector subtraction process S310 is terminated. Thus, the flow vector of the partial image 202 is offset by the corrected flow vector tgt (i, j).

(Grouping process S312)
Referring to FIG. 21, the grouping unit 170 initializes (substitutes “0”) a vertical variable j for specifying a block (S600). Subsequently, the grouping unit 170 adds “1” to the vertical variable j and initializes the horizontal variable i (substitutes “0”) (S602). Next, the grouping unit 170 adds “1” to the horizontal variable i (S604).

  Next, the grouping unit 170 initializes a vertical variable n (substitutes “0”) in order to identify a block to be compared (S606). Subsequently, the grouping unit 170 adds “1” to the vertical variable n and initializes the horizontal variable m (substitutes “0”) (S608). Next, the grouping unit 170 adds “1” to the horizontal variable m (S610).

  Then, the grouping unit 170 determines whether the block (i, j) and the block (m, n) of the previous image data are flow vectors corresponding to the moving object 2 based on the determination reference line 204. Determination is made (S612). Specifically, if the block (i, j) exists in the left region from the determination reference line 204, it is determined whether or not the flow vector is directed rightward. If the block (i, j) exists in the right region from the determination reference line 204, It is determined whether or not the flow vector is directed leftward. Similarly, if the block (m, n) exists in the left region from the determination reference line 204, it is determined whether or not the flow vector is directed rightward. If the block (m, n) exists in the right region from the determination reference line 204, the flow is determined. It is determined whether or not the vector is directed leftward. As a result, if the flow vector corresponds to the moving object 2 (YES in S612), the process proceeds to step S614, and if either does not correspond to the moving object 2 (NO in S612), the process proceeds to step S618. To do.

  Subsequently, the grouping unit 170 compares the block (i, j) of the previous image data with the block (m, n) and determines whether or not the direction of the flow vector and the size thereof are approximated (S614). . As a result, when it is determined that the flow vectors are approximate (YES in S614), the grouping unit 170 groups the blocks into a block group (moving object 2) (S616). At this time, when one or both of the blocks are already specified as a block group together with another block, the other blocks that are the block group are also specified as the same block group. However, when the horizontal variable i = m and the vertical variable j = n, the same block is obtained, so the processing of steps S612 to S616 is not executed. If the flow vector is not approximate (NO in S614), the process proceeds to step S618.

  Subsequently, the grouping unit 170 determines whether or not the horizontal variable m is 80 or more, which is the maximum value of the horizontal block of the partial image 202 (S618), and if the horizontal variable m is less than the maximum value (in S618). NO), the process from step S610 is repeated. If the horizontal variable m is equal to or greater than the maximum value (YES in S618), the grouping unit 170 determines whether the vertical variable n is equal to or greater than 16 that is the maximum value of the vertical block of the partial image 202 ( S620). If the vertical variable n is less than the maximum value (NO in S620), the processing from step S608 is repeated. If the vertical variable n is equal to or greater than the maximum value (YES in S620), the process proceeds to step S622.

  Subsequently, the grouping unit 170 determines whether or not the horizontal variable i is 80 or more, which is the maximum value of the horizontal block of the partial image 202 (S622), and if the horizontal variable i is less than the maximum value (in S622). NO), the process from step S604 is repeated. If the horizontal variable i is equal to or greater than the maximum value (YES in S622), the grouping unit 170 determines whether the vertical variable j is equal to or greater than 16 that is the maximum value of the vertical block of the partial image 202 (S624). ). If vertical variable j is less than the maximum value (NO in S624), the processing from step S602 is repeated. If the vertical variable j is equal to or greater than the maximum value (YES in S624), the grouping process S312 ends. In this way, blocks with similar flow vectors can be identified as the moving object 2.

  As described above, according to the outside environment recognition device 120 and the outside environment recognition method described above, even when the host vehicle 1 turns, it is possible to avoid erroneous detection of the stationary object 3 as the moving object 2 and to move. The detection accuracy of the object 2 can be improved.

  Conventionally, in order to appropriately determine the moving object 2, a stereo camera or a laser radar has to be used in order to obtain a distance to the stationary object 3. Since the moving object 2 can be detected without using an expensive device, cost reduction and processing load can be reduced.

  Also provided are a program that causes a computer to function as the vehicle exterior environment recognition device 120 and a computer-readable storage medium that stores the program, such as a flexible disk, magneto-optical disk, ROM, CD, DVD, and BD. Here, the program refers to data processing means described in an arbitrary language or description method.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to this embodiment. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Is done.

  For example, in the above-described embodiment, the image cutout unit 160, the flow vector derivation unit 162, the movement state acquisition unit 164, the corrected flow vector generation unit 166, the flow vector subtraction unit 168, the grouping unit 170, the moving object determination unit 172, The collision determination unit 174, the notification unit 176, and the image holding unit 178 are configured to operate by software by the central control unit 154. However, the functional unit described above can be configured by hardware.

  In the above-described embodiment, the flow vector in either the horizontal direction or the vertical direction of the partial image 202 is targeted, but the moving object 2 approaching through the imaging device 110 installed in the host vehicle 1 is detected. The processing load can be reduced by performing the processing only on the flow vector in the horizontal direction.

  Note that each step of the vehicle environment recognition method of the present specification does not necessarily have to be processed in time series in the order described in the flowchart, and may include processing in parallel or a subroutine.

  INDUSTRIAL APPLICABILITY The present invention can be used for a vehicle environment recognition apparatus and a vehicle environment recognition method that recognize an environment outside the host vehicle.

DESCRIPTION OF SYMBOLS 1 ... Own vehicle 2 ... Moving object 3 ... Stationary object 3a ... Long-distance stationary object 3b ... Short-distance stationary object 110 ... Imaging device 120 ... Outside-vehicle environment recognition apparatus 150 ... I / F part (image data acquisition part)
160 ... Image cutout part 162 ... Flow vector derivation part 164 ... Movement state acquisition part 166 ... Correction flow vector generation part 168 ... Flow vector subtraction part 170 ... Grouping part 202 ... Partial image 204 ... Determination reference line

Claims (5)

An image data acquisition unit for acquiring image data obtained by imaging the environment outside the vehicle through a wide angle lens;
An image cutout unit that cuts out a partial image having a predetermined position and size from an image based on the image data;
A flow vector deriving unit for deriving a flow vector at an arbitrary part of the partial image;
A movement state acquisition unit for acquiring a movement state of the wide-angle lens;
Assuming two stationary objects at two different distances from the wide-angle lens, estimating a flow vector of the two stationary objects based on the moving state and generating one corrected flow vector, a corrected flow vector generating unit When,
A flow vector subtraction unit that subtracts the corrected flow vector from the derived flow vector;
A grouping unit for grouping flow vectors indicating moving objects among the subtracted flow vectors based on a determination reference line at a horizontal center of the image data;
A vehicle exterior environment recognition device comprising:   The correction flow vector generation unit is a flow vector having a relatively large scalar amount toward the determination reference line or a small scalar amount toward the opposite direction to the determination reference line, out of the two estimated flow vectors. The outside environment recognition apparatus according to claim 1, wherein the correction flow vector is a correction flow vector.   The correction flow vector generation unit uses the longest rightward flow direction or the shortest leftward flow vector as a correction flow vector in the left region of the determination reference line out of the two estimated flow vectors, and in the right region of the determination reference line, The vehicle exterior environment recognition apparatus according to claim 2, wherein the leftmost longest flow vector or the rightmost shortest flow vector is used as a correction flow vector. The moving state acquisition unit acquires a moving speed and an angular speed of the wide-angle lens,
The outside environment recognition device according to claim 1, wherein the corrected flow vector generation unit estimates a flow vector based on the moving speed and the angular speed. Obtain image data of the environment outside the vehicle through a wide-angle lens,
Cut out a partial image of a predetermined position and size from the image based on the image data,
Deriving a flow vector at an arbitrary part of the partial image,
Obtaining the movement state of the wide-angle lens;
Assuming two stationary objects at two different distances from the wide-angle lens, estimating flow vectors of the two stationary objects based on the moving state, and generating one corrected flow vector,
Subtracting the corrected flow vector from the derived flow vector;
A method of recognizing an environment outside a vehicle, wherein flow vectors indicating moving objects are grouped out of the subtracted flow vectors based on a determination reference line at a horizontal center of the image data.

Images