JP6174884B2 - Outside environment recognition device and outside environment recognition method - Google Patents

Description

  The present invention relates to an outside environment recognition device and an outside environment recognition method for recognizing an environment outside the host vehicle, and more particularly, an outside environment recognition device and an outside environment that recognize a moving object outside the host vehicle and determine a collision with the host vehicle. It relates to the recognition method.

  2. Description of the Related Art Conventionally, there is a technology for mounting a wide-angle (wide angle of view) camera in front of the host vehicle, and displaying an environmental image in front of the host vehicle captured by the wide-angle camera on a display in the passenger compartment to assist the driver's visual confirmation. Are known. In such a technique, for example, it is detected that the own vehicle is temporarily stopped through the control information in the own vehicle, and control such as automatically switching the display on the monitor device to an environmental image captured by a wide-angle camera is performed. It has been broken.

  In order to detect an object such as a vehicle or an obstacle located in front of the host vehicle from an environmental image through such a wide-angle camera, an image processing technique such as an optical flow is being studied. Such an optical flow is represented by a two-dimensional velocity vector field on the image, that is, a velocity field accompanying movement of a moving object on the image. In such a technique, for example, a point that can be recognized as the same part is set as a feature point between two images consecutive in the time direction imaged in a predetermined cycle, and the movement of the feature point is a vector (hereinafter referred to as a feature point) , Referred to as “flow vector”). Then, by calculating this flow vector in the entire area in the captured image, for example, the position of the moving object in the image and the moving speed and movement of the moving object only by the presence or absence of the flow vector when the host vehicle is stopped. Information such as direction can be detected.

  In the optical flow, the approach of a moving object can be suitably detected when the host vehicle is stopped as described above, but a technique for detecting a moving object while traveling is also being studied. For example, during traveling, a moving object obtained in advance is corrected with a background flow vector in order to obtain an accurate flow vector (for example, Patent Document 1). In addition, a technique has been disclosed in which a spatial model that is the basis of such a background flow vector is generated using distance data (for example, Patent Document 2).

Japanese Patent No. 3239521 Japanese Patent No. 3776094

  In the techniques of Patent Document 1 and Patent Document 2 described above, the apparent flow vector of the entire screen generated by the movement of the host vehicle is derived from the optical flow and subtracted from the entire optical flow. Yes. However, in order to derive the apparent flow vector, not only the traveling direction and speed of the host vehicle but also the distance from the host vehicle to a stationary object or moving object existing in the background is required. Such a distance to a stationary object or moving object must be measured each time using a visual sensor such as a stereo camera or a laser radar, resulting in an increase in installation cost and processing load. Even if the background flow vector can be derived without using the distance to the stationary object or the moving object, if the correction is made for the entire screen, the processing load also increases. Therefore, the techniques of Patent Document 1 and Patent Document 2 are not suitable for in-vehicle use that requires low-cost and real-time processing.

  In view of such a problem, the present invention provides a moving object with high accuracy and low processing load even when the host vehicle turns without providing a special device for calculating the distance between the host vehicle and a stationary object. It is an object to provide a vehicle environment recognition apparatus and a vehicle environment recognition method capable of detecting

In order to solve the above-described problem, an external environment recognition apparatus according to the present invention includes an image data acquisition unit that acquires image data obtained by imaging an external environment through an imaging device, and a flow vector for each block obtained by dividing an image based on the image data. A flow vector deriving unit for deriving, a grouping unit for approximating the direction and size of the flow vector and generating a block group by grouping adjacent blocks and obtaining a representative flow vector as a representative value thereof A moving state acquisition unit that acquires the behavior of the imaging device during turning, a correction flow vector extraction unit that extracts a ground flow vector according to the turning behavior acquired by the movement state acquisition unit, and a representative flow vector , the block group of the flow vector of the ground, at the boundary change is the degree of change in flow vector representative sites And corresponding flow vector subtraction unit that subtracts the flow vector of the site of the block group, in accordance with the scalar quantity of the flow vector after subtraction, as long as it is within a predetermined range of scalar quantity is zero or near zero, the block group and stationary, if not within a predetermined range of the amount scalar 0 or near 0, a stationary object determination unit shall be the moving object to block group,
It is characterized by providing.

The vehicle exterior environment recognition apparatus further includes a data holding unit that holds a plurality of ground flow vectors respectively corresponding to a plurality of different turning behaviors, and the correction flow vector extraction unit includes a plurality of correction flow vector extraction units that are held in the data holding unit. from the flow vector of the ground, depending on the behavior during turning movement state acquisition unit has acquired, it may be extracted flow vector of the earth surface.

  The vehicle exterior environment recognition apparatus further includes a collision determination unit that performs a collision determination between the moving object and the host vehicle using the flow vector after subtraction of the block group that is set as the moving object in the stationary object determination unit as the flow vector of the moving object. Also good.

In order to solve the above-described problem, an external environment recognition method according to the present invention acquires image data obtained by imaging an external environment through an imaging device, derives a flow vector for each block obtained by dividing an image based on the image data, and the flow vector The direction and its size are approximated, and adjacent blocks are grouped together to generate a block group, a representative flow vector that is the representative value is obtained, and the behavior of the imaging device during turning is obtained. The flow vector of the part corresponding to the representative part located at the boundary where the change degree of the flow vector in the block group of the ground flow vector changes from the representative flow vector is extracted from the representative flow vector. subtraction, of the block group, in accordance with the scalar quantity of the flow vector after subtraction, scalar quantity is zero or near zero at Within the range, the blocks and stationary, if not within a predetermined range of scalar quantity is zero or near zero, and wherein the moving object and to Rukoto the blocks.

  According to the present invention, by not providing a special device for calculating the distance between the host vehicle and a stationary object, high accuracy can be achieved even when the host vehicle turns while avoiding an increase in cost and processing load. In addition, the moving object can be detected with a low processing load.

It is the block diagram which showed the connection relation of the environment recognition system. It is explanatory drawing for demonstrating the image produced | generated with an imaging device. It is explanatory drawing for demonstrating a partial image. It is explanatory drawing for demonstrating the determination process of the moving object by a flow vector. It is explanatory drawing for demonstrating the determination process of the moving object by a flow vector. It is explanatory drawing for demonstrating the influence of the moving speed of the own vehicle. It is explanatory drawing which showed the flow vector of the ground. It is explanatory drawing for demonstrating the influence of the moving speed when the own vehicle goes straight. It is explanatory drawing for demonstrating the change of the flow vector of a screen vertical direction. It is explanatory drawing which showed the flow vector of the partial image. It is the functional block diagram which showed the schematic function of the external environment recognition apparatus. It is explanatory drawing for demonstrating the process of a flow vector derivation | leading-out part. It is explanatory drawing for demonstrating a block group. It is the flowchart which showed the flow of the whole process of the environment recognition method outside a vehicle.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The dimensions, materials, and other specific numerical values shown in the embodiment are merely examples for facilitating understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted, and elements not directly related to the present invention are not illustrated. To do.

(Environment recognition system 100)
FIG. 1 is a block diagram showing a connection relationship of the environment recognition system 100. The environment recognition system 100 includes an imaging device 110, a vehicle exterior environment recognition device 120, and a vehicle control device (ECU: Engine Control Unit) 130 provided in the host vehicle 1.

  The imaging device 110 includes an imaging element such as a wide angle (wide field angle) lens and a CCD (Charge-Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor), and an optical axis in front of the host vehicle 1. Is installed with the side facing down slightly from the horizontal. The imaging device 110 can capture an environment corresponding to the front of the host vehicle 1 and generate a color image or a monochrome image composed of three hues (R (red), G (green), and B (blue)). . Here, the wide-angle lens refers to a lens having a relatively wide angle of view (viewing angle) compared to a standard lens (for example, 120 degrees or more) or a relatively short focal length. For example, a fish-eye lens is used as a wide-angle lens, A hemispherical space centered on the imaging device 110 can be imaged.

  Note that the projection method of the imaging apparatus 110 according to the present embodiment uses a three-dimensional projection, but various existing projection methods such as a perspective projection can be employed. Here, an example in which a fish-eye lens is used as the wide-angle lens will be described, but various wide-angle lenses such as a conical (axicon) prism can also be used. The reason why the angle of view of the lens is set to 120 degrees or more is to allow the vehicle 1 to properly grasp the situation of the orthogonal road when the host vehicle 1 stops at the stop line on the cross road or the T-shaped road.

  FIG. 2 is an explanatory diagram for explaining an image generated by the imaging apparatus 110. As can be understood with reference to FIG. 2, an image 200 (hereinafter referred to as “environment image”) 200 generated by the imaging device 110 has a monotonically increasing angle with respect to the distance from the center of the image through a circumferential fisheye lens. (3D projection method). Therefore, it is possible to acquire a wide range of environmental images 200 at a time while maintaining the resolution of the image in the traveling direction of the host vehicle 1.

  In addition, the imaging device 110 continuously generates image data obtained by converting the environment image 200 into data, for example, every 1/10 second (10 fps). Each functional unit in the following embodiment performs each process triggered by such update of image data. Therefore, each functional unit repeats the process in units of frames.

  The outside environment recognition device 120 acquires image data indicating the environment image 200 from the imaging device 110, and derives a flow vector based on the optical flow using so-called pattern matching. However, the vehicle environment recognition apparatus 120 derives a flow vector using a partial image obtained by cutting out only a necessary portion in the environmental image 200.

  FIG. 3 is an explanatory diagram for explaining the partial image 202. As described above, in the environment image 200 shown in FIG. 2, since a wide-angle image can be acquired at a time, not only the front of the host vehicle 1 but also a moving object approaching from either the left or right direction (horizontal direction) It becomes possible to specify. However, in the environment image 200, since the distance from the center of the image and the angle are approximately proportional, the image is distorted as the distance from the center increases. For example, in the area close to the four corners in the environment image 200 of FIG. 2, an object that actually extends horizontally is displayed curved in the environment image 200, and its end includes a vertical component. Can be visually recognized. Here, “horizontal” indicates the screen horizontal direction of the captured environment image 200, and “vertical” indicates the screen vertical direction of the captured environment image 200.

  Therefore, the outside environment recognition apparatus 120 includes a region indicated by a broken line in FIG. 2 where there is a moving object to be specified in the traffic environment in the environment image 200 shown in FIG. A partial image 202 shown in FIG. 3 is cut out by combining the central region and the left and right regions in the horizontal direction. Then, a flow vector based on the optical flow is derived for the partial image 202.

  Subsequently, the vehicle environment recognition apparatus 120 includes a speed sensor 124 that detects the moving speed of the host vehicle 1 and an angular speed sensor that detects an angular speed centered on the vertical axis when the host vehicle 1 turns as shown in FIG. 126, the behavior of the host vehicle 1 (the behavior of the imaging device 110) is measured. Then, the vehicle exterior environment recognition device 120 acquires a flow vector of the ground that is inherent in the partial image 202 and is affected by the behavior of the host vehicle 1. Next, the vehicle environment recognition apparatus 120 subtracts the ground flow vector from the partial flow vector for the partial image 202 to derive a flow vector from which the influence of the behavior of the host vehicle 1 is removed. Then, the outside environment recognition device 120 identifies a stationary object or a moving object based on the flow vector from which the influence of the behavior of the host vehicle 1 is removed, and the moving object is determined by its size, moving speed, and moving direction. The type of is determined. Here, the stationary object indicates an object such as a building, a road, a guardrail, or a background space, and the moving object indicates an object that moves independently such as a vehicle, a bicycle, or a pedestrian.

  Moreover, the vehicle exterior environment recognition apparatus 120 determines whether there is a high possibility that the moving object approaching the vehicle 1 from the side in the horizontal direction at high speed and the vehicle 1 will collide, for example. be able to. Here, when it is determined that the possibility of a collision is high, the vehicle exterior environment recognition device 120 notifies the driver of the fact through the display 122 and outputs information indicating the fact to the vehicle control device 130. . The processing of the outside environment recognition device 120 will be described in detail later.

  The vehicle control device 130 avoids a collision with a moving object specified by the outside environment recognition device 120. Specifically, the vehicle control device 130 acquires the current traveling state of the host vehicle 1 through the steering angle sensor 132 that detects the steering angle, the speed sensor 124 described above, and the like, and controls the actuator 134 to determine the preceding vehicle. Keep the distance between cars safe. Here, the actuator 134 is a vehicle control actuator used for controlling a brake, a throttle valve, a steering angle, and the like. Further, when a collision with a moving object is assumed, the vehicle control device 130 controls the actuator 134 to automatically brake the host vehicle 1. Such a vehicle control device 130 can also be formed integrally with the outside environment recognition device 120.

(Moving object judgment process by flow vector)
4 and 5 are explanatory diagrams for explaining the moving object determination processing based on the optical flow flow vector according to the present embodiment. Here, the features of the present embodiment will be described in comparison with conventional problems.

  Assuming that the moving object 2 crosses the front when the host vehicle 1 is traveling straight without turning as shown in FIG. When a flow vector is derived for the partial image 202 shown in FIG. 3, the flow vector is indicated by a plurality of arrows as shown in FIG. The length of the arrow indicates the size of the flow vector, and the direction of the arrow indicates the direction of the flow vector. In addition, the number of flow vectors is essentially the same as the number of blocks composed of a predetermined number of pixels, which is a derived unit. However, here, the number of flow vectors is limited to facilitate understanding.

  Since the flow vector is generated by the movement of the feature point in the image, as can be understood with reference to FIG. 4B, not only the moving object 2 but also the stationary object 3 is accompanied by the movement of the host vehicle 1. Also produces a flow vector. In this embodiment, since a wide-angle lens is used, it can be seen that the flow vector of the stationary object 3 increases in size as it shifts to the side as compared with the center of the partial image 202. .

  In addition, when the imaging device 110 is positioned at the front end of the host vehicle 1 and in the center in the vehicle width direction and installed so that the imaging direction is equal to the traveling direction, as shown in FIG. The horizontal direction of the flow vector is reversed left and right with a central reference line 204 extending perpendicularly to the horizontal center of the partial image 202.

  At this time, if the partial image 202 shown in FIG. 4B is divided into a left region and a right region based on the central reference line 204, the flow vector of the stationary object 3 is generated in a direction away from the central reference line 204. On the other hand, it can be seen that the flow vector of the moving object 2 approaching the host vehicle 1 is generated in a direction approaching the central reference line 204.

  Therefore, a flow vector that divides the region horizontally in accordance with the center reference line 204 and that faces the right direction in the left region or a flow vector that faces the left direction in the right region as shown in FIG. 4B is detected. Then, they can be specified as the moving object 2.

  On the other hand, as shown in FIG. 5 (a), when the host vehicle 1 travels with a turn, the flow vector is represented as shown in FIG. 5 (b). Here, the flow vector of the stationary object 3 is generated in a direction approaching the central reference line 204 even though the stationary object 3 in the region 206 indicated by the alternate long and short dash line exists in the right region of the central reference line 204. This is because the position where the horizontal direction of the flow vector is reversed left and right coincides with the vanishing point ahead (infinity) when the host vehicle 1 goes straight, but when the host vehicle 1 turns, the moving speed of the host vehicle 1 This is caused by moving in the horizontal direction (right side in this case) depending on the angular velocity and the distance to the stationary object 3.

  Then, the flow vector in the area 206 indicated by the alternate long and short dash line is the flow vector facing the left direction in the right area regardless of whether it is the moving object 2 or the stationary object 3. Therefore, it is erroneously determined as the moving object 2. In this case, it is possible not to perform the determination of the moving object 2 in the area corresponding to the inside of the turn, that is, the right area, but the moving object 2 existing in the area cannot be detected. May increase risk.

  Further, even when the host vehicle 1 does not involve turning, the length of the flow vector of the moving object 2 that crosses the front of the host vehicle 1 will be described below when the host vehicle 1 is stopped and when the host vehicle 1 is traveling straight. Will be different.

  FIG. 6 is an explanatory diagram for explaining the influence of the moving speed of the host vehicle 1. As shown in FIG. 6A, when the moving object 2a crosses the front as indicated by an alternate long and short dash line when the host vehicle 1 is stopped, the partial image 202 is located when the moving object 2a is located at the point A. When the moving object 2a is located at the point B, it is projected onto the part B ′ of the partial image 202. Therefore, the flow vector of the moving object 2a when the host vehicle 1 is stopped is from the part A 'to the part B'. On the other hand, when the moving object 2a crosses the front when the host vehicle 1 is traveling straight in the traveling direction as shown in FIG. 6B, when the moving object 2a is located at the point A, the partial image 202 When projected onto the part A ″ and the moving object 2 a is located at the point B, it is projected onto the part B ″ of the partial image 202. Therefore, the flow vector of the moving object 2a when the host vehicle 1 is traveling straight goes from the part A "to the part B".

  Comparing FIG. 6 (a) and FIG. 6 (b), the length of the flow vector is as shown in FIG. 6 (a) even though the moving amount of the moving object 2a is the same from point A to point B. It can be understood that FIG. 6B is shorter. Then, when calculating the collision speed by calculating the moving speed of the moving object 2a approaching the host vehicle 1 from the length of the flow vector in FIG. 6B, the moving speed of the moving object 2a is higher than the actual speed. It may be misrecognized late, which may delay the warning to the driver and increase the risk of collision.

  As described with reference to FIGS. 4 to 6, the lengths and directions of the flow vectors of the moving object 2 and the stationary object 3 differ depending on the moving speed and the angular speed of the host vehicle 1. Therefore, if the influence due to the behavior of the own vehicle 1 can be accurately estimated, the influence due to the behavior of the own vehicle 1 is offset from the actually observed flow vector, the flow vector of the stationary object 3 is excluded, and the moving object 2 It should be possible to accurately grasp the absolute movement of the.

  However, in order to accurately estimate the flow vectors of the moving object 2 and the stationary object 3 based on the behavior of the own vehicle 1, not only the moving speed and angular velocity of the own vehicle 1 but also the moving object 2 and the stationary object from the own vehicle 1. A distance of up to 3 is also required. Such a distance to the moving object 2 or the stationary object 3 can be derived by performing a calculation based on a three-point positioning method using a stereo camera or using a laser radar, but is measured each time. This will cause an increase in installation cost and processing load.

  Therefore, in the present embodiment, attention is paid to the ground flow vector (hereinafter, the ground flow vector is also referred to as a correction flow vector). Here, the ground is set such that the imaging device 110 is positioned at the front end of the host vehicle 1 and at the center in the vehicle width direction, the imaging direction is equal to the traveling direction, and the optical axis is horizontal or downward. In this case, the ground surface that can be imaged is shown. When the imaging device 110 is installed in this manner, the captured image shows the ground on which the host vehicle 1 is expected to travel. Although the positional relationship between the host vehicle 1 and the ground changes instantaneously depending on the behavior of the host vehicle 1, the shape of the ground, and the like, the positional relationship is constantly constant, and the positional relationship is related to the imaging device 110 being attached to the vehicle. It can be determined from the installation conditions when installed.

  FIG. 7 is an explanatory view showing the flow vector of the ground, and FIG. 8 is an explanatory view for explaining the influence of the moving speed when the host vehicle 1 goes straight. For example, when the host vehicle 1 is stopped, all the ground flow vectors are zero. Further, when the host vehicle 1 is traveling straight without turning, as shown in FIG. 7A, the vehicle is reversed left and right at the center reference line 204 located at the center in the horizontal direction in the partial image 202, to the side. The heading flow vector is displayed. Further, when the host vehicle 1 is traveling while turning rightward, as shown in FIG. 7 (b), the right and left sides of the center reference line 204 are reversed right and left, and a flow vector directed to the side is displayed. Is done. As described above, when only the ground is shown by perspective projection in the image picked up by the image pickup device 110, when the image pickup device 110 moves forward integrally with the host vehicle 1, FIG. 7A and FIG. As shown, a flow vector is generated in a radial manner with the infinity point at the front as a vanishing point, and the longer the distance is, the shorter the distance is. This can be understood from the following description using FIG.

  When the host vehicle 1 is traveling straight at the moving speed C, the stationary object 3a that is relatively far from the host vehicle 1 and the stationary object 3b that is relatively close to the host vehicle 1 are at the relative speed C. Therefore, the relative movement at the relative speed C is projected onto the partial image 202 and becomes a flow vector. As shown in FIG. 8, although the stationary object 3a and the stationary object 3b have the same relative speed C, the lengths of the flow vectors differ from the own vehicle 1 because the distance from the own vehicle 1 is different. The flow vector of the stationary object 3 a located at a relatively far position is shorter than the flow vector of the stationary object 3 b located relatively close to the host vehicle 1. Therefore, the flow vector is relatively short near the center in the horizontal direction on the screen, and the flow vector becomes longer as it shifts from the side of the image. In addition, the flow vector is relatively short near the horizon in the image, and the flow vector becomes longer as it approaches the lower end of the image, that is, the front side of the ground.

FIG. 9 is an explanatory diagram for explaining a change in the flow vector in the vertical direction of the screen. FIG. 9B shows the length of the flow vector in the i direction in the screen vertical direction of i ₀ of the partial image 202 in FIG. 9A. Referring to FIG. 9B, as described above, the flow vector in the area corresponding to the ground becomes shorter as it shifts from the lower end of the screen to the center of the screen (horizon). However, the length of the flow vector in the region corresponding to the standing object, for example, the stationary object 3 in FIG. 9A does not change. Therefore, the boundary where the degree of change in the flow vector changes can be set as the boundary between the ground and the standing object (representative part described later).

  Further, the ground flow vector is correlated with the moving speed and angular velocity of the host vehicle 1, and if the moving speed and angular velocity of the host vehicle 1 can be specified, the turning radius can be obtained, and each feature point of the ground from the host vehicle 1 can be obtained. Without knowing the distance, the ground flow vector is also determined uniformly. Therefore, the mounting position of the imaging device 110 and the internal parameters (such as the angle of view and the projection method) of the imaging device 110 are determined without calculating the ground flow vector in real time every time according to the moving speed and angular velocity of the host vehicle 1. If so, the flow vector of the ground can be held in advance. In the present embodiment, a plurality of ground flow vectors are stored in advance as correction tables in association with a plurality of different moving speeds and different angular velocities. In this way, the processing load for calculating the ground flow vector can be reduced.

  FIG. 10 is an explanatory diagram showing a flow vector of the partial image 202. Here, the relationship between the flow vector of the partial image 202 and the flow vector of the ground will be described. The moving speed and angular velocity of the own vehicle 1 are specified, and the flow vector of the ground image corresponding to the specified moving speed and angular velocity is referred to, from the flow vector for the partial image 202 as shown in FIG. Assume that the ground flow vector (correction flow vector) is subtracted as shown in b). Then, as shown in FIG. 10, it is possible to obtain a flow vector from which the influence of the behavior of the host vehicle 1 is removed. Accordingly, only the standing objects (moving object 2 and stationary object 3) standing on the ground (with a vertical component) remain in the flow vector after subtraction, and the subtracted flow vector is the original of the standing object. The flow vector will be shown.

  However, the stationary object 3 of the standing objects is not at risk of collision unless the host vehicle 1 is actively approached, but the moving object 2 has a risk of collision regardless of the behavior of the host vehicle 1. is there. Therefore, in this embodiment, the specified standing object is further distinguished into the moving object 2 and the stationary object 3. Here, in order to distinguish the moving object 2 and the stationary object 3, attention is paid to the following flow vector characteristics. That is, the stationary object 3 is continuous with the ground and moves integrally with the ground on the screen. Therefore, as shown in FIG. The flow vector of the part in contact with the horizontal component of the screen becomes equal. In other words, when the ground flow vector is subtracted from the flow vector of the stationary object 3 in the partial image 202, the scalar quantity of the flow vector after the subtraction is substantially 0 (zero). On the other hand, although the moving object 2 is continuous with the ground, it moves relatively on the ground in parallel with the ground, so the flow vector of the part where the moving object 2 and the ground are in contact with each other. The flow vector and the horizontal component are different. In other words, when the ground flow vector is subtracted from the flow vector of the moving object 2 in the partial image 202, the scalar quantity of the flow vector after the subtraction does not become zero.

  Accordingly, among the specified standing objects, an object having a flow vector component equal to the flow vector of the part (representative part) where the flow vector is in contact with the ground is defined as a stationary object 3, and the other object is a moving object 2. By doing so, it becomes possible to distinguish the moving object 2 and the stationary object 3. As described above, in the present embodiment, the corrected flow vector is subtracted from the flow vector of the partial image 202, and the flow vector corresponding to the stationary object 3 is specified among the flow vectors after the subtraction. The flow vector of the object 2 is also specified. Hereinafter, the vehicle environment recognition apparatus 120 for realizing such processing will be specifically described, and then the vehicle environment recognition method will be described in detail.

(Vehicle environment recognition device 120)
FIG. 11 is a functional block diagram showing a schematic function of the outside environment recognition device 120. As shown in FIG. 11, the vehicle exterior environment recognition device 120 includes an I / F unit 150, a data holding unit 152, and a central control unit 154.

  The I / F unit (image data acquisition unit) 150 is an interface for performing bidirectional information exchange with the imaging device 110 and the vehicle control device 130, and for example, acquires image data generated by the imaging device 110. . Further, the I / F unit 150 acquires the moving speed and the angular speed from the speed sensor 124 and the angular speed sensor 126. The data holding unit 152 includes a RAM, a flash memory, an HDD, and the like, and holds various pieces of information necessary for processing of each functional unit described below. In the present embodiment, the image data is temporarily held until at least the next frame for pattern matching in the flow vector derivation process. Here, the image data in the previous frame stored in the data holding unit 152 is referred to as “previous image data”, and the image data in the current frame is referred to as “current image data”. In addition, the data holding unit 152 holds in advance a correction table in which a plurality of ground flow vectors are associated with a plurality of different moving speeds and different angular velocities.

  The central control unit 154 is configured by a semiconductor integrated circuit including a central processing unit (CPU), a ROM storing a program and the like, a RAM as a work area, and the like, and through the system bus 156, the I / F unit 150 and the data holding unit. 152 is controlled. In this embodiment, the central control unit 154 includes an image cutout unit 160, a flow vector derivation unit 162, a grouping unit 164, a movement state acquisition unit 166, a corrected flow vector extraction unit 168, a flow vector subtraction unit 170, a stationary It also functions as an object determination unit 172, a moving object determination unit 174, a collision determination unit 176, a notification unit 178, and an image holding unit 180.

  The image cutout unit 160 is a partial image as shown in FIG. 3 having a predetermined position and size from the environmental image 200 as shown in FIG. 2 based on the current image data acquired by the I / F unit 150. 202 is cut out.

  The flow vector deriving unit 162 derives a flow vector for each block obtained by dividing the partial image 202 cut out by the image cutout unit 160 through so-called pattern matching.

  FIG. 12 is an explanatory diagram for explaining the processing of the flow vector deriving unit 162. The flow vector deriving unit 162 is a block arbitrarily extracted from the partial image 202 shown in FIG. 12A based on the previous image data stored in the data holding unit 152 (here, an array of horizontal 8 pixels × vertical 8 pixels). A block 210b corresponding to (configured) 210a (having a high correlation value) is detected from the partial image 202 shown in FIG. 12B based on the current image data (pattern matching). At this time, the movement trajectory indicated by the arrow in FIG. 12B from the block 210a corresponding to the previous image data to the block 210b corresponding to the current image data becomes the flow vector in the optical flow.

  As such pattern matching, it is conceivable to compare luminance values in units of blocks indicating arbitrary image positions between previous image data and current image data. For example, the SAD (Sum of Absolute Difference) that takes the difference in luminance value, the SSD (Sum of Squared intensity Difference) that uses the difference squared, and the similarity of the variance value obtained by subtracting the average value from the luminance value of each pixel. There are methods such as NCC (Normalized Cross Correlation). The vehicle exterior environment recognition apparatus 120 performs such block-based flow vector derivation processing for all blocks displayed in the partial image 202 (for example, 640 pixels × 128 pixels). Further, the number of pixels in the block can be arbitrarily set, but here, the block is assumed to be 8 horizontal pixels × 8 vertical pixels.

  In this way, the flow vector deriving unit 162 can derive the flow vector of the partial image 202 corresponding to the previous image data, as shown in FIG.

  The grouping unit 164 sequentially compares the flow vectors of adjacent blocks from the flow vector derived by the flow vector deriving unit 162, and approximates the direction and the magnitude of the flow vector (for example, the direction is within ± 30 degrees and Blocks whose size difference is within 8 pixels are grouped. Then, the grouped blocks are set as candidates for standing objects (moving object 2 and stationary object 3).

  FIG. 13 is an explanatory diagram for explaining the block group 4. For example, the plurality of block groups 4 shown in FIG. 13 are grouped because the flow vectors in the blocks are directed in the same direction and their lengths are close to each other. Then, a representative flow vector that is a representative value of the flow vectors of the block group 4 is obtained. The representative flow vector may be an average value of a plurality of flow vectors grouped in the block group 4, or may be a value with high reliability calculated based on an index of coincidence such as SAD in flow vector derivation. Of these flow vectors, the flow vector located on the lower side of the screen may be used.

  In addition, the grouping unit 164 uses a part where the block group 4 is in contact with the ground, that is, a lower part of the block group 4 as a representative part. As described with reference to FIG. 9, the boundary between the standing object and the ground can be a part where the change degree of the flow vector in the vertical direction of the screen changes. Therefore, the grouping unit 164 can sequentially determine the flow vectors arranged in the vertical direction of the grouped block group 4 from the lower side of the screen, and can set a portion where the horizontal component does not change as a representative portion.

  In the present embodiment, the flow vectors are grouped in advance by the grouping unit 164, and then the corrected flow vector is subtracted from the representative flow vector. However, the present invention is not limited to this, and the corrected flow vector is first subtracted. Later, it may be grouped.

  The movement state acquisition unit 166 acquires the behavior of the imaging device 110 having a wide-angle lens, for example, the movement speed and the angular speed. Although the moving speed and the angular velocity of the imaging device 110 may be directly measured, in this embodiment, the behavior (movement) of the host vehicle 1 that moves on the same movement locus as the imaging device 110 is fixed. Speed and angular velocity). Then, the moving speed and angular velocity of the imaging device 110 are derived from the relationship between the turning center of the host vehicle 1 and the installation position of the imaging device 110. Here, the turning center can be calculated using, for example, a two-wheel model by using the wheel position and the steering angle of the host vehicle 1. Further, the relationship between the moving speed and angular speed and the turning center may be measured in advance. The means for specifying the moving speed and the angular speed of the host vehicle 1 is not limited to the speed sensor 124 and the angular speed sensor 126 described above. For example, the travel distance per unit time is used as the moving speed, or the integrated value of the angular acceleration or the steering angle. Can be determined as an angular velocity or based on various detection means.

  The correction flow vector extraction unit 168 refers to the correction table according to the movement speed and angular velocity acquired by the movement state acquisition unit 166, and extracts one correction flow vector from a plurality of correction flow vectors held in the data holding unit 152. To do. The resolution of the correction table is, for example, a moving speed 1 km interval (range 1 to 15 km / h) and an angular velocity 1 deg interval (range -30 to 30 deg / s). When a correction flow vector with a resolution higher than the resolution is required, linear interpolation may be performed using a plurality of correction flow vectors before and after the target moving speed and angular velocity. It is also possible to select the corrected flow vector so that the stationary object 3 is not erroneously detected as the moving object 2 by subtracting the corrected flow vector in a flow vector subtracting unit 170 described later.

  The flow vector subtraction unit 170 subtracts the flow vector of the part corresponding to the representative part of the block group 4 from the corrected flow vector extracted by the correction flow vector extraction part 168 from the representative flow vector derived by the grouping part 164. In this way, the flow vector of the standing object (the moving object 2 and the stationary object 3) itself that does not include the influence of the behavior of the host vehicle 1 as shown in FIG. 10 can be derived. Here, the correction flow vector is not subtracted from the entire flow vector derived by the flow vector deriving unit 162, and the correction flow vector is subtracted only from the flow vector corresponding to the representative part of the block group 4, so that the processing load is greatly reduced. can do. Even if the flow vector deriving unit 162 cannot derive an accurate flow vector due to a calculation error in the flow vector deriving unit 162 or unevenness of the ground, the subtraction target is limited, so that the influence on the subtraction result is small.

  The stationary object determination unit 172 has no flow vector remaining after the subtraction according to the scalar quantity of the flow vector subtracted by the flow vector subtraction unit 170, in particular, the horizontal component of the screen (flow vector = 0 or a predetermined value near zero). Within range) A plurality of block groups 4 are determined to be stationary objects 3. Accordingly, the scalar quantity of the flow vector subtracted by the flow vector subtraction unit 170 is not 0, that is, the horizontal flow of the representative flow vector of the block group 4 and the corrected flow vector of the part where the block group 4 is in contact with the ground. If the direction component is different, the moving object 2 is determined. Thus, the moving object 2 and the stationary object 3 can be distinguished.

  The moving object determination unit 174 is corrected by the flow vector subtraction unit 170 that is not the stationary object 3 by the stationary object determination unit 172, that is, the grounding position and size of the block group 4 determined to be the moving object 2. The position, traveling direction, and speed of the moving object 2 constituting the block group 4 are determined according to the subsequent (after subtraction) flow vector.

  The collision determination unit 176 determines whether or not the own vehicle 1 and the moving object 2 are likely to collide according to the traveling direction and speed of the own vehicle 1 and the position, traveling direction, and speed of the identified moving object 2. To determine. Since various existing techniques can be applied to the collision determination, detailed description thereof is omitted here.

  When the collision determination unit 176 determines that the possibility of a collision is high, the notification unit 178 notifies the driver to that effect through a buzzer (voice output means) and a warning light (display 122). In addition, information indicating that is output to the vehicle control device 130 through the I / F unit 150.

  The image holding unit 180 overwrites the previous image data of the data holding unit 152 with the current image data. Thus, in the next frame, the current image data can be used as the previous image data.

(External vehicle environment recognition method)
FIG. 14 is a flowchart showing the overall processing flow of the vehicle exterior environment recognition method. FIG. 14 shows an overall flow related to an interrupt process when image data captured through a wide-angle lens is transmitted from the imaging device 110.

  When the interruption by the vehicle exterior environment recognition method occurs, the I / F unit 150 acquires the current image data transmitted from the imaging device 110 (S300). The image cutout unit 160 then selects a partial image 202 (for example, horizontal 640 pixels) having a predetermined position and size from the environment image 200 (for example, horizontal 640 pixels × vertical 480 pixels) based on the acquired current image data. X vertical 128 pixels: horizontal 80 blocks x vertical 16 blocks) are cut out (S302). Next, the flow vector deriving unit 162 derives a plurality of flow vectors in the partial image 202 (S304). Then, the grouping unit 164 sequentially compares the flow vectors of adjacent blocks from the flow vector derived by the flow vector deriving unit 162, groups the blocks whose flow vector directions and sizes are approximated, and represents the representative value thereof. A representative flow vector is obtained (S306).

  Subsequently, the movement state acquisition unit 166 acquires the movement speed and angular velocity of the host vehicle 1 at that time (S308), and the correction flow vector extraction unit 168 refers to the correction table, and according to the movement speed and angular velocity, One correction flow vector is extracted from the data holding unit 152 (S310). The flow vector subtracting unit 170 subtracts the flow vector of the part corresponding to the representative part of each block group 4 from the corrected flow vector from each representative flow vector derived by the grouping part 164 (S312). The stationary object determination unit 172 identifies the stationary object 3 according to the scalar quantity of the flow vector subtracted by the flow vector subtraction unit 170, and distinguishes the stationary object 3 from the moving object 2 (S314). Then, the moving object determination unit 174 configures the block group 4 according to the size of the block group 4 determined as the moving object 2 by the stationary object determination unit 172 and the speed of the block group 4. The position, traveling direction, and speed of the moving object 2 are determined (S316).

  The collision determination unit 176 determines whether or not the own vehicle 1 and the moving object 2 are likely to collide according to the traveling direction and speed of the own vehicle 1 and the position, traveling direction and speed of the identified moving object 2. Determination is made (S318). As a result, when it is determined that there is a high possibility of collision with the moving object 2 (YES in S318), the notification unit 178 notifies the driver that the possibility of collision is high and notifies the vehicle control device 130 to that effect. Is output (S320). If it is not determined that the possibility of collision with the moving object 2 is high (NO in S318), the process proceeds to step S322.

  Finally, in order to use the current image data as the previous image data in the next frame, the image holding unit 180 overwrites the previous image data in the data holding unit 152 with the current image data (S322).

  As described above, according to the outside environment recognition device 120 and the outside environment recognition method as described above, the moving object 2 can be detected with high accuracy and low processing load even when the host vehicle 1 turns, from the front side. It is possible to accurately predict the collision time between the approaching moving object 2 and the host vehicle 1.

  Conventionally, in order to appropriately determine the moving object 2, a stereo camera or a laser radar must be used in order to obtain a distance to the stationary object 3, but in this embodiment, the influence of the behavior of the host vehicle 1 is affected. Since the calculation is performed only by subtraction of the ground flow vector (correction flow vector) and the correction flow vector is calculated in advance, it is possible to reduce the cost and the processing load.

  Also provided are a program that causes a computer to function as the vehicle exterior environment recognition device 120 and a computer-readable storage medium that stores the program, such as a flexible disk, magneto-optical disk, ROM, CD, DVD, and BD. Here, the program refers to data processing means described in an arbitrary language or description method.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to this embodiment. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Is done.

  For example, in the above-described embodiment, the image cutout unit 160, the flow vector derivation unit 162, the grouping unit 164, the movement state acquisition unit 166, the corrected flow vector extraction unit 168, the flow vector subtraction unit 170, the stationary object determination unit 172, The moving object determination unit 174, the collision determination unit 176, the notification unit 178, and the image holding unit 180 are configured to be operated by software by the central control unit 154. However, the functional unit described above can be configured by hardware.

  Note that each step of the vehicle environment recognition method of the present specification does not necessarily have to be processed in time series in the order described in the flowchart, and may include processing in parallel or a subroutine.

  The present invention relates to an outside environment recognition device and an outside environment recognition method for recognizing an environment outside the host vehicle, and more particularly, an outside environment recognition device and an outside environment that recognize a moving object outside the host vehicle and determine a collision with the host vehicle. It can be used as a recognition method.

DESCRIPTION OF SYMBOLS 1 ... Own vehicle 2 ... Moving object 3 ... Stationary object 110 ... Imaging device 120 ... Outside vehicle environment recognition apparatus 150 ... I / F part (image data acquisition part)
152 ... Data holding unit 160 ... Image cropping unit 162 ... Flow vector deriving unit 164 ... Grouping unit 166 ... Movement state acquisition unit 168 ... Correction flow vector extraction unit 170 ... Flow vector subtraction unit 172 ... Stationary object determination unit 174 ... Movement Object determination unit 176 ... collision determination unit 202 ... partial image

Claims (4)

An image data acquisition unit for acquiring image data obtained by imaging the environment outside the vehicle through the imaging device;
A flow vector deriving unit for deriving a flow vector for each block obtained by dividing the image based on the image data;
A grouping unit that approximates the direction of the flow vector and the size thereof, groups adjacent blocks together to generate a block group, and obtains a representative flow vector serving as a representative value;
A moving state acquisition unit that acquires a behavior at the time of turning of the imaging device;
A correction flow vector extraction unit that extracts a flow vector of the ground according to the behavior during turning acquired by the movement state acquisition unit;
A flow vector subtracting unit that subtracts a flow vector of a part corresponding to a representative part located at a boundary where the degree of change of the flow vector in the block group of the ground flow vector changes from the representative flow vector;
In the block group, if the scalar quantity is 0 or within a predetermined range near 0 according to the scalar quantity of the flow vector after subtraction , the block group is regarded as a stationary object , and the scalar quantity is 0 or 0. if not within the predetermined range in the vicinity, and stationary judgment unit shall be the moving object to the block group,
A vehicle exterior environment recognition device comprising: A data holding unit for holding a plurality of ground flow vectors respectively corresponding to a plurality of different turning behaviors;
The correction flow vector extraction unit extracts the ground flow vector from the plurality of ground flow vectors held in the data holding unit according to the turning behavior acquired by the movement state acquisition unit. The outside environment recognition device according to claim 1 characterized by things.   A collision determination unit that performs a collision determination between the moving object and the host vehicle using the flow vector after the subtraction of the block group set as a moving object in the stationary object determination unit as the flow vector of the moving object; The external environment recognition device according to claim 1 or 2, characterized in that Obtain image data that captures the environment outside the vehicle through the imaging device,
Deriving a flow vector for each block obtained by dividing an image based on the image data;
The direction of the flow vector and the size thereof are approximate, and adjacent blocks are grouped to generate a block group, and a representative flow vector serving as a representative value is obtained,
Obtaining the behavior of the imaging device when turning,
According to the behavior at the time of turning, the flow vector of the ground is extracted,
From the representative flow vector, subtract the flow vector of the part corresponding to the representative part located at the boundary where the change degree of the flow vector in the block group of the ground flow vector changes ,
In the block group, if the scalar quantity is 0 or within a predetermined range near 0 according to the scalar quantity of the flow vector after subtraction , the block group is regarded as a stationary object , and the scalar quantity is 0 or 0. if not within the predetermined range in the vicinity outside the vehicle environment recognizing method comprising moving object and to Rukoto the block group.

Images