JP4830822B2 - Object detection apparatus and object detection method - Google Patents

Description

  The present invention relates to an object detection device and an object detection method.

A conventional object detection apparatus is described in, for example, Patent Document 1 below.
In the following prior art, in order to detect a moving body, an optical flow (a vector indicating a motion on an image) is calculated from a captured image of a vehicle-mounted camera. In addition, the movement of the vehicle is estimated from the rotational speed of the vehicle wheel and the steering angle of the steering wheel. Further, a distance model to the object existing in front of the host vehicle is calculated to generate a space model that models the space in front of the host vehicle. Then, based on the calculated optical flow, the movement of the vehicle, and the space model, a moving body is detected from the captured image.
JP 2004-56763 A

  In the above prior art, a space model ahead of the host vehicle is generated, and a moving object is detected from the captured image based on the space model and the movement of the host vehicle. For this reason, since it takes a processing time to estimate the motion of the host vehicle and to calculate the distance to the object, there is a problem that it is difficult to detect the moving object at high speed.

The present invention extracts a feature point from an image ahead of the host vehicle imaged by the imaging means, calculates movement speed information on an image of a pixel representing the extracted feature point, is adjacent to the image in the vertical direction, and moves Pixels having velocity information in a predetermined range are grouped. Then, the position coordinates of the pixel located at the uppermost part and the pixel located at the lowermost part of the grouped pixels are converted into position coordinates on an overhead view plane which is a horizontal plane in the real space and a plane in the vehicle front-rear direction. Further, based on the conversion result, it is determined whether the grouped pixel is a planar object or a three-dimensional object, and based on the relationship between the horizontal position coordinate and the moving speed information in the image of the grouped pixel determined to be a three-dimensional object. To determine whether the object is a stationary object or a moving object.

According to the present invention, from the position coordinates that are grouped based on the moving speed information on the image representing the feature point of the captured image and converted to the overhead plane of the top and bottom pixels of each grouped pixel. Since the moving object is determined based on the relationship between the horizontal position coordinate of the pixel determined to be a three-dimensional object and the moving speed information, the plane object and the three-dimensional object are determined. The moving object can be detected at high speed without calculating the distance.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram illustrating a configuration example of an object detection apparatus according to the first embodiment.

  The object detection device 10 is mounted on a vehicle and stored in a camera 101 that is an imaging unit that images the front of the vehicle, an image temporary recording unit 102 that temporarily records an image captured by the camera 101, and an image temporary recording unit 102. A feature point extraction unit 103 that extracts a feature point by performing image processing on the captured image; a coordinate conversion unit 104 that converts the position coordinates of the feature point extracted by the feature point extraction unit 103 into a position coordinate on a three-dimensional image; A feature information based on the velocity value of the feature point calculated by the movement information calculation unit 105 and the movement information calculation unit 105 that calculates the velocity and direction on the image of the feature point extracted by the feature point extraction unit 103 as movement information. The grouping unit 106 that groups points, the information of the three-dimensional coordinates converted by the coordinate conversion unit 104, and the special information calculated by the movement information calculation unit 105 When the feature point is a plane object or a three-dimensional object based on the point movement information and the grouping information of the feature points grouped by the grouping unit 106, and when the feature point is a three-dimensional object Includes a control unit 100 having an object attribute determination unit 107 that determines whether or not the object is a moving object.

  The camera 101 is a camera having an image sensor such as a CCD or a CMOS, for example, and continuously images the front of the vehicle and outputs an image captured for each frame to the image temporary recording unit 102. An image captured by the camera 101 is temporarily recorded in the image temporary recording unit 102. Here, as shown in FIG. 2, the camera 101 is installed in the upper front part of the vehicle interior, its optical axis LS is directed in the front front direction (Z direction) of the vehicle, and the horizontal axis X (not shown) of the imaging surface is the road surface. The vertical axis Y (not shown) of the imaging surface is set to be perpendicular to the road surface.

  An example of an image captured by the camera 101 (an image ahead of the host vehicle) is shown in FIG. An image captured by the camera 101 is represented by an xy coordinate system in which the upper left of the image is the origin and the x axis is from left to right and the y axis is from top to bottom. Note that in FIG. 3, the captured image includes a boundary line of a runway such as a curb, a white line, and an outer wall installed on the left and right side of the runway, and a pedestrian moving from left to right.

  The feature point extraction unit 103 reads the image captured by the camera 101 from the image temporary recording unit 102, and binarizes the read captured image using a predetermined threshold value, thereby detecting the edge of the object present in the image. Extract. FIG. 4A shows an example of the extracted vertical edge. Next, thinning processing is performed on each extracted edge to narrow the edge width, and the center of the edge is accurately set (see FIG. 4B). Further, the edge is expanded in the horizontal direction so that the edge width of the thinned edge becomes a constant width, for example, a width corresponding to three pixels (see FIG. 4C). By this operation, the extracted edges are normalized, and an edge image having a uniform width for each edge can be obtained.

  The movement information calculation unit 105 updates the counter value of the pixel counter of the pixel corresponding to the edge. Here, the pixel counter is a counter set for each pixel, and when the pixel corresponds to an edge, the counter value of the pixel counter is incremented by +1, and when the pixel does not correspond to the edge, the counter value of the pixel counter Is a counter initialized to 0. The counter value updating process is performed for each frame continuously captured by the camera 101. As a result of this operation, the counter value of the pixel counter increases for pixels with a long time corresponding to the edge, and the counter value of the pixel counter decreases for a pixel with a short time corresponding to the edge.

  Since the change in the counter value of the pixel counter represents the moving direction and moving amount of the edge, the moving direction and moving speed of the edge on the captured image can be calculated from the counter value. Details will be described below.

  FIG. 4 is a diagram illustrating a specific example of processing performed to obtain an edge image by normalizing the extracted edges.

  First, binarization processing is performed on the edge image. The binarization process is a process in which a pixel at a position where an edge is detected is set to 1 and a pixel at a position where no edge is detected is set to 0. A binarized image as shown in FIG. 4A is generated by binarization processing.

  Next, thinning processing is performed on the generated binary image. The thinning process is a process of reducing the edge width of the detected edge until a predetermined pixel width is reached. In FIG. 4B, the edge width of the edge is thinned until the predetermined pixel width reaches 1 pixel. Then, the center position that is the center of the edge is set by thinning the edge until a predetermined pixel width is obtained. Here, as an example, the case of thinning to one pixel is described, but thinning may be performed to other numbers of pixels.

  Next, an expansion process is performed to expand the edge width of the thinned edge. The expansion process is a process of expanding the edge width from the center position set by thinning toward the edge moving direction and expanding the edge width from the center position to the opposite direction to the edge moving direction. For example, in FIG. 4C, one pixel is expanded from the edge center position x0 to the edge movement direction (the positive direction of the x-axis), and the edge movement direction from the edge center position x0 is opposite to the edge movement direction (the x-axis direction). 1 pixel is expanded in the negative direction), and the edge width is expanded to 3 pixels.

  By performing the thinning process and the expansion process in this way, the edge width of the extracted edge image is standardized to a predetermined width in the edge moving direction.

  Next, a count-up process is performed on the edge whose edge width is standardized. The count-up process is a process for counting up the value of the memory address at the position where the edge is detected and initializing the value of the memory address at the position where the edge is not detected.

  Hereinafter, the edge count-up process will be described with reference to FIGS. Here, for the sake of simplicity, it is assumed that the edge moves in the positive direction of the x axis. Note that the edge can be similarly described when moving in the negative x-axis direction, the y-axis direction, or two-dimensionally.

  As shown in FIG. 4C, the edge has a center position of the edge at a position x0 in a certain frame, a position x0 + 1 of one pixel in the moving direction of the edge from the center position, and opposite to the moving direction of the edge from the center position. In the direction, it is expanded to a position x0-1 of one pixel.

  In such a case, the count values at the positions x0-1, x0, and x0 + 1 where the edge is detected are incremented by 1, and the count value at the position where the edge is not detected is reset. For example, in FIG. 4D, since an edge is detected at positions x0-1, x0, x0 + 1 at time t, the count is incremented by 1 at each position, the count value at position x0 + 1 is 1, and position x0 Is 3, and the count value at position x0-1 is 5.

  Then, as shown in FIG. 4E, since the edge does not move even at time t + 1, the edge is detected at each of the positions x0-1, x0, x0 + 1, and the count values at the positions x0-1, x0, x0 + 1 are detected. Is further incremented by 1 so that the count value at position x0-1 is 2, the count value at position x0 is 4, and the count value at position x0 + 1 is 6.

Further, as shown in FIG. 4 (f), at time t + 2, the edge is shifted by one pixel in the positive direction of the x axis, and the edge is detected at positions x0, x0 + 1, and x0 + 2.
Accordingly, the count values at the positions x0, x0 + 1, and x0 + 2 where the edge is detected are counted up, and the count values at the position x0-1 where the edge is not detected are reset.

  As a result, the count value at position x0 + 2 is 1, the count value at position x0 + 1 is 3, and the count value at position x0 is 5, as shown in FIG. Further, the count value at the position x0-1 where no edge is detected is reset to zero.

  In this way, the count value at the position where the edge is detected is counted up, and the count value at the position where the edge is not detected is reset.

  In FIG. 4, as the position for detecting the count value, the center position (x0) of the edge, the position of one pixel (x0 + 1) from the center position to the edge movement direction, and the direction from the center position to the edge movement direction are opposite. The count value is detected at three positions of the position (x0-1) of one pixel. If the slope of the count value described later is obtained, the number of counts can be counted as long as it is two or more with respect to the edge moving direction. The value may be detected.

  If the frame rate is set sufficiently higher than the speed at which the edge moves, the edge is detected a plurality of times at the same position between consecutive frames.

  For example, in the example of FIG. 4, two edges at time t and time t + 1 are detected at the position x0. Therefore, when the count value at the position where the edge is detected is counted up, the count value becomes equal to the time (number of frames) during which the edge is detected at that position. In particular, the smallest count value h of the edge count values represents how many frames the edge has been in the same position after moving.

  Next, the moving speed, moving direction and position of the edge are calculated.

  First, the inclination of the count value in the movement direction is calculated, and the movement direction, movement speed, and position of the edge are calculated based on this inclination.

  For example, in the case of FIG. 4E, the count values at positions x0-1, x0, and x0 + 1 are 6, 4, and 2, respectively.

  Therefore, by subtracting the count value 2 of x0 + 1 from the count value 6 of the position x0-1, the slope of the count value can be calculated as H = (6-2) / 2 = 2.

this is,
H = {(time from the edge moving to the position x0-1 to the present)-(time after the edge has moved to the position x0 + 1)} / (2 pixels)
Thus, the time (number of frames) required for the edge to pass through one pixel at the position x0 is calculated.

  Accordingly, the slope H of the count value determines how many frames it takes for the edge to move by one pixel, and the edge moving speed 1 / H can be calculated based on the slope H of the count value. .

  In FIG. 4 (e), two frames are required to move one pixel, so the edge moving speed can be calculated as 1/2 (pixel / frame).

Similarly, in FIG. 4F, H = (5-1) / 2 = 2, so that the edge moving speed is ½ (pixel / frame).

  Further, the edge moving direction can be determined by the magnitude of the count value. The count value at the position where the edge is moved and the edge is newly detected is 1, which is the smallest value among the count values at each position.

  Therefore, the count value in the direction in which the edge moves is small, and the count value in the direction opposite to the direction in which the edge moves is large, so that the edge moving direction can be determined.

Furthermore, if the frame rate is set sufficiently higher than the speed at which the edge moves, it can be assumed that the detection target object moves at a constant speed. In addition, the smallest count value h among the count values at the current position represents the time when the edge is detected at the position, that is, how many frames the edge has moved to stay at the same position. ing.
As a result, the position of the edge is determined by assuming that the center position of the edge is x0, the position of the edge = x0 + h / H.
It can ask for.

For example, in FIG. 4F, the edge speed is ½ (pixel / frame), and since the edge is detected at the same position continuously for one frame at time t + 2, the edge position at time t + 2 is It can be calculated that 1 (frame) × {1/2 (pixel / frame)} = 0.5 pixel is moved from the position x0.

  From the above, it is possible to count up the count value at the position where the edge is detected and calculate the moving speed and moving direction of the edge based on the slope of the counted up count value.

  Next, a velocity image is generated by classifying the velocity components of the edges present on the captured image into predetermined class values. As shown in FIG. 5, the velocity image in the present embodiment represents the edge pixels where the velocity is detected as a round dot, and the pixel having a higher moving velocity indicates a larger dot. Further, the moving direction is represented by expressing the speed toward the right as a black point and the speed toward the left as a white point. In FIG. 5, the speed toward the right side of the image is detected from the curb and the white line on the right side of the traveling path of the host vehicle, and the speed toward the left side of the image is detected from the outer wall on the left side of the traveling path. For a pedestrian who moves from the left side to the right side of the travel path, the speed toward the right side of the image is detected.

  The grouping unit 106 sets a region for dividing the speed image in order to extract a three-dimensional object from the calculated speed image. That is, as shown in FIG. 6, a plurality of strip-shaped areas are set on the speed image, and the speed image is divided into the plurality of areas.

  Next, a three-dimensional object is detected by grouping pixels that are continuous at the same speed in the vertical direction of the image for each region. In other words, each area is scanned from the bottom to the top of the image, and if there is a pixel with speed in the area, the speed difference with the pixel with speed adjacent to that pixel is compared and the speed is compared. When the difference is equal to or less than the threshold value T1, it can be estimated that the objects are moving at the same speed with respect to the vehicle. Then, the upper end position (the uppermost part in the claims) and the lower end position (the lowermost part in the claims) of the pixels grouped at the same speed are detected. By this process, lower ends BL1 to BL11 and upper ends TL1 to TL11 in FIG. 6 are detected.

The coordinate conversion unit 104 has position coordinates of lower end points BP1 to BP11 (not shown) and upper end points TP1 to TP11 (not shown) having the coordinates of the center positions of the lower ends BL1 to BL11 and the upper ends TL1 to TL11 extracted on the xy plane. As a point on a ZX coordinate having a specified area (hereinafter referred to as a specified ZX coordinate), a three-dimensional position coordinate (a position coordinate in an overhead plane that is a plane in the horizontal direction and the vehicle front-rear direction in real space) Convert to

  Here, the coordinates of each point are (x, y), the height from the road surface of the camera is Ch (m), the depression angle of the camera is Tr (rad), the vertical size of the image is Ih, and the horizontal size of the image is Iw. If the angular resolution per pixel in the height direction is PYr (rad) and the angular resolution per pixel in the horizontal direction is PXr (rad), the points TP1 to TP11 and BP1 to BP11 on the xy plane are Is converted into coordinates (Z, X) on the ZX plane.

(Formula 1) Z = (Ch) / (TAN (Tr + (y−Ih / 2) × PYr))
(Formula 2) X = x × TAN ((Z-Iw / 2) × PXr)
The converted points of the upper end points TP1 to TP11 and the lower end points BP1 to BP11 are referred to as upper end point coordinate conversion points RT1 to RT11 and lower end point coordinate conversion points RB1 to RB11.

  The object attribute determination unit 107 determines in which region the coordinate conversion points RT1 to RT11 of the upper end point and the coordinate conversion points RB1 to RB11 of the lower end point are located among the regions set by dividing the prescribed ZX plane. Perform (see FIG. 7). Here, as the vehicle moves up and down, the captured image of the camera 101 also moves up and down, and the region where the coordinate conversion point is located may also change. In order to avoid this influence, it is desirable to set the area to be set by dividing the ZX plane in meter order. In this embodiment, the x-axis direction is −5.25 m> x, −5.25 ≦ x <−3.5 m, −3.5 m ≦ x <−1.75 m, −1.75 m ≦ x <0 m, 0 m ≦ x <1.75 m, 1.75 m ≦ x <3.5 m, 3.5 m ≦ x <5.25 m, 5.25 m ≦ x, and the z-axis direction is 0 ≦ z <10 m, 10 m ≦ z <20 m, 20 m ≦ z <30 m, 30 m ≦ z <40 m, 40 m ≦ z <50 m, and divided into 5 regions 11 to 15, regions 21 to 25, regions 31 to 35, regions 41 to 41 Region 45, region 51 to region 55, region 61 to region 65, region 71 to region 75, and region 81 to region 85 are set. Further, the region 11 to the region 15 are the region 10, the region 21 to the region 25 are the region 20, the region 31 to the region 35 are the region 30, the region 41 to the region 45 are the region 40, and the region 51 to the region 55 are the region 50. The region 61 to the region 65 are the region 60, the region 71 to the region 75 are the region 70, and the region 81 to the region 85 are the region 80.

  If the coordinate transformation point of the upper end point and the coordinate transformation point of the lower end point of the same group as the upper end point in the same vertical region of the xy plane are located in the same region of the prescribed ZX plane, the upper end point and the lower end point Is on the road surface, that is, is a plane object on the road surface. In addition, if the coordinate transformation point of the upper end point is located outside the prescribed ZX plane and the coordinate transformation point of the lower end point of the same group as the upper end point is located in the region of the prescribed ZX plane, only the lower end point is on the road surface. That is, it is determined that the object is a three-dimensional object. Here, in order to detect the road boundary, +1 is added to the counter value of the counter in the area where the coordinate conversion point of the lower end point is located among the counters set in each area, and the position information distribution of the lower end point is calculated.

  For example, in this embodiment, the coordinate transformation points RT1 to RT5 of the upper end points TP1 to TP5 are projected outside the prescribed ZX plane shown in FIG. 7, and therefore the group including the upper end points TP1 to TP5 is determined to be a three-dimensional object. The edges including the upper end points TP1 to TP5 are determined as the three-dimensional objects OB1 to OB5 on the xy plane. Since the coordinate transformation points RT6 to RT11 of the upper end points TP6 to TP11 are located in the same area as the coordinate transformation points RB6 to RB11 of the lower end points BP6 to BP11, the group including the upper end points TP6 to TP11 is located on the road surface. It is determined that there is. Here, in order to detect the road boundary, +1 is added to the counter value of the counter in the area where the coordinate conversion points RB1 to RB11 of the lower end point are located.

  And as shown in FIG. 8, the area | region where possibility that a track boundary line exists is extracted from the position distribution of the coordinate conversion point of the obtained lower end point. In other words, in the obtained position distribution of the coordinate transformation point of the lower end point, if there are counter values in a plurality of z-axis regions in the same x-axis region, a road boundary exists linearly along the vehicle in front of the host vehicle. Since it can be estimated, an area with a counter value is extracted as a road boundary area. For example, in FIG. 8, since there are counter values in a plurality of z-axis regions (regions where the lower end points RB1, RB4, and RB5 are located) in the region 30 in the same range of the x-axis, these regions are used as runway boundary regions. Extract. The same applies to the region 50 and the region 80.

  Then, it is determined whether or not a straight line indicating a road boundary exists in the extracted area (see FIG. 8). That is, based on the coordinate conversion point of the lower end point existing in the extracted region, regression analysis is performed in the xy coordinate system, and the slope of the straight line passing through the lower end point is calculated. For example, in the region 30 of this embodiment, the regression analysis in the xy coordinate system is performed from the coordinate positions of the lower end points BP1, BP4, and BP5 in the xy coordinates corresponding to the coordinate conversion points RB1, RB4, and RB5 of the lower end points existing in the region 30. To calculate a slope a3 of a straight line passing through the lower end points BP1, BP4, and BP5. Similarly, straight line inclinations a5 and a8 corresponding to the coordinate conversion points of the lower end points located in the regions 50 and 80 are calculated. Then, if the calculated slope of the straight line is within a predetermined range, it is determined that a straight line indicating a road boundary exists in the extracted region. That is, the points (for example, PL1 to PL5 in FIG. 8) whose coordinates are the left end coordinates of the x-axis region where the coordinate conversion point of the lower end point is located and the representative coordinates (for example, center coordinates) of each Z-axis region are converted into the xy coordinate system. Then, the straight line slope Tn0a1 calculated by regression analysis, the right end coordinate of the x-axis area where the coordinate conversion point of the lower end point is located, and a point having the coordinates (for example, the center coordinates) of each Z-axis area as coordinates (for example, FIG. The slope of the straight line an calculated based on the lower end point is included in the range of the slope defined by the slope Tn0a2 of the straight line calculated by converting the PR1 to PR5) of 8 into the xy coordinate system and performing regression analysis. In this case, it is determined that the straight line Ln indicating the road boundary exists in the extracted area.

  For example, in the region 30 of the present embodiment, the point PL1 with coordinates (x, z) = (− 3.5, 5), the point PL2 with coordinates (x, z) = (− 3.5, 15), the coordinates Point PL3 with (x, z) = (− 3.5, 25), point PL4 with coordinates (x, z) = (− 3.5, 35), coordinates (x, z) = (− 3.5, 45) The point PL5 of 45) is converted into xy coordinates, and regression analysis is performed to calculate the slope T30a1 of the straight line connecting the points, the point PR1 of coordinates (x, z) = (− 1.75, 5), the coordinates (x , Z) = point PR2 of (−1.75, 15), point PR3 of coordinates (x, z) = (− 1.75,25), coordinates (x, z) = (− 1.75,35) Point PR4, coordinate PR (x, z) = (− 1.75, 45) is converted to xy coordinates, and regression analysis is performed to calculate the slope T30a2 of the straight line connecting the points. Since the slope a3 of the straight line connecting the lower end points BP1, BP4, BP5 on the xy plane corresponding to the conversion points RB1, RB4, RB5 is within the range defined by T30a1 and T30a2, the lower end points BP1, BP4, BP5 Is determined to be the straight line L3 indicating the road boundary (see FIG. 9). Similarly, in the region 50, it is determined that the straight line connecting the lower end points BP6 and BP7 on the xy plane corresponding to the coordinate transformation points RB6 and RB7 of the lower end point is the straight line L5 indicating the road boundary. The straight line connecting the lower end points BP8 to BP11 on the xy plane corresponding to the coordinate conversion points RB8 to RB11 of the point is determined to be the straight line L8 indicating the road boundary.

  Then, as shown in FIG. 9, a graph indicating the relationship between the horizontal position and the speed of the detected three-dimensional object image is calculated in each region delimited by the road boundary. In this graph, a three-dimensional object located on the left side of the center of the image, for example, a background such as an outer wall, is located farther from the host vehicle as it is located on the right side of the image, so the pixel speed is reduced. In other words, if the vertical axis is set to speed (plus the speed to move to the left, minus the speed to move to the right) and the x axis of the image (the left end of the image is the origin), In the left half of the image, the relationship between the speed and the x coordinate is a straight line descending to the right. On the other hand, if there is a coordinate that does not have a straight line relationship that falls to the right, there may be a non-stationary object, that is, a moving object in the region. In this case, as shown in FIG. 9, when a coordinate having a speed (right speed) opposite to the background speed (left speed) is detected, the moving body moving in the direction opposite to the background is detected. It is determined that it exists. Further, if a coordinate having a speed faster than the background is detected, it is determined that a moving object exists.

  For example, in this embodiment, based on the road boundary detected in the area 30, the area 50, and the area 80, the left side of the road boundary L3, that is, the left side of the area 10, the area 20, and the area 30 is defined as a section a. And set a section b between the runway boundary L3 and the runway boundary L5, that is, a part on the right side of the region 30, a part on the left side of the region 40, and the region 50, and a section b between the runway boundary L5 and the runway boundary L8. A part on the right side of the area 50, that is, a part on the left side of the area 60, the area 70, and the area 80 is set as a section c, and a part on the right side of the runway boundary L8, that is, the right side of the area 80 is defined as the section d. OB2 that moves to the right by determining OB1, OB4, and OB5 as stationary three-dimensional objects (here, outer walls) from the relationship between the horizontal position and speed of the three-dimensional objects OB1 to OB5 existing in the section b. , OB3 and moving body (here pedestrian) A constant.

  Also, for the three-dimensional object existing on the right side of the image, the relationship between the speed (plus the speed to move to the left and minus the speed to move to the right) and the x coordinate of the image (the right end of the image is the origin) in the same way as the left side. If it is not a straight line relationship that rises to the right, it is determined that there is a high possibility that a solid object exists in the area, and if a speed opposite to the background is detected, the background is reversed. A moving body that moves, for example, a pedestrian that moves from the right side to the left side of the image can be detected.

  Further, by using the vehicle behavior (the traveling direction of the vehicle), as described above, it is determined whether it is necessary to separately detect the moving body on the left and right of the captured image, or whether the moving body can be detected in the entire captured image. It is also possible. For example, when the vehicle is steered to the right, the area corresponding to the background of the front image is detected only at the speed of moving to the left side, so that the moving object is detected on the left and right of the captured image as described above. It is possible to detect the moving body in a lump for the entire captured image without performing the process. The same applies when the vehicle is steered to the left.

  Here, as described above, since the moving object is determined by comparing the horizontal position and speed of the three-dimensional object corresponding to the background and the moving object, for example, only the three-dimensional object corresponding to the pedestrian can be detected. If a three-dimensional object such as the corresponding outer wall cannot be extracted, the moving object cannot be determined. In this case, when it is determined again as a three-dimensional object in the next detection, it is determined whether or not it is a moving body from the speed change (described later).

FIG. 10 is a flowchart showing processing of the object detection apparatus 10 in the present embodiment.
This process is executed as a program that is activated when an ignition switch (not shown) is turned on.

  In step S <b> 101, an image ahead of the host vehicle captured by the camera 101 and recorded in the image temporary recording unit 102 is output to the feature point extraction unit 103 at a predetermined cycle. After this, the flow moves to step S102.

  In step S102, the feature point extraction unit 103 performs edge extraction processing on the image, extracts the outline of the object existing in the captured image as an edge image, and normalizes the edge image. After this, the flow moves to step S103.

  In step S103, the movement information calculation unit 105 calculates the edge speed, and calculates a speed image in which the calculated speed is represented by a predetermined gradation. After this, the flow moves to step S104.

  In step S104, a strip area for object detection is set on the velocity image calculated by the grouping unit 106. After this, the flow moves to step S105.

  In step S105, the grouping unit 106 checks whether there is a pixel having speed in each strip area from the bottom to the top. When there are pixels having speed, the grouping unit 106 performs grouping on the assumption that these pixels represent the same object. After this, the flow moves to step S106.

  In step S106, for each group in each strip region, the center coordinate of the pixel located at the top is set as the upper end point, and the center coordinate of the pixel located at the bottom is set as the lower end point. After this, the flow moves to step S107.

  In step S107, the coordinate conversion unit 104 converts the coordinates of the detected upper end point and lower end point to the ZX plane using (Expression 1) and (Expression 2). After this, the flow moves to step S108.

  In step S108, the object attribute determination unit 107 determines which region of the prescribed ZX plane in which the position of the coordinate transformation point of the upper end point and the coordinate transformation point of the lower end point of the same group as the upper end point defines the x-axis range and the z-axis range. If both the coordinate transformation point of the upper end point and the coordinate transformation point of the lower end point of the same group as the upper end point are located in the same area of the prescribed ZX plane, the upper end point and the lower end point are If the object is determined to be a plane object, and only the coordinate transformation point of the lower end point of the same group as the upper end point is located on the specified ZX plane, the object including the upper end point and the lower end point is determined to be a three-dimensional object To do. Also, the counter value of the counter in the area where the lower end point is located is incremented by +1. After this, the flow proceeds to step S109.

  In step S109, the object attribute determination unit 107 determines whether the object including the upper end point and the lower end point is a planar object or a three-dimensional object for all detected upper and lower end points (hereinafter, object attribute determination). Is called). When the object attribute determination is performed for all the detected upper and lower end points, the process proceeds to step S110. On the other hand, when the object attribute determination is not performed for all the detected upper and lower end points, the process returns to step S105.

  In step S110, the object attribute determination unit 107 determines that the region where the counter value exists in a plurality of z-axis regions in the same x-axis region from the position distribution of the coordinate conversion point of the lower end point on the ZX plane has a road boundary line. It is extracted as a region that is likely to be. After this, the flow proceeds to step S111.

  In step S111, the object attribute determination unit 107 performs regression analysis in the xy coordinate system on the lower end point of the xy plane corresponding to the coordinate conversion point of the lower end point existing in the extracted region, and a straight line connecting the lower end points. Is calculated. After this, the flow proceeds to step S112.

  In step S112, the straight line slope calculated in step S111 by the object attribute determination unit 107 is a point having the coordinates of the left end coordinate of the x-axis region and the representative coordinate of each z-axis region in the region where the lower end point is located (see FIG. 8 and the straight line connecting the right end coordinate of the x-axis region and the point having the representative coordinate of each z-axis region (PR1 to PR5 in FIG. 8) as the coordinate range. For example, it is determined that a straight line indicating a road boundary exists in the extracted region, and the straight line calculated in step S111 is detected as a straight line indicating the road boundary. After this, the flow proceeds to step S113.

  In step S113, the object attribute determination unit 107 determines whether or not straight lines indicating all the road boundaries have been detected in the extracted region. If a straight line indicating all the road boundaries is detected, the process proceeds to step S114. On the other hand, when the straight line which shows all the track boundaries is not detected, it returns to step S110.

  In step S114, the object attribute determination unit 107 calculates the relationship between the speed and the x coordinate between three-dimensional objects (outer walls, pedestrians) existing in each section, for each section divided by the road boundary, If there is a three-dimensional object in which the speed in the reverse direction is detected, it is determined that there is a moving object that approaches the host vehicle, and the flow proceeds to step S116. On the other hand, if there is no object whose speed in the direction opposite to the background is detected, it is determined that there is a possibility that a moving object candidate exists, and the process proceeds to step S115.

  In step S115, the object attribute determination unit 107 determines whether the moving object candidate is a moving object. If there is a moving object candidate in the previous section in which it is determined that a moving object candidate exists, the previous speed and the current speed are compared. When the moving object candidate is a stationary object, the moving object candidate approaches the own vehicle as the own vehicle moves forward, and the moving speed of the pixels corresponding to the moving object candidate becomes faster. Therefore, if the speed of the coordinate conversion point at the lower end point is faster than the previous time, it is determined that there is no moving object, and the speed of the coordinate conversion point at the lower end point is not faster than the previous time. Determines that there is a moving object. After this, the flow proceeds to step S116.

  In step S116, it is determined whether or not a moving object has been detected in all sections delimited by the road boundary. If all moving objects have been detected, the flow proceeds to step S117. On the other hand, when not all the moving bodies have been detected, the flow returns to step S114.

  In step S117, it is determined whether or not the ignition switch of the host vehicle has been turned off. If the ignition switch is not turned off, the process returns to step S101 and is repeated. On the other hand, if the ignition switch is turned off, the flow moves to step S118 and the process ends.

  According to the object detection apparatus of the embodiment of the present invention described above, the following operational effects can be obtained.

  A camera 101 (imaging unit) that captures an image ahead of the host vehicle, a feature point extraction unit 103 that extracts a feature point from the image captured by the camera 101, and movement information that calculates velocity information of pixels representing the feature point Based on the calculation unit 105, the coordinate conversion unit 104 that converts the position coordinates of the pixel into a three-dimensional position coordinate in real space, the position coordinates of the pixel in the three dimensions, and the velocity information of the pixel An object attribute determination unit 107 that determines that a feature point is a moving object is provided.

  According to this configuration, it is possible to determine that a feature point is a moving body based on the moving speed and position coordinates of the feature point extracted from the image captured by the camera 101.

  Further, according to the object detection apparatus of the embodiment of the present invention, the object attribute determination unit 107 determines the speed of the pixel in the relationship between the three-dimensional horizontal position coordinate of the pixel and the speed information of the pixel. When there is a coordinate having velocity information different from the information, the feature point is determined to be a moving object.

  According to this configuration, the moving object is identified from the relationship between the position in the horizontal direction on the image of the detected three-dimensional object and the speed. Since the three-dimensional object as the background moves in the horizontal direction on the image as the distance from the host vehicle increases, and the speed on the image decreases, the horizontal position on the image and the speed have a proportional (straight) relationship. On the other hand, there is no proportional relationship between the horizontal position on the image and the speed of the object existing in front of the three-dimensional object as the background. Therefore, it is possible to estimate the movement of the background without estimating the movement of the host vehicle or calculating the distance to the object, and separate the moving object from the background as a moving object. Body detection can be performed at high speed.

  The object detection apparatus according to the embodiment of the present invention further includes the grouping unit 106 that groups the pixels that are adjacent in the vertical direction of the image and whose velocity information is in a predetermined range, and the object attribute determination The unit 107 performs the determination based on the three-dimensional position coordinates of the pixel located at the top and the pixel located at the bottom among the pixels grouped by the grouping unit 106.

  According to this configuration, the object detection area is set on the speed image calculated based on the image captured by the camera 101, and the points that are adjacent to each other in the vertical direction of the image and whose moving speed is in the predetermined range are grouped. And the uppermost part (upper end) and the lowermost part (lower end) are detected. This determination can be performed efficiently by converting the coordinates of the pixels located at the top and bottom to the ZX plane and determining the moving body based on the position coordinates.

  Further, according to the object detection apparatus of the embodiment of the present invention, the object attribute determination unit 107 determines that the pixel located at the top and the pixel located at the bottom fall within the three-dimensional predetermined range. If the movable body is located in the same area among a plurality of areas divided and set to a predetermined number, the moving body is determined to be a plane object, and the pixel located at the top is the three-dimensional area. The moving body is determined to be a three-dimensional object when the pixel located outside and located at the bottom is located in any one of the plurality of three-dimensional areas.

  According to this configuration, the coordinates of the position of the uppermost pixel (upper end point) and the lowermost pixel (lower end point) are converted to the ZX plane using the camera parameters. Plane objects such as white lines and road surface markers that exist on the road surface exist on the road surface at both the upper and lower end points. Therefore, when coordinate conversion is performed on the ZX plane, the upper and lower end points are on the specified ZX plane. Projected. On the other hand, since the lower end point of the three-dimensional object exists on the road surface and the upper end point does not exist on the road surface, only the lower end point is projected on the prescribed ZX plane, and the upper end point is greatly deviated from the prescribed ZX plane. Become. Therefore, even in the configuration of a monocular camera, it is possible to determine a planar object and a three-dimensional object from the coordinate conversion results of the upper end point and the lower end point of the grouped area.

  Further, according to the object detection device of the embodiment of the present invention, the object attribute determination unit 107 sets the position coordinate in the three-dimensional position of the pixel located at the bottom of the pixel in the vertical direction of the image. Based on this, a road boundary representing the boundary of the road of the host vehicle is detected, and the moving body determination is performed for each section divided by the road boundary.

  According to this configuration, it is possible to detect the road boundary from the position of the coordinate conversion point of the lower end point, and to detect the moving body for each section divided by the detected road boundary, even under a complicated background, A moving body can be detected efficiently.

  Furthermore, according to the object detection apparatus of the embodiment of the present invention, the feature point is the edge of the object.

  According to this configuration, the edge of the object may be detected by the feature point extraction unit 103 from the image captured by the camera 101, and the above determination can be performed with simple image processing.

  Further, according to the object detection method of the embodiment of the present invention described above, the following operational effects can be obtained.

  Captures an image ahead of the host vehicle, extracts feature points from the captured image, calculates pixel velocity information representing the extracted feature points, and converts the pixel position coordinates into three-dimensional position coordinates in real space Then, the feature point is determined to be a moving object based on the position coordinates of the pixel in the three dimensions and the speed information of the pixel.

  According to this method, it is possible to determine that a feature point is a moving object based on speed information and position coordinates of the feature point extracted from the captured image.

  Further, according to the object detection method of the embodiment of the present invention, the determination of the moving body is performed by determining whether the pixel is based on the relationship between the three-dimensional horizontal position coordinate of the pixel and the velocity information of the pixel. When there is a coordinate having speed information different from the speed information, the feature point is determined to be a moving object.

  According to this method, the moving object is identified from the relationship between the position in the horizontal direction on the image of the detected three-dimensional object and the speed. Since the three-dimensional object as the background moves in the horizontal direction on the image as the distance from the host vehicle increases, and the speed on the image decreases, the horizontal position on the image and the speed have a proportional (straight) relationship. On the other hand, there is no proportional relationship between the horizontal position on the image and the speed of the object existing in front of the three-dimensional object as the background. Therefore, it is possible to estimate the movement of the background without estimating the movement of the host vehicle or calculating the distance to the object, and separate the moving object from the background as a moving object. Body detection can be performed at high speed.

  According to the object detection method of the embodiment of the present invention, the pixels that are adjacent to each other in the vertical direction of the image and whose velocity information is in a predetermined range are grouped, and the determination of the moving body is performed as a group. The determination is performed based on the three-dimensional position coordinates of the pixel located at the top and the pixel located at the bottom of the converted pixels.

  According to this method, an object detection area is set on a speed image calculated based on a captured image, and in each area, points that are adjacent in the vertical direction of the image and whose moving speed is within a predetermined range are grouped. The uppermost part (upper end) and the lowermost part (lower end) are detected. This determination can be performed efficiently by converting the coordinates of the pixels located at the top and bottom to the ZX plane and determining the moving body based on the position coordinates.

  Further, according to the object detection method of the embodiment of the present invention, the determination that the object is the moving body is performed by determining whether the pixel located at the top and the pixel located at the bottom are the three-dimensional predetermined When a plurality of areas set by dividing a range into a predetermined number are located in the same area, the moving body is determined to be a plane object, and the pixel located at the top is the three-dimensional And determining that the moving object is a three-dimensional object when the pixel located outside the region and located at the bottom is located in any one of the plurality of three-dimensional regions. .

  According to this method, the coordinates of the pixel located at the uppermost position (upper end point) and the pixel located at the lowermost position (lower end point) are converted to the ZX plane using camera parameters. Plane objects such as white lines and road surface markers that exist on the road surface exist on the road surface at both the upper and lower end points. Therefore, when coordinate conversion is performed on the ZX plane, the upper and lower end points are on the specified ZX plane. Projected. On the other hand, since the lower end point of the three-dimensional object exists on the road surface and the upper end point does not exist on the road surface, only the lower end point is projected on the prescribed ZX plane, and the upper end point is greatly deviated from the prescribed ZX plane. Become. Therefore, even in the configuration of a monocular camera, it is possible to determine a planar object and a three-dimensional object from the coordinate conversion results of the upper end point and the lower end point of the grouped area.

  Further, according to the object detection method of the embodiment of the present invention, the determination that the object is the moving body is performed by using the three-dimensional position coordinates of the pixel located at the lowest part in the vertical direction of the image among the pixels. Based on this, a road boundary representing the road boundary of the vehicle is detected, and the determination is performed for each section divided by the road boundary.

  According to this method, it is possible to detect the road boundary from the position of the coordinate conversion point of the lower end point, and to detect the moving body for each section divided by the detected road boundary, even under a complicated background, A moving body can be detected efficiently.

  Furthermore, according to the object detection method of the embodiment of the present invention, the feature point is the edge of the object.

  According to this method, it is only necessary to detect the edge of the object from the captured image, and the above determination can be made by simple image processing.

  In the present embodiment, for example, even when the same three-dimensional object or moving object cannot be grouped at the same speed based on the speed information due to the edge detection result, the planar object and the three-dimensional object are detected.

Here, as shown in FIG. 11, of the _P 1 _{1 to P} 1 13 detected on the outer _wall, P 1 1 to P ₁ 3 and _P 1 _4~P 1 13 are grouped separately, _{P 2} 1 to P ₂ 13 _of, P 2 1 to P ₂ 3 and _P 2 _4~P 2 13 are grouped _separately, of the _{P 3 1~P 3 13, P 3} 1~P 3 3 and _{P 3} 4~P ₃ 13 are grouped _separately, of the _{P 4 1~P 4 12, P 4} 1, P 4 2 and _P 4 _{3 to P} 4 12 are grouped _separately, P 5 1 to P ₅ A case where P ₅ 1, P ₅ 2 and P _{5 3} to P ₅ 10 among 10 are grouped separately will be described.

  1, FIG. 2 is a diagram illustrating an example of installation of the camera 101 on the vehicle, FIG. 3 is a diagram illustrating an image captured in front of the camera 101, and the edge illustrated in FIG. 4. Since a diagram illustrating an example of normalization and a diagram illustrating the velocity image illustrated in FIG. 5 are the same as those in the first embodiment, description thereof is omitted. In addition, the road boundary is detected based on the lower end point described in FIGS. 8 to 9, and the moving body is detected from the relationship between the horizontal position and the speed of the three-dimensional object for each area divided by the detected road boundary. The method to do is the same as the processing in the first embodiment, and therefore the description is omitted.

First, an area for detecting an object is set in the calculated speed image. That is, as shown in FIG. 11, a plurality of strip-shaped areas are set on the speed image, and the speed image is divided. In this embodiment, pixels having a speed from the lower end to the upper end of the image are sequentially scanned and detected for each region, and the detected pixels are coordinate-converted to the ZX plane based on the above-described (Expression 1) and (Expression 2). Then, it is determined whether or not the result of the coordinate conversion is located within the prescribed ZX plane shown in FIG. If the result of the coordinate transformation is located within the prescribed ZX plane, it is determined as a candidate point for a plane object. On the other hand, if it is not located within the prescribed ZX plane, it is determined as a candidate point for a three-dimensional object. The prescribed ZX plane is the same region as that described in the first embodiment, and the description of the region setting method is omitted. For example in the present _embodiment, P 1 1 to P ₁ 3 is determined as a candidate point of the planar articles located in the same area in the prescribed ZX _plane, the _P 1 4~P 1 13 is within the specified ZX plane Since it is not located, it is determined as a three-dimensional object candidate point. _{Similarly, P 2 1~P 2 3, P} 3 1~P 3 3, P 4 1, P 4 2, P 5 1, P 5 2, P 6 1~P 6 4, P 7 1~P 7 4 , P ₈ 1, P ₈ 2, P ₉ 1 to P ₉ 3, P ₁₀ 1, P ₁₀ 2, P ₁₁ 1 to P ₁₁ 3 are determined as plane object candidate points, and P ₂ 4 to P ₂ 13, P _{3 4~P 3 13, P 4 3~P} 4 12, P 5 3~P 5 10 is determined to the three-dimensional object candidate points.

Next, the solid object is determined based on the determination result of the planar object candidate point and the solid object candidate point. That is, when the point determined to be a planar object candidate point and the point determined to be a three-dimensional object candidate point exist continuously in the vertical direction, a portion of the three-dimensional object that exists near the road surface is present. Since it can be presumed that the object has been erroneously determined to be a planar object candidate point, as shown in FIG. 12, an object in which a planar object candidate point and a three-dimensional object candidate point continuously exist in the vertical direction is determined as a three-dimensional object. . For example, in this embodiment, since the planar object candidate points _P 1 1 to P ₁ 3 and the three-dimensional object candidate points _P 1 _4~P 1 13 is continuous in the longitudinal _direction, three-dimensional object the P 1 _{1 to P} 1 13 It is determined as OB1. _Similarly, three-dimensional object the _{P 2 1~P 2 13 OB2, P} 3 1~P 3 13 a solid object _{OB3, P} 4 _1~P 4 12 a three-dimensional object _{OB4, P} 5 _1~P 5 10 solid object OB5 Judge as.

Furthermore, as shown in FIG. 12, the lower end point is extracted from the points determined to be a plane object and a three-dimensional object, and the ZX where the coordinate conversion point to the ZX plane of the lower end point is located as in the first embodiment. The counter value of the counter in the plane area is incremented by 1 to calculate the position distribution of the lower end point. For example, in this embodiment, P ₁ 1, P ₂ 1, P ₃ 1, P ₄ 1, P ₅ 1, P ₆ 1, P ₇ 1, P ₈ 1, P ₉ 1, P ₁₀ 1, P ₁₁ 1 Is detected as the lower end point, and the counter value of the counter in the area where the coordinate conversion points RB1 to RB11 of the lower end points shown in FIG. Hereinafter, by performing the same operation as in the first embodiment, it is possible to detect the road boundary, the three-dimensional object, and the moving object in the three-dimensional object.

  FIG. 13 is a flowchart showing the processing of the object detection apparatus 10 in the present embodiment. The process shown in FIG. 13 is executed as a program to be started when the ignition switch is turned on. In FIG. 13, the same processing numbers as those in the first embodiment shown in FIG. 10 are assigned the same step numbers, and the differences will be mainly described below.

  In step S118, scanning is performed from the lower end to the upper end of the image within the strip area for object detection set in step S104, and the coordinates of pixels having speed are expressed in the ZX plane using (Expression 1) and (Expression 2). Convert coordinates up. After this, the flow moves to step S119.

  In step S119, if the coordinate conversion point of the pixel is located in the prescribed ZX plane, it is determined as a plane object candidate point. If the coordinate conversion point of the pixel is not located in the prescribed ZX plane, the three-dimensional object candidate point is determined. judge. After this, the flow proceeds to step S120.

  In step S120, all of the pixels for which the velocity has been calculated are coordinate-transformed on the ZX plane, and it is determined whether or not the detection of the planar object candidate point and the three-dimensional object candidate point has been completed. When the detection of the planar object candidate point and the three-dimensional object candidate point is completed, the flow proceeds to step S121. On the other hand, if the detection of the planar object candidate point and the three-dimensional object candidate point is not completed, the flow returns to step S118.

  In step S121, when a planar object candidate point and a three-dimensional object candidate point exist continuously in the vertical direction in each region, an object including the planar object candidate point and the three-dimensional object candidate point is determined as one solid object. To do. After this, the flow proceeds to step S122.

  In step S122, in each region, a point at the bottom of the points determined to be a planar object and a three-dimensional object is detected as a lower end point, and a region where a coordinate conversion point on the ZX plane of the detected lower end point is located. +1 is added to the counter value of the counter. After this, the flow proceeds to step S123.

  In step S123, it is determined whether or not the calculation of the position distribution information of the lower end point of each area is completed in each area based on the counter value. When the calculation of the position distribution information of the lower end point of each region is completed, the flow proceeds to step S110. On the other hand, if the calculation of the position distribution information of the lower end point of each region has not been completed, the flow returns to step S121.

  Thereafter, the road boundary, the three-dimensional object, and the moving object are detected by the same processing as in the first embodiment.

  According to the object detection apparatus of the embodiment of the present invention described above, the following functions and effects can be obtained in addition to the functions and effects of the first example.

  The object attribute determination unit 107 determines the pixel as a planar object candidate point when the pixel is located in any one of the three-dimensional regions, and the pixel is the three-dimensional object. The pixel is determined as a solid object candidate point when positioned outside a plurality of regions, and the planar object candidate point is determined when the planar object candidate point and the solid object candidate point are consecutively positioned in the image in the vertical direction. The three-dimensional object candidate point is determined as one solid object.

  According to this configuration, an object detection area is set on a speed image calculated based on an image captured by the camera 101, and each pixel having speed is coordinate-converted to the ZX plane, and then the position of the coordinate conversion point on the ZX plane is determined. Based on the coordinates, it is determined whether each pixel is a planar object candidate point or a three-dimensional object candidate point. Here, when the planar object candidate point and the three-dimensional object candidate point exist continuously in the vertical direction on the image, the object having the planar object candidate point and the three-dimensional object candidate point is determined as one solid object. This operation makes it possible to accurately determine a three-dimensional object even when pixels that make up the same object cannot calculate the same speed due to some influence.

  Further, according to the object detection method of the embodiment of the present invention described above, the following functions and effects can be obtained in addition to the functions and effects of the first example.

  The determination that the feature point is a moving object is that when the pixel is located in any one of the three-dimensional regions, the pixel is determined as a plane object candidate point, When the pixel is determined as a three-dimensional object candidate point when positioned outside the plurality of three-dimensional regions, and the planar object candidate point and the three-dimensional object candidate point are consecutively positioned in the vertical direction in the image The method includes determining the plane object candidate point and the solid object candidate point as one solid object.

  According to this method, an object detection area is set on a speed image calculated based on a captured image, and each pixel having speed is coordinate-converted to the ZX plane, and then based on the position coordinates of the coordinate conversion point on the ZX plane. Thus, it is determined whether each pixel is a planar object candidate point or a three-dimensional object candidate point. Here, when the planar object candidate point and the three-dimensional object candidate point exist continuously in the vertical direction on the image, the object having the planar object candidate point and the three-dimensional object candidate point is determined as one solid object. This operation makes it possible to accurately determine a three-dimensional object even when the same speed is not calculated due to some influence even though the pixels constitute the same object.

    In the present embodiment, the moving object is detected even when the relationship between the horizontal position and the speed of the three-dimensional object is not calculated in the section divided by the road boundary.

    Here, as shown in FIG. 14, the case where a white line and a parked vehicle exist between an outer wall and a pedestrian is demonstrated.

  The block diagram shown in FIG. 1, the diagram showing an example of installation of the camera 101 in the vehicle shown in FIG. 2, and the diagram showing an example of edge normalization shown in FIG. Explanation is omitted for the same reason.

  First, a velocity image is generated by classifying the velocity components of the edges existing on the captured image into predetermined class values. As shown in FIG. 15, the velocity image in the present embodiment represents the edge pixels where the velocity is detected as a round dot, and the pixel having a higher moving velocity indicates a larger dot. Further, the moving direction is represented by expressing the speed toward the right as a black point and the speed toward the left as a white point.

  In FIG. 15, the speed toward the right side of the image is detected from the outer wall on the right side of the traveling path of the host vehicle, and the speed toward the left side of the image is detected from the outer wall on the left side of the traveling path. Further, for a pedestrian moving from the left side of the traveling road to the right, the speed toward the right side of the image is detected, and from the vehicle parked on the left side of the traveling path, the speed toward the left side is detected. Furthermore, it is assumed that the speed toward the left side of the image is detected from the white line on the right side of the travel road and the speed toward the right side of the image is detected from the white line on the left side of the travel path due to the behavior of the host vehicle.

  Next, object detection is performed based on the velocity image. Since the region setting method for detecting an object and the pixel grouping method for determining the same object are the same as those in the first embodiment, the description thereof is omitted. In the present embodiment, as shown in FIG. 16, upper ends TL1 to TL15 and lower ends BL1 to BL15 are detected in the captured image shown in FIG.

  Next, the position coordinates of the lower end points BP1 to BP15 (not shown) and the upper end points TP1 to TP15 (not shown) having the coordinates of the center positions of the lower ends BL1 to BL15 and the upper ends TL1 to TL15 extracted on the xy plane are defined. Are converted into three-dimensional position coordinates (three-dimensional position coordinates in the real space) as points on a ZX plane (hereinafter referred to as a prescribed ZX plane) having the following area. Since this conversion method is the same as that of the first embodiment, description thereof is omitted. In the present embodiment, the converted points of the upper end points TP1 to TP15 and the lower end points BP1 to BP15 are the upper end point coordinate conversion points RT1 to RT15 and the lower end point coordinate conversion points RB1 to RB15.

  Next, a plane object and a three-dimensional object are determined on the basis of which of the areas where the coordinate conversion points of the upper end point and the lower end point are located by dividing the prescribed ZX plane. In order to detect the road boundary, among the counters set in each area, the counter value of the area where the coordinate conversion point of the lower end point is added is added to calculate the position information distribution of the lower end point. Since the ZX plane dividing method, the plane object and solid part determining method, and the method of calculating the position information distribution of the lower end point are the same as those in the first embodiment, description thereof will be omitted.

  In the present embodiment, as shown in FIG. 17, the coordinate transformation points RT1, RT4, RT6 to RT9, RT13 to RT15 of the upper end points TP1, TP4, TP6 to TP9, T13 to TP15 are defined in the prescribed ZX plane shown in FIG. Since the projection is performed outside, the group including the upper end points TP1, TP4, TP6 to TP9, and T13 to TP15 is determined to be a three-dimensional object, and the edge including the upper end points TP1, TP4, TP6 to TP9, and T13 to TP15 is determined to be xy. It is determined as a three-dimensional object OB1 to OB9 on the plane. The coordinate transformation points RT2, RT3, RT5, RT10 to RT12 of the upper end points TP2, TP3, TP5, TP10 to TP12 are coordinate transformation points RB2, RB3, RB5, RB10 of the lower end points BP2, BP3, BP5, BP10 to BP12. Since it is located in the same region as RB12, it is determined that the group including the upper end points TP2, TP3, TP5, TP10 to TP12 is an object located on the road surface. Here, in order to detect the road boundary, +1 is added to the counter value of the counter in the region where the coordinate conversion points RB1 to RB15 of the lower end point are located.

  Then, as shown in FIG. 18, from the obtained position distribution of the coordinate conversion point of the lower end point, an area where there is a high possibility that the road boundary line exists is extracted, and whether or not there is a straight line indicating the extracted road boundary Judgment is made. Since the extraction method and the straight line existence determination method are the same as those in the first embodiment, description thereof is omitted. In the present embodiment, in the area 10, it is determined that the straight line connecting the lower end points BP1, BP4, BP8 on the xy plane corresponding to the coordinate conversion points RB1, RB4, RB8 of the lower end point is the straight line L1 indicating the road boundary. In the region 30, it is determined that the straight line connecting the lower end points BP2, BP3, BP5 on the xy plane corresponding to the coordinate transformation points RB2, RB3, RB5 of the lower end point is the straight line L3 indicating the road boundary. The straight line connecting the lower end points BP10, BP11, and BP12 on the xy plane corresponding to the point coordinate conversion points RB10, RB11, and RB12 is determined to be the straight line L6 indicating the road boundary, and in the region 80, the lower end point n coordinate conversion point A straight line connecting the lower end points BP13, BP14, and BP15 on the xy plane corresponding to RB13, RB14, and RB15 is determined to be a straight line L8 that indicates a road boundary.

  Next, as shown in FIG. 19, a straight line constituted by a three-dimensional object continuously present in the distance from the vicinity of the host vehicle is detected as a background among straight lines indicating the road boundary. In general, the background is characterized in that stationary three-dimensional objects are linearly arranged in the distance from the vicinity of the host vehicle, the speed in the vicinity is high, and the speed decreases as the distance increases. Therefore, the road boundary showing such characteristics is extracted as the background. Here, when there are a plurality of road boundaries composed of three-dimensional objects, a road boundary located farther from the host vehicle is selected. In this embodiment, the road boundary lines L1 and L8, which are the intersections of the outer wall and the road surface, are detected as the background.

  Next, as shown in FIG. 19, a prediction formula for calculating the speed of the stationary object at an arbitrary position on the image is calculated from the relationship between the detected background image position and speed. Since the solid object serving as the background is a stationary object, the speed of the solid object serving as the background represents the speed of the stationary object at each position of the image. For example, the speed of the three-dimensional objects OB1, OB2, and OB5 represents the speed of the stationary object at each image position. Therefore, by calculating a prediction formula indicating the relationship between the image position and the speed between a plurality of three-dimensional objects, the speed of a stationary object at an arbitrary position on the image can be obtained. In this embodiment, for the object on the left side of the image, regression analysis is performed using the xy coordinates on the images of the three-dimensional objects OB1, OB2, and OB5 as explanatory variables and the speed as an objective variable. calculate. Similarly, with respect to the object on the right side of the image, regression analysis is performed using the xy coordinates on the images of the three-dimensional objects OB7, OB8, and OB9 as explanatory variables and the speed as an objective variable, and a prediction formula for the speed of a stationary object is calculated. .

  Next, as shown in FIG. 19, the velocity information (speed value and direction) at the lower end position of the three-dimensional object detected by the image processing and the stationary object at the lower end position of the three-dimensional object obtained from the calculated prediction formula. When the speed information is different from the speed information, the three-dimensional object is detected as a moving object.

  In the case of the present invention, since the moving body that faces the host vehicle is a detection target as the moving body, the direction of the speed of the three-dimensional object is the same direction as the speed of the stationary object calculated from the prediction formula, and the speed of the three-dimensional object If the difference between this value and the value of the speed of the stationary object calculated from the prediction formula is equal to or less than a predetermined value, it is determined that the three-dimensional object is a stationary object that is not a moving object that comes toward the host vehicle.

  Also, if it is a moving object, it is likely to exist on the road surface and is present in front of the background, so on the image, the lower end of the three-dimensional object exists below the lower end of the background, and the upper end of the three-dimensional object is Generally, it exists at a position higher than the lower end of the background. Furthermore, if the speed component at the lower end position of the three-dimensional object has a speed component in the opposite direction to the stationary object calculated from the prediction formula among the moving objects that are moving in the direction of the own vehicle, the path of the own vehicle is surely It can be said that it is a moving body that comes in the direction.

  Therefore, by adding the determination condition of a three-dimensional object having a speed opposite to the speed calculated from the prediction formula and having an upper end position higher than the lower end position of the background, not only whether it is a moving object but also the own vehicle It is possible to easily determine that the mobile body is traveling in the direction of the path.

  In the present embodiment, the three-dimensional objects OB3 and OB4 have speed information equivalent to the speed information of the stationary object calculated from the prediction formula, and thus are determined to be stationary objects. The three-dimensional object OB6 has a speed direction opposite to that of the stationary object calculated from the prediction formula, and its upper end position is higher than the lower end position of the background (running road boundary L1). It is determined that the moving body is moving in the direction.

    FIG. 20 is a flowchart showing the processing of the object detection apparatus 10 in the present embodiment, and for easily determining a moving body that surely faces the direction of the host vehicle in the above-described moving body determination method. It is. The process shown in FIG. 20 is executed as a program to be started when the ignition switch is turned on. In FIG. 20, the same processing numbers as those in the first embodiment shown in FIG. 10 are assigned the same step numbers, and the differences will be mainly described below.

  In step S131, among the road boundary straight lines detected in step S112, a straight line composed of a three-dimensional object that continuously exists far from the vicinity of the vehicle is detected as a road boundary serving as a background. After this, the flow moves to step S132.

  In step S132, multiple regression analysis is performed using the velocity information of the three-dimensional object constituting the background road boundary detected in step S131 as an objective variable and the image xy coordinates at the lower end point as explanatory variables. The prediction formula of the speed of the stationary object at the position of is calculated. After this, the flow moves to step 133.

  In step S133, the position of the road boundary that is the background detected in step S131 is compared with the upper edge position of the solid object, and when the upper edge position of the solid object exists above the position of the road boundary on the image. The three-dimensional object is detected as a moving object candidate. After this, the flow moves to step S134.

  In step S134, the speed information of the moving object candidate detected in step S133 is compared with the speed information of the stationary object obtained from the prediction formula at the moving object candidate position, and when the absolute value of the speed difference is within a predetermined range. The moving object candidate is determined as a stationary object, and the flow proceeds to step S116. On the other hand, if the absolute value of the speed difference is outside the predetermined range, the mobile object candidate is determined as a mobile object, and the flow proceeds to step S135.

  In step S135, if the direction of the speed of the three-dimensional object determined as the moving object is opposite to the direction of the speed of the stationary object obtained from the prediction formula, the moving object is determined as the moving object toward the center of the course of the own vehicle. To do. After this, the flow moves to step S116.

  Thereafter, all mobile objects are detected by the same process as in the first embodiment, and the process ends.

  According to the object detection device of the present embodiment described above, the following operational effects can be obtained.

  The object attribute determination unit 107 detects a reference stationary object and a three-dimensional object to be determined as a moving object based on the three-dimensional position coordinates and the pixel velocity information, and serves as the reference. Based on the speed information of the pixels constituting the stationary object and the speed information constituting the three-dimensional object to be determined as the moving object, the three-dimensional object to be determined as the moving object is a moving object. It was decided to determine that there was.

  Further, according to the object detection apparatus of the present embodiment, the reference stationary object is a solid object as a background.

  In addition, according to the object detection apparatus of the present embodiment, the object attribute determination unit 107 includes the grouping unit 106 that groups the pixels that are adjacent in the vertical direction of the image and whose velocity information is in a predetermined range. Detects the three-dimensional position coordinates of the pixel located at the top and the pixel located at the bottom among the pixels grouped by the grouping unit 106, and the pixel located at the top is detected. The pixel is located outside the plurality of regions set by dividing the three-dimensional predetermined range into a predetermined number, and the detected pixel located at the bottom is one of the three-dimensional regions The grouped pixels are determined as solid objects, and the pixels located at the bottom of the detected plurality of solid objects are in three dimensions in the real space. When the velocity information of the pixels located on the straight line in the set coordinates and located at the lowermost part of the plurality of solid objects changes at a constant rate in the straight line direction, the solid object is a background stationary object. I decided to judge.

  Further, according to the object detection apparatus of the present embodiment, the object attribute determination unit 107 includes the position coordinates and speed of the pixels located at the lowermost part of the plurality of three-dimensional objects that are the background stationary objects. Based on the above, the speed of the stationary object at the position coordinates of the three-dimensional object to be determined as the moving object is calculated, and the calculated velocity information of the stationary object and the three-dimensional object to be determined as the moving object are different. In this case, the three-dimensional object to be determined as the moving body is determined to be a moving body.

  According to this configuration, since the stationary solid object continuously present in the distance from the vicinity of the own vehicle is the background, the background can be easily detected from the detected three-dimensional object. In addition, the background, which is a stationary object, has a feature that the speed in the vicinity of the host vehicle is large and the speed decreases as the vehicle is farther away, so that a prediction formula that expresses the relationship between the detected image position of the background and the speed must be calculated. Thus, it is possible to predict the velocity information of the stationary object at an arbitrary position on the image. Therefore, by comparing the speed information of the detected three-dimensional object with the speed information of the background (stationary object) calculated by the prediction formula at the position of the three-dimensional object, it is easy to determine whether the three-dimensional object is a moving object. Can be determined.

  In addition, according to this configuration, since the moving object is detected based on the comparison of the speed information between the background and the three-dimensional object, it is possible to detect the moving object even when there are not a plurality of three-dimensional objects. It is possible to detect a moving body all at once without dividing an image even under a complicated background.

  Further, according to the object detection apparatus of the present embodiment, the object attribute determination unit 107 has a speed in a direction opposite to that of the three-dimensional object that is the background stationary object and is the background stationary object. The three-dimensional object having the upper end position at a position higher than the lower end position of the certain three-dimensional object is determined as the moving object.

  According to this configuration, the moving body can be easily detected based on the speed direction of the three-dimensional object and the position of the upper end.

  Moreover, according to the object detection method of this Embodiment demonstrated above, the following effects can be acquired.

  Based on the position coordinates in the three dimensions and the velocity information of the pixels, a reference stationary object and a three-dimensional object to be determined as a moving object are detected, and the pixels constituting the reference stationary object are detected. Based on the speed information and the speed information constituting the three-dimensional object to be determined as the moving object, it is determined that the three-dimensional object to be determined as the moving object is a moving object. .

  Moreover, according to the object detection method of the present embodiment, the reference stationary object is a three-dimensional object as a background.

  Further, according to the object detection method of the present embodiment, the pixels that are adjacent in the vertical direction of the image and whose velocity information is in a predetermined range are grouped, and the pixel located at the top of the grouped pixels is positioned. A plurality of pixels in which the three-dimensional position coordinates of the pixel and the pixel located at the lowermost part are detected and the detected pixel located at the uppermost part is set by dividing the three-dimensional predetermined range into a predetermined number When the pixel located outside the region and located at the bottom is located in any one of the three-dimensional regions, the grouped pixels are determined as a three-dimensional object and detected. The pixels positioned at the bottom of the plurality of three-dimensional objects are arranged on a straight line in three-dimensional position coordinates in the real space, and the image positioned at the bottom of the plurality of three-dimensional objects. If the speed information changes at a constant rate in the line direction, was possible to determine that the stationary object as a background of the three-dimensional object.

  Further, according to the object detection method of the present embodiment, as the moving body, based on the position coordinates and the speed of the pixel located at the bottom of the plurality of three-dimensional objects that are the stationary objects as the background When the velocity information of the stationary object at the position coordinates of the three-dimensional object to be determined is different from the calculated stationary object and the three-dimensional object to be determined as the moving object, The solid object to be determined is determined to be a moving object.

  According to this method, since the stationary solid object continuously existing far from the vicinity of the own vehicle is the background, the background can be easily detected from the detected three-dimensional object. In addition, the background, which is a stationary object, has a feature that the speed in the vicinity of the host vehicle is large and the speed decreases as the vehicle is farther away, so that a prediction formula that expresses the relationship between the detected image position of the background and the speed must be calculated. Thus, it is possible to predict the velocity information of the stationary object at an arbitrary position on the image. Therefore, by comparing the speed information of the detected three-dimensional object with the speed information of the background (stationary object) calculated by the prediction formula at the position of the three-dimensional object, it is easy to determine whether the three-dimensional object is a moving object. Can be determined.

  In addition, according to this method, since the moving object is detected based on the comparison of the speed information between the background and the three-dimensional object, it is possible to detect the moving object even when there are not a plurality of three-dimensional objects. It is possible to detect a moving body all at once without dividing an image even under a complicated background.

  Further, according to the object detection method of the present embodiment, it has a speed in a direction opposite to that of the three-dimensional object that is the background stationary object, and from the lower end position of the three-dimensional object that is the background stationary object. A three-dimensional object having an upper end position at a higher position is determined as a moving object.

  According to this method, it is possible to easily detect the moving body based on the speed direction of the three-dimensional object and the position of the upper end.

    In the present embodiment, even when the relationship between the horizontal position and the speed of the three-dimensional object is not calculated in the section divided by the road boundary, the moving object is detected and the direction of the moving object is determined.

    Here, as shown in FIG. 21, the case where a white line and a parked vehicle exist between an outer wall and a pedestrian will be described. Here, the description will be made on the assumption that the moving speed of the pedestrian moving from the left side to the right side of the traveling path is slower than the moving speed of the screen as the host vehicle travels and the speed toward the left side is detected on the screen.

  The block diagram shown in FIG. 1, the diagram showing an example of installation of the camera 101 in the vehicle shown in FIG. 2, and the diagram showing an example of edge normalization shown in FIG. Explanation is omitted for the same reason. Moreover, about each explanatory drawing of FIGS. 16-19, since it is the same as that of Example 3, description is abbreviate | omitted.

  In this embodiment, the moving object is detected by comparing the speed information of the stationary object obtained from the prediction formula and the speed information of the detected three-dimensional object, and the moving direction of the moving object is determined. Here, the case where the speed information of the stationary object obtained from the prediction formula is different from the speed information of the detected three-dimensional object, and the three-dimensional object has speed information in the opposite direction has been described in the third embodiment, and the description thereof will be omitted. .

  FIG. 22 shows the relationship between the moving direction of the host vehicle and the moving body around the host vehicle, and FIG. 23 shows the relationship between the moving direction of the moving body in real space and the moving direction on the screen. Here, the speed at which the background on the screen moves to the left while the host vehicle travels from time t to time t + 1 is defined as Vb.

  Consider the movement on the screen of the moving body P1 approaching at a speed Vp perpendicular to the course of the host vehicle while the host vehicle is traveling. Assuming that the speed Vp on the screen is Vm, the moving body P1 appears to move rightward on the screen when Vm> Vb and approaches the path of the host vehicle when Vm = Vb. If it approaches, it will appear to be stationary on the screen, and if it approaches the path of the vehicle with Vm <Vb, it will move to the left on the screen even though it moves to the right in real space. (Section D1 in FIG. 23).

  Next, let us consider the movement of the moving body P21 on the screen while the host vehicle travels away at a speed Vp in the traveling direction of the host vehicle (referred to as back movement in FIGS. 23 and 24). Assuming that the horizontal position on the screen of the moving body P21 at time t is x (t) and the horizontal position on the screen of the moving body P21 at time t + 1 is x (t + 1), x (t + 1) −x (t)> When it is 0, it appears to move rightward on the screen, and when x (t + 1) −x (t) = 0, it appears to be stationary on the screen, and x (t + 1) −. When x (t) <0, it appears to move leftward on the screen. Here, since the moving body P21 is moving away from the traveling direction of the host vehicle, the position change on the screen is usually smaller than the position change of the background, and the moving body P21 moving away from the traveling direction of the host vehicle is It looks like it moves in the left direction above (section D21 in FIG. 23).

  Next, the movement on the screen of the three-dimensional object P2 that is stopped while the host vehicle is traveling will be considered. The three-dimensional object P2 has a speed equivalent to the background. Accordingly, the stopped three-dimensional object P2 appears to move leftward at the speed Vb on the screen (section D2 in FIG. 23). Here, it is assumed that the moving body whose speed is ± Vs or less in the real space is stopped.

  Next, consider the movement on the screen of the moving body P22 that approaches the traveling direction of the host vehicle at the speed Vp while the host vehicle is traveling. If the horizontal position on the screen of the moving body P22 at time t is x (t) and the horizontal position on the screen of the moving body P22 at time t + 1 is x (t + 1), then x (t + 1) −x (t)> When it is 0, it appears to move rightward on the screen, and when x (t + 1) −x (t) = 0, it appears to be stationary on the screen, and x (t + 1) −. When x (t) <0, it appears to move leftward on the screen. Here, since the moving body P11 is moving so as to approach the traveling direction of the own vehicle, the position change on the screen is usually larger than the position change of the background, and the moving body P22 moving away from the traveling direction of the own vehicle It looks like it moves in the left direction above (section D22 in FIG. 23).

  Next, consider the movement on the screen of the moving body P3 moving away at a speed Vp perpendicular to the course of the host vehicle while the host vehicle is traveling. If the speed Vp on the screen is Vm, the moving body P3 appears to move leftward at the speed Vm + Vb on the screen (section D3 in FIG. 23).

    FIG. 24 is a flowchart showing processing of the object detection apparatus 10 in the present embodiment. The process shown in FIG. 24 is executed as a program to be started when the ignition switch is turned on. In FIG. 24, the same process number as the process in Example 1 and Example 3 shown in FIG. 23 is assigned with the same step number, and the difference will be mainly described below.

  In step S141, the speed information of the three-dimensional object determined as the moving object is compared with the speed information of the stationary object obtained from the prediction formula for the moving object detected in step S134. The moving direction is determined.

  A flowchart showing details of step S141 is shown in FIG.

  In step S1411, the flow branches based on the difference dV = | Vm | − | Vb | between the absolute value of the detected velocity Vm of the moving object and the absolute value of the velocity Vb of the stationary object obtained from the prediction formula. Is called. If dV> 0, the flow moves to step S1412. On the other hand, if dV ≦ 0, the flow moves to step S1415.

  In step S1412, it is determined whether or not | dV | is equal to or greater than a predetermined value. If | dV | is equal to or greater than the predetermined value, the flow moves to step S1413. On the other hand, if | dV | is less than the predetermined value, the flow moves to step S1414.

  In step S1413, it is determined that the moving body meets the condition of the section D3 in FIG. 23, and the moving body moves away from the course center of the host vehicle.

  In step S1414, it is determined that the moving body meets the condition of section D22 in FIG. 23, and the moving body approaches the traveling direction of the host vehicle.

  In step S1415, it is determined whether or not | dV | is equal to or greater than a predetermined value. If | dV | is equal to or greater than the predetermined value, the flow moves to step S1416. On the other hand, if | dV | is less than the predetermined value, the flow moves to step S1416.

  In step S1416, it is determined that the moving body satisfies the condition of the section D1 in FIG. 23 and the moving body gradually approaches the center of the course of the host vehicle.

  In step S1417, it is determined that the moving body satisfies the condition of the section D21 in FIG. 23 and the moving body moves away in the traveling direction of the host vehicle.

  Thereafter, all moving object detection processing is performed by the same processing as in the first embodiment, and the processing is terminated.

  According to the object detection device of the present embodiment described above, the following operational effects can be obtained.

  The object attribute determination unit 107 determines a moving direction in real space of a three-dimensional object having the same speed as the three-dimensional object that is the stationary object serving as the background.

  According to this configuration, it is possible to determine the moving direction of the moving body other than the moving body that approaches the center of the course of the host vehicle based on the speed of the stationary object calculated by the prediction formula and the detected speed difference of the moving body. Become.

  Moreover, according to the object detection method of this Embodiment demonstrated above, the following effects can be acquired.

  The moving direction in the real space of the three-dimensional object having the same speed as that of the three-dimensional object as the background stationary object is determined.

  According to this method, it is possible to determine the moving direction of a moving body other than a moving body approaching the course center of the host vehicle based on the speed of the stationary object calculated by the prediction formula and the detected speed difference of the moving body. Become.

  As described above, according to the present invention, a planar object and a three-dimensional object are identified from an image captured by a camera in which the horizontal axis and the vertical axis of the imaging surface are set parallel and perpendicular to the ground surface, respectively, and the three-dimensional object is a moving object. It is also possible to identify whether or not the moving object can be detected with a simple sensor configuration.

  In addition, since a planar object and a three-dimensional object are identified based on the speed and position map of the image feature points (edges) on the image, a moving object can be detected at high speed with a simple algorithm.

  In addition, since a moving object is detected from the relationship between the detected horizontal position and speed of a plurality of three-dimensional objects, there is no need to estimate the movement of the vehicle or calculate the distance to the object. Can be detected.

  In addition, the road boundary is detected from the position coordinates of the coordinate conversion point, and the relationship between the horizontal position and speed of multiple solid objects is obtained for each section divided by the road boundary. Can detect moving objects.

  In addition, even when the speed detection accuracy is insufficient and the speed of pixels of feature points representing the same object varies, it is possible to accurately detect a planar object and a three-dimensional object.

  Also, by comparing the speed information of the detected three-dimensional object with the speed information of the background (stationary object) calculated by the prediction formula at the position of the three-dimensional object, it is easy to determine whether the three-dimensional object is a moving object. Can be determined.

  Further, the moving object can be easily detected based on the detected speed direction of the three-dimensional object and the position of the upper end.

  Further, based on the difference between the speed of the stationary object calculated by the prediction formula and the detected speed of the moving body, it is possible to determine the moving direction of the moving body other than the moving body approaching the course center of the host vehicle.

  As mentioned above, although the Example of this invention was explained in full detail with drawing, an Example is only an illustration of this invention and this invention is not limited only to the structure of an Example. Accordingly, it is a matter of course that the present invention includes any design change within a range not departing from the gist of the present invention.

  For example, the block diagram is not limited to that shown in the above embodiment, and any configuration having equivalent functions may be used.

  The camera mounting position is not limited to the position described in the embodiment. The optical axis of the camera faces the front front direction (Z direction) of the vehicle, and the horizontal axis and the vertical axis of the imaging surface are parallel to the road surface, respectively. It is only necessary to be set to be vertical.

  In addition, in normalizing the detected edge width, the edge width is not limited to three pixels, and an arbitrary number of pixels can be set. In this case, since the pixel at the center of the edge is used in the subsequent processing, the number of pixels with the edge width is desirably an odd number.

  Further, the number of areas set by dividing the ZX plane is not limited to that shown in the above embodiment, and can be set by dividing it into an arbitrary number.

  Further, the vertical and horizontal ranges of the ZX plane can be set to arbitrary values.

  Moreover, although the example which mounts the object detection apparatus 10 in the vehicle which drive | works a road was demonstrated in the said Example, you may mount in another mobile body.

  Furthermore, in the above embodiment, examples of curbstones, white lines, and contact points between the outer wall and the road surface have been described as the road boundary. However, the present invention is not limited to this example. Boundary with other areas (fields, fields, etc.) may be detected.

It is a block diagram which shows the structural example of one Embodiment of an object detection apparatus. It is a figure which shows the example of installation to the vehicle of the camera 101. FIG. 3 is a diagram illustrating an example of an image captured by a camera 101. FIG. It is a figure which shows the example of each process performed in order to normalize the extracted edge and to obtain an edge image. It is a figure which shows the example of a speed image. It is a figure which shows the example which set the object detection area | region on the speed image, grouped the pixel which has the same speed as a lower end point in each object detection area, and detected the position of the upper end point of the grouped area | region. It is a figure which shows the example which carries out coordinate conversion of the upper end point and lower end point detected on the speed image on a ZX plane, and determines whether an object is a planar object or a solid object. It is a figure which shows the example which detects a road boundary line from the position distribution on the ZX plane of the coordinate transformation point of a lower end point. It is a figure which shows the specific example in the case of detecting a moving body from the relationship of the horizontal direction position and speed of the image of the solid object detected within the area | region divided by the detected track boundary. 3 is a flowchart illustrating a process of the first embodiment of the object detection apparatus 10. An example is shown in which an object detection area is set on a speed image, and a pixel having a speed in each object detection area is coordinate-converted to a ZX plane to determine whether it is a planar object candidate point or a three-dimensional object candidate point. FIG. It is a figure which shows the example which determines a solid object from the alignment method of the detected planar object candidate point and a solid object candidate point. It is a flowchart which shows the process of the 2nd Example of the object detection apparatus. In Example 3, it is a figure which shows the example of the image imaged with the camera. It is a figure which shows the example of a speed image. It is a figure which shows the example which set the object detection area | region on the speed image, grouped the pixel which has the same speed as a lower end point in each object detection area, and detected the position of the upper end point of the grouped area | region. It is a figure which shows the example which carries out coordinate conversion of the upper end point and lower end point detected on the speed image on a ZX plane, and determines whether an object is a planar object or a solid object. It is a figure which shows the example which detects a road boundary line from the position distribution on the ZX plane of the coordinate transformation point of a lower end point. The background is detected from the detected road boundary, and a prediction formula for calculating the speed of the stationary object at an arbitrary position on the image is determined from the relationship between the position coordinates and the speed of the solid object serving as the background. It is explanatory drawing which detects a mobile body by comparing with the speed information of the detected solid object. 10 is a flowchart of processing according to the third embodiment. It is a figure which shows the example of a speed image. It is a figure which shows the relationship between the moving direction of the own vehicle and the mobile body around the own vehicle. It is a figure which shows the relation between the moving direction in the real space of the moving body and the moving direction on the screen. 10 is a flowchart of processing in Embodiment 4. 14 is a flowchart of processing for determining a moving direction of a moving object in the fourth embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Object detection apparatus 100 Control part 101 Camera (imaging part)
102 image temporary recording unit 103 feature point extraction unit 104 coordinate conversion unit 105 movement information calculation unit 106 grouping unit 107 object attribute determination unit

Claims (14)

An imaging unit that captures an image in front of the host vehicle;
A feature point extraction unit that extracts feature points from the image captured by the imaging unit;
A movement information calculation unit for calculating movement speed information of pixels representing the feature points on the image;
A grouping unit that groups the pixels adjacent in the vertical direction of the image and having the moving speed information in a predetermined range;
Among the pixels grouped by the grouping unit, the position coordinates of the pixel located at the top and the pixel located at the bottom in the vertical direction of the image are based on the height and depression angle from the road surface of the imaging unit. A coordinate conversion unit that converts position coordinates in an overhead view plane that is a plane in the horizontal direction and the vehicle longitudinal direction in real space;
The same region among the plurality of regions in which the pixel located at the top and the pixel located at the bottom are set by dividing a predetermined range of the overhead view plane into a predetermined number in the horizontal direction and the vehicle front-rear direction If the pixel is located in the group, the grouped pixels are determined to be a plane object,
The pixel located in the uppermost part is located outside the plurality of areas set in the overhead view plane, and the pixel located in the lowermost part is located in any one of the plurality of areas set in the overhead view plane When doing so, the grouped pixels are determined as solid objects,
The pixel located at the bottom of the grouped pixels determined to be the three-dimensional object is determined to be the three-dimensional object when aligned on a predetermined straight line in the relationship between the horizontal position coordinate and the moving speed information in the image. An object attribute determination unit that determines the grouped pixels as a stationary object and determines the grouped pixels determined as the three-dimensional object as a moving object when not aligned on the predetermined straight line;
An object detection apparatus comprising: The coordinate conversion unit is configured to display, from the pixels grouped by the grouping unit, the position coordinates from the pixel located at the bottom to the pixel located at the top in the vertical direction of the image, Converted to
The object attribute determination unit determines that the pixel is a plane object candidate point when the result of the coordinate conversion unit is located in any one of a plurality of regions set in the overhead view plane ,
If the result of the coordinate transformation is not located within a plurality of regions set in the overhead view plane, the pixel is determined as a three-dimensional object candidate point,
The planar object candidate point and the three-dimensional object candidate point are determined as one solid object when the planar object candidate point and the three-dimensional object candidate point are continuously located in the vertical direction in the image. The object detection apparatus according to claim 1. The object attribute determination unit
Of the grouped pixels, a road boundary representing the boundary of the road of the host vehicle is detected based on the position coordinates in the overhead view of the pixel located at the bottom in the vertical direction of the image, and is divided by the road boundary The object detection apparatus according to claim 1 , wherein the grouped pixel is determined to be a moving object for each divided section . Take an image of the front of your vehicle,
Extract feature points from the captured image,
Calculating moving speed information of a pixel representing the extracted feature point on the image;
Grouping the pixels adjacent in the vertical direction of the image and having the moving speed information in a predetermined range;
Among the grouped pixels, in the vertical direction of the image, the position coordinates of the pixel located at the top and the pixel located at the bottom are based on the height and depression angle from the road surface of the position where the image was captured, Convert to position coordinates in the overhead view plane that is the horizontal plane in the real space and the plane in the vehicle longitudinal direction ,
The same region among the plurality of regions in which the pixel located at the top and the pixel located at the bottom are set by dividing a predetermined range of the overhead view plane into a predetermined number in the horizontal direction and the vehicle front-rear direction If the pixel is located in the group, the grouped pixels are determined to be a plane object,
The pixel located in the uppermost part is located outside the plurality of areas set in the overhead view plane, and the pixel located in the lowermost part is located in any one of the plurality of areas set in the overhead view plane When doing so, the grouped pixels are determined as solid objects,
The pixel located at the bottom of the grouped pixels determined to be the three-dimensional object is determined to be the three-dimensional object when aligned on a predetermined straight line in the relationship between the horizontal position coordinate and the moving speed information in the image. An object detection method characterized in that a grouped pixel is determined as a stationary object, and a grouped pixel determined as a three-dimensional object is determined as a moving object when not aligned on the predetermined straight line . The determination of being a mobile object is
Among the pixels grouped by the grouping unit, in the vertical direction of the image, the position coordinates from the pixel located at the bottom to the pixel located at the top are converted into position coordinates on the overhead view plane,
When the result of the coordinate conversion unit is located in any one of a plurality of regions set in the overhead view plane, the pixel is determined as a plane object candidate point,
If the result of the coordinate transformation is not located within a plurality of regions set in the overhead view plane, the pixel is determined as a three-dimensional object candidate point,
Determining the planar object candidate point and the solid object candidate point as one solid object when the planar object candidate point and the solid object candidate point are continuously located in the vertical direction in the image. The object detection method according to claim 4 . The determination of being a mobile object is
Of the grouped pixels , a road boundary representing the boundary of the road of the host vehicle is detected based on the position coordinates in the overhead view of the pixel located at the bottom in the vertical direction of the image, and is divided by the road boundary The object detection method according to claim 4, wherein the object detection method is performed for each divided section . The object attribute determination unit
Among the grouped pixels, detect a road boundary that represents the boundary of the road of the host vehicle based on the position coordinates in the overhead view of the pixel located at the bottom,
Among the running road boundaries, those that are constituted by a three-dimensional object that continuously exists far away from the vicinity of the host vehicle are detected as a background,
From the relationship between the background and detected grouped pixels and the moving speed information, the moving speed information of a stationary object at an arbitrary position on the image is estimated,
Based on the moving speed information of the stationary object at the arbitrary position and the moving speed information of the grouped pixels determined as the three-dimensional object, it is determined that the grouped pixels determined as the three-dimensional object are moving objects. object detection apparatus according to claim 1, characterized in that. The arbitrary position is a lower end position of the grouped pixels determined to be the three-dimensional object ,
The object attribute determination unit
When the moving speed information of the stationary object at the lower end position of the pixels grouped it is determined that the three-dimensional object, moving speed information of the pixels grouped it is determined that the three-dimensional object are different, and wherein determining that the mobile The object detection apparatus according to claim 7 . The object attribute determination unit
With the stationary object having a velocity in the opposite direction in the lower end position of the pixels grouped it is determined that the three-dimensional object, in a position higher than the lower end position of the stationary object at the lower end position of the pixels grouped it is determined that the three-dimensional object The object detection apparatus according to claim 8 , wherein the grouped pixels determined to be a three-dimensional object having an upper end position are determined to be moving objects. The object attribute determination unit
To claim 8 or claim 9, characterized in that to determine the movement direction in the real space of the three-dimensional object with the moving speed information in the same direction as the stationary object at the lower end position of the pixels grouped it is determined that the three-dimensional object The object detection apparatus described. Among the grouped pixels, detect a road boundary that represents the boundary of the road of the host vehicle based on the position coordinates in the overhead view of the pixel located at the bottom,
Among the running road boundaries, those that are constituted by a three-dimensional object that continuously exists far away from the vicinity of the host vehicle are detected as a background,
From the relationship between the background and detected grouped pixels and the moving speed information, the moving speed information of a stationary object at an arbitrary position on the image is estimated,
Based on the moving speed information of the stationary object at the arbitrary position and the moving speed information of the grouped pixels determined as the three-dimensional object, it is determined that the grouped pixels determined as the three-dimensional object are moving objects. the object detection method as claimed in claim 4, characterized in that. The arbitrary position is set as the lower end position of the grouped pixels determined as the three-dimensional object ,
When the moving speed information of the stationary object at the lower end position of the pixels grouped it is determined that the three-dimensional object, moving speed information of the pixels grouped it is determined that the three-dimensional object are different, and wherein determining that the mobile The object detection method according to claim 11 . With the stationary object having a velocity in the opposite direction in the lower end position of the pixels grouped it is determined that the three-dimensional object, in a position higher than the lower end position of the stationary object at the lower end position of the pixels grouped it is determined that the three-dimensional object The object detection method according to claim 12 , wherein the grouped pixels determined to be a three-dimensional object having an upper end position are determined to be moving objects. To claim 12 or claim 13, characterized in that to determine the movement direction in the real space of the three-dimensional object with the moving speed information in the same direction as the stationary object at the lower end position of the pixels grouped it is determined that the three-dimensional object The object detection method described.

Images