JP5320828B2 - Pedestrian detection device and pedestrian detection method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a pedestrian detection apparatus and a pedestrian detection method for preventing background from being erroneously determined as pedestrians. <P>SOLUTION: The pedestrian detection apparatus has a camera 10, a feature point extraction part 21 for extracting feature points from a captured image, a movement information calculation part 22 for calculating movement information including moving speed and/or the moving direction about the extracted feature points, a determination target extraction part 31 for extracting a target area including a moving target solid from the captured image according to the calculated feature point movement information, and a pedestrian determination means 34 for comparing the movement information about the extracted target area with the movement information about a comparison area set around the target area, and determining whether or not the target solid included in the extracted target area is a pedestrian according to the result of comparing the movement information. <P>COPYRIGHT: (C)2010,JPO&amp;INPIT

Description

  The present invention relates to a pedestrian detection device and a pedestrian detection method for detecting a pedestrian based on a captured image.

A moving body detection device and a movement detection method for detecting a moving body such as a pedestrian or a vehicle using an optical flow (image speed information) from a captured image of an in-vehicle camera have been proposed. For example, a moving object detection method is proposed in which an optical flow of a captured image is obtained, and the moving object is detected by comparing the optical flow of the background of the captured image extracted based on the movement of the moving object and the optical flow of the entire captured image. (See Patent Document 1).

Japanese Patent Laid-Open No. 2004-56763

  In the conventional moving body detection apparatus or the moving body detection method, it is necessary to accurately calculate the captured image and the optical flow of the background in order to detect the moving body. It is difficult to calculate an accurate optical flow because the edges of the shape are combined in a complicated manner and the temporal association of these pixels is also complicated. For this reason, there has been a problem that erroneous detection of determining the background as a pedestrian may occur simply by comparing the optical flow of the captured image with the optical flow of the background.

  The problem to be solved by the present invention is to provide a pedestrian detection device and a pedestrian detection method that reduce erroneous detection of the background as a pedestrian.

  In the present invention, based on the movement information of the target area extracted from the captured image and the movement information of the comparison area set around the target area, the target three-dimensional object included in the extracted target area is a pedestrian. The above-mentioned problem is solved by determining whether or not.

  According to the present invention, by comparing the movement information of the target area including the target three-dimensional object and the movement information of the comparison area, it is possible to observe the relative movement of the target three-dimensional object with respect to the background existing in the vicinity. It is possible to reduce erroneous detection of the background as a pedestrian.

<First Embodiment>
FIG. 1 is a diagram illustrating an example of a block configuration of an in-vehicle device 1000 including a pedestrian detection device 100. The pedestrian detection device 100 according to the present embodiment is a device that is mounted on a vehicle and detects a pedestrian based on a captured image in front of the vehicle (front along the traveling direction).

  As shown in FIG. 1, the in-vehicle device 1000 of the present embodiment acquires pedestrian detection information from the pedestrian detection device 100 and the pedestrian detection device 100, and receives information based on the acquired pedestrian detection information as a user or And an external device such as a vehicle controller 200 that outputs to the vehicle side. As an external device, as shown in FIG. 1, in addition to a vehicle controller 200 that controls driving of the vehicle, an alarm device 300 that notifies the presence of a pedestrian, a travel support device 400 that provides travel support according to the presence of the pedestrian, One or more output devices 500 having a display and / or speakers and presenting information regarding the presence of a pedestrian are provided. The arithmetic processing of the pedestrian detection device 100, the vehicle controller 200, the alarm device 300, the travel support device 400, and the output device 500 of the present embodiment is a process configured by combining operation circuits such as a CPU, MPU, DSP, and FPGA. Executed by means.

  Note that the processing means of these devices are connected by an in-vehicle LAN such as a CAN (Controller Area Network).

  The pedestrian detection device 100 according to the present embodiment includes a camera 10, an image memory 11 that stores imaging data of the camera 10, an image processing unit 20 that processes a captured image, and a determination processing unit 30.

Hereinafter, each configuration provided in the image processing apparatus 100 will be described.

  The camera 10 is a camera having an image sensor such as a CCD (Charge-Coupled Devices) or a CMOS (Complementary Metal-Oxide Semiconductor), for example, continuously around the vehicle (vehicle front, vehicle rear, vehicle side, etc.) in a predetermined cycle. The image memory 11 that outputs the image captured for each frame to the image memory 11 stores the image data in an accessible state. As the image memory 11, an HDD, CD, MD, DVD, optical disk, or other recording medium is used.

  FIG. 2 shows an installation example of the camera 10. As shown in FIG. 2, the camera 10 is installed in front of the upper part of the vehicle interior. As shown in the figure, in the present embodiment, one camera 10 is installed in the vehicle. That is, in this embodiment, the surroundings of the vehicle are imaged by the monocular camera 10.

  Further, the camera 10 has its optical axis LS oriented in the front front direction (Z direction) of the vehicle, the horizontal axis X (not shown) of the imaging surface is parallel to the road surface, and the vertical axis Y (not shown) of the imaging surface. (Omitted) is set to be approximately perpendicular to the road surface.

  An example of an image captured by the camera 10 (an image ahead of the host vehicle) is shown in FIG. The captured image captured by the camera 10 is represented by an xy coordinate system defined by an x axis extending from left to right and an y axis extending from top to bottom with the upper left vertex of the image as the origin. In the image example shown in FIG. 3, the captured image includes left and right white lines, boundary lines of roads such as outer walls, and pedestrians moving from the left side to the right side of the road.

  Next, the image processing unit 20 will be described. The image processing unit 20 includes a feature point extraction unit 21, a movement information calculation unit 22, a grouping unit 23, and a coordinate conversion unit 24, and processes a captured image captured by the camera 10.

  First, the feature point extraction unit 21 will be described. The feature point extraction unit 21 extracts feature points from the captured image captured by the camera 10. The feature point extraction unit 21 acquires an image captured by the camera 10 from the image memory 11, and binarizes the acquired captured image using a predetermined threshold value, thereby extracting the edge of an object present in the image. Extract. FIG. 4A shows an example of the extracted vertical edge. Next, thinning processing is performed on each extracted edge to narrow the edge width, and the center of the edge is accurately set (see FIG. 4B). Further, the edge is expanded in the horizontal direction so that the edge width of the thinned edge becomes a constant width, for example, a width corresponding to three pixels (see FIG. 4C). By this operation, the extracted edges are normalized, and an edge image having a uniform width for each edge is obtained.

  The movement information calculation unit 22 calculates movement information including the movement direction and / or movement speed of the feature points extracted by the feature point extraction unit 21 and the pixels corresponding to the periphery of the feature points. The obtained moving speed and moving direction of the characteristic part are stored in association with the imaging timing identifier or the frame identifier. This pixel movement information includes “pixel movement speed” and “pixel movement direction” along with pixel identification information. When a plurality of feature parts exist in one image data, the movement information is calculated for all the feature parts.

  Hereinafter, the movement information calculation unit 22 of the present embodiment will be described.

  The movement information calculation unit 22 according to the present embodiment counts up the count value of the pixel at the position where the edge corresponding to the extension of the object is detected based on the information of the image of the object captured by the camera 10. Based on the slope of the value, the moving speed and moving direction of the edge are calculated.

  In addition, the movement information calculation unit 20 updates the counter value of the pixel counter of the pixel corresponding to the edge included in each image data with a predetermined method for image data with different imaging timings. Here, the pixel counter is a counter set for each pixel. When the pixel corresponds to the edge, the counter value of the pixel counter is incremented by +1. When the pixel does not correspond to the edge, the counter value of the pixel counter is increased. This is a counter that is set to 0 (initialized). This counter value update process is performed for each frame that is repeatedly imaged by the camera 10 in a predetermined cycle. When this operation is performed, the counter value of the corresponding pixel counter increases for a pixel having a long time corresponding to an edge, while the counter value of the corresponding pixel counter decreases for a pixel having a short time corresponding to an edge.

  This change in the counter value of the pixel counter represents the moving direction and moving amount of the edge. For this reason, the moving direction and moving speed of the edge on the captured image are calculated based on the counter value. Since the coordinate system of the image represents the azimuth, the moving direction and moving speed of the edge and the feature corresponding to the edge can be obtained.

  Furthermore, the movement information calculation method performed by the movement information calculation unit 22 will be described with reference to FIG. FIG. 4 is a diagram for explaining the movement information calculation process, that is, a process for obtaining an edge image in which the extracted edge is normalized and calculating the movement direction and the movement speed from the edge counter value (dwell time). It is a figure for demonstrating concretely.

  First, the feature point extraction unit 21 performs binarization processing on the edge image. The binarization process is a process in which a pixel at a position where an edge is detected is set to 1 and a pixel at a position where no edge is detected is set to 0. FIG. 4A shows an example of the binarized image of the extracted vertical edge.

  Next, as shown in FIG. 4B, thinning processing is performed on the generated binary image. The thinning process is a process of reducing the edge width of the detected edge until a predetermined pixel width is reached. That is, the edge width is narrowed by performing thinning processing on each extracted edge. In this example, as shown in FIG. 4B, the edge width of the edge is thinned until the predetermined pixel width becomes one pixel. In this way, the edge is thinned to a predetermined pixel width, thereby setting the center position as the center of the edge. Note that, in this example, an example in which one pixel is thinned is shown, but the number of pixels to be thinned is not particularly limited.

  Next, an expansion process is performed to expand the edge width of the thinned edge. The expansion process is performed so that the edge width is constant from the center position set by thinning toward the edge movement direction, and the edge width is also changed from the center position to the direction opposite to the edge movement direction. It is a process of expanding. In this example, the edge is expanded in the horizontal direction so that the edge width of the thinned edge becomes a width corresponding to three pixels. By this process, the extracted edges are normalized, and an edge image having a uniform width is obtained.

  Specifically, as shown in FIG. 4C, one pixel is expanded from the edge center position x0 in the edge movement direction (the positive direction of the x-axis) and opposite to the edge movement direction from the edge center position x0. The edge width is expanded to 3 pixels by expanding one pixel in the direction (negative direction of the x axis).

  By performing the thinning process and the expansion process in this way, the edge width of the extracted edge image is standardized by standardizing to a predetermined width in the edge moving direction.

  Next, the count-up process performed for the movement information calculation unit 22 to calculate movement information will be described. The count-up process mentioned here is a process for counting up the value of the memory address corresponding to the position of the pixel where the edge is detected and initializing the value of the memory address corresponding to the position of the pixel where the edge is not detected. It is.

  Hereinafter, the edge count-up process by the movement information calculation unit 22 will be described with reference to FIGS. For convenience of explanation, here, a case where the edge moves in the positive direction of the x-axis will be described as an example. Even when the edge moves in the negative x-axis direction, the y-axis direction, or two-dimensionally, the basic processing method is common.

  As shown in FIG. 4C, the edge has a center position of the edge at a position x0 in a certain frame. Then, the pixel is expanded from the center position to the position x0 + 1 of one pixel in the edge moving direction, and similarly expanded from the center position to the position x0-1 of one pixel in the direction opposite to the edge moving direction.

  The count value of the memory address corresponding to the position where such an edge is detected, “x0-1”, “x0”, “x0 + 1” is incremented by +1. On the other hand, the count value of the memory address corresponding to the position where the edge is not detected is reset.

  For example, in FIG. 4D, edges are detected at positions “x0-1”, “x0”, and “x0 + 1” at time t. Therefore, the count value of the memory address corresponding to each position is incremented by one. As a result, the count value at the position “x0 + 1” is “1”, the count value at the position “x0” is “3”, and the count value at the position “x0-1” is “5”.

  Next, as shown in FIG. 4E, since the edge does not move even at time t + 1, the edge is detected at each of the positions “x0-1”, “x0”, and “x0 + 1”. . Therefore, the count values at the positions “x0-1”, “x0”, and “x0 + 1” are further incremented by one. As a result, the count value at position “x0 + 1” is 2, the count value at position “x0” is 4, and the count value at position “x0-1” is 6.

  Further, as shown in FIG. 4F, at time t + 2, the edge is shifted by one pixel in the positive direction of the x-axis, and the edge is detected at positions “x0”, “x0 + 1”, and “x0 + 2”. Therefore, the count value of the memory address corresponding to the positions “x0”, “x0 + 1”, and “x0 + 2” where the edge is detected is counted up. On the other hand, the count value at the position “x0-1” where no edge is detected is reset to “zero”. As a result, the count value at position “x0 + 2” is 1, the count value at position “x0 + 1” is 3, and the count value at position “x0” is 5, as shown in FIG. Further, the count value at the position “x0-1” where no edge is detected is reset to “0”.

  As described above, the movement information calculation unit 22 counts up the count value of the memory address corresponding to the position where the edge is detected, and resets the count value of the memory address corresponding to the position where the edge is not detected.

  In the description based on FIG. 4, as the position for detecting the count value, the center position “x0” of the edge, the position “x0 + 1” of one pixel in the moving direction of the edge from the center position, and the position of the edge from the center position. The count value is detected at three positions of the position “x0-1” of one pixel in the direction opposite to the moving direction. However, if the slope of the count value described later is obtained, the arrangement and number of points for detecting the count value are not limited. . That is, as long as the count value can be detected at two or more locations in the edge moving direction, the count value may be detected in any number.

  If the frame rate is set sufficiently higher than the speed at which the edge moves, the edge is detected a plurality of times at the same position between consecutive frames. For example, in the example of FIG. 4, the edge is detected twice at the position x0 in the continuous frame at time t and frame at time t + 1. Therefore, when the count value of the memory address corresponding to the position where the edge is detected is counted up, the count value correlates with the time (number of frames, dwell time) at which the edge is detected at that position. In particular, the smallest count value h of the edge count values represents how many frames the edge has been in the same position after moving.

  Next, an edge moving speed, moving direction, and position calculation method in this embodiment will be described. In this embodiment, the inclination of the count value is calculated, and the moving speed, moving direction, and position of the edge are calculated based on this inclination.

  For example, in the case of FIG. 4E, the count values at the positions “x0-1”, “x0”, and “x0 + 1” are “6”, “4”, and “2”, respectively. When the count value “2” of “x0 + 1” is subtracted from the count value “6” of the position “x0-1”, the slope H of the count value can be calculated as H = (6-2) / 2 = 2.

  This means H = {(time from the edge moving to the position x0-1 to the present) − (time after the edge moves to the position x0 + 1)} / (2 pixels). That is, by calculating the inclination H, the time (number of frames) required for the edge to pass through one pixel at the position x0 is calculated.

  Therefore, the slope H of the count value corresponds to how many frames it takes for the edge to move by one pixel, and the edge moving speed 1 / H is calculated based on the slope H of the count value. In FIG. 4E, since 2 frames are required to move 1 pixel, the edge moving speed is calculated as 1/2 (pixel / frame). Similarly, in FIG. 4F, H = (5-1) / 2 = 2, so that the edge moving speed is ½ (pixel / frame).

  Next, a method for determining the edge moving direction based on the magnitude of the count value will be described. Since the edge moves to a position where there is no edge and the count value at the position where the edge is newly detected is 1, the count value at each position is the smallest value. Therefore, the count value in the direction in which the edge moves is small, and the count value in the direction opposite to the direction in which the edge moves is large. By using this tendency, the moving direction of the edge can be determined.

  Furthermore, if the frame rate is set sufficiently higher than the speed at which the edge moves, it can be assumed that the detection target object moves at a constant speed. In addition, the smallest count value h among the count values at the current position represents the time when the edge is detected at the position, that is, how many frames the edge has moved to stay at the same position. ing. Thus, the edge position can be obtained by “edge position = x0 + h / H” where x0 is the center position of the edge. For example, in FIG. 4F, the edge speed is ½ (pixel / frame), and the edge is detected at the same position continuously for one frame at time t + 2, so the position of the edge at time t + 2 Can be calculated as moving from the position x0 by “1 (frame) × {1/2 (pixel / frame)} = 0.5 pixel”.

  From the above, it is possible to count up the count value of the memory address corresponding to the position where the edge is detected, and to calculate the moving speed and moving direction of the edge based on the slope of the counted up count value.

  Further, the movement information calculation unit 22 classifies the movement information of the edges existing on the captured image into predetermined class values, and generates a movement image that represents the feature of the movement information. FIG. 5 shows an example of the moving image. As shown in FIG. 5, in the moving image of the present embodiment, the pixel of the edge where the movement information is detected is indicated by a circle, and the pixel having the higher movement speed is indicated by a larger circle so that the pixel movement information is obtained. Express. In addition, the moving direction of the pixel is represented by a solid black mark that represents a pixel that moves to the right, that is, the right direction, and the moving direction is represented by a white mark that represents a pixel that moves to the left, that is, the left direction. Express. In FIG. 5, the speed toward the right side of the image is detected from the outer wall on the right side of the traveling road of the host vehicle and the white line, and the speed toward the left side and the right side of the image is detected from the outer wall on the left side of the traveling path. This shows a situation where the left steering is returned to neutral, and the vanishing point is present on the left outer wall. For a pedestrian who moves from the left side to the right side of the travel path, the speed toward the right side of the image is detected. As described above, the moving image can express movement information including the moving speed and the moving direction.

  Next, the grouping unit 23 will be described. The grouping unit 23 groups the pixel groups adjacent in the substantially vertical direction in the image while the movement speed calculated by the movement information calculation unit 22 is within a predetermined range. The grouping unit 23 sets an area for dividing the moving image in order to extract a three-dimensional object from the calculated moving image. That is, as shown in FIG. 6, a plurality of strip-shaped areas are set on the moving image, and the moving image is divided into the plurality of areas. Next, for each region, continuous pixels are grouped at a moving speed within a predetermined range value in the vertical direction of the image, and a three-dimensional object is detected based on the grouped pixels. Specifically, the moving image (see FIG. 5) is searched in the vertical direction (y-axis direction). When a pixel A having movement information is found, a pixel B adjacent to the pixel A is searched. Same if the pixel B has movement information, the difference in the speed direction between the pixel A and the pixel B is within the threshold value Re, and the difference in the speed magnitude between the pixel A and the pixel B is within the threshold value Tv Judged to be continuous vertically at the moving speed. Next, the presence / absence of movement information is similarly determined for the pixel C adjacent to the pixel B, and whether the difference in speed direction between the pixel A and the pixel C is within the threshold value Re or whether the difference in speed magnitude is within the threshold value Tv. Judging. Thereafter, the process is repeated until one of the conditions is not satisfied. Here, the feature that “pixels having common movement information in the vertical direction are continuous” is a feature on a stereoscopic image. That is, each area is scanned from the lower part to the upper part of the image, and when there is a pixel having speed in the area, the speed difference with the pixel having speed adjacent to the pixel is compared. When the speed difference is equal to or less than the threshold value T1, it can be estimated that the objects move at the same speed with respect to the vehicle.

  Next, the coordinate conversion unit 24 will be described. The coordinate conversion unit 24 converts the pixels included in the image into converted coordinates in an overhead view viewed from a predetermined viewpoint. The conversion coordinates are conversion coordinates that are given a predetermined area (area) and are divided into divided areas. First, the coordinate conversion unit 24 detects the upper end position (the pixel existing at the highest position) and the lower end position (the pixel existing at the lowest position) of the pixels grouped by the grouping unit 23. By this process, lower end positions BL1 to BL16 and upper end positions TL1 to TL16 shown in FIG. 6 are detected.

  Then, the coordinate conversion unit 24 has lower end points BP1 to BP16 (not shown) and upper end points TP1 to TP16 (not shown) whose coordinates are the center positions of the lower ends BL1 to BL16 and the upper ends TL1 to TL16 extracted on the xy plane. Are converted into three-dimensional position coordinates (three-dimensional position coordinates in real space) as points on a ZX plane (hereinafter referred to as a prescribed ZX plane) having a prescribed area.

  Here, the coordinates of each point are (x, y), the height from the road surface of the camera is Ch (m), the depression angle of the camera is Tr (rad), the vertical size of the image is Ih, and the horizontal size of the image is Iw. When the angular resolution per pixel in the height direction is PYr (rad) and the angular resolution per pixel in the horizontal direction is PXr (rad), the points TP1 to TP16 and BP1 to BP16 on the xy plane are expressed by the following equations: Is converted into coordinates (Z, X) on the ZX plane.

(Formula 1) Z = (Ch) / (TAN (Tr + (y−Ih / 2) × PYr))
(Formula 2) X = x × TAN ((Z-Iw / 2) × PXr)
The converted points of the upper end points TP1 to TP16 are referred to as upper end coordinate conversion points RT1 to RT16, and the converted points of the lower end points BP1 to BP16 are referred to as lower end coordinate conversion points RB1 to RB16. Some correspondences are shown in FIG.

  Information obtained by the coordinate conversion unit 24 is sent to the determination processing unit 24. Further, movement information and information on feature points are also sent to the determination processing unit 30 as necessary.

  Next, the determination processing unit 30 will be described. The determination processing unit 30 determines whether the target three-dimensional object extracted based on the image processed by the image processing unit 20 is a pedestrian, and detects a pedestrian.

  Specifically, the determination processing unit 30 of the present embodiment includes a determination target extraction unit 31, a pedestrian candidate extraction unit 32, a pedestrian candidate verification unit 33, and a pedestrian determination unit 34.

  Hereinafter, each structure with which the determination process part 30 is provided is demonstrated.

  First, the determination target extraction unit 31 will be described. The determination target extraction unit 31 extracts a target region including a moving target three-dimensional object from the captured image based on the movement information of the feature points calculated by the movement information calculation unit 22 of the image processing unit 20. The determination target extraction unit 31 preferably extracts a target region including a target three-dimensional object that is highly likely to be a pedestrian from the viewpoint that the pedestrian detection device 100 of the present embodiment finally detects a pedestrian. Since the pedestrian is a three-dimensional object and a moving object, the image corresponding to the pedestrian includes a feature as a three-dimensional object and a feature as a moving object. For this reason, the determination target extraction unit 31 has a function of extracting a three-dimensional object as a determination target candidate and / or a function of extracting a moving object as a determination target candidate. As a three-dimensional object extraction method and a moving object extraction method, a known method can be used in addition to the method described below. Here, “move” includes both a case where the three-dimensional object itself moves (actively moves) and a case where the three-dimensional object relatively moves as the pedestrian detection device 100 moves. .

  Although not particularly limited, the determination target extraction unit 31 of the present embodiment extracts a target region including a target three-dimensional object from the captured image by the following two methods.

  As a first method, the determination target extraction unit 31 sets the pixel coordinates present at the highest position (upper end) and the lowest position (lower end) of the converted coordinates among the group of pixels grouped by the grouping unit 23. Based on the coordinates of the existing pixels, a target area including the target three-dimensional object is extracted from the captured image.

  The determination target extraction unit 31 according to the present embodiment includes a pixel located at the lower end of the pixel group grouped by the grouping unit 23 while the coordinate of the pixel located at the upper end in the transformation coordinate belongs to the outside of the transformation coordinate area. If the coordinate of belongs to any of the divided regions, the region corresponding to the grouped pixel group is extracted as the target region including the target three-dimensional object. For example, as in OB1 shown in FIG. 7, the conversion coordinate of the pixel TP1 located at the upper end is located outside the area of the ZX conversion coordinate, and the conversion coordinate of the pixel BP1 located at the lower end is located within the ZX conversion coordinate. The determination target extraction unit 31 extracts a region to which OB1 belongs as a target region including a target three-dimensional object.

  On the other hand, in the pixel group grouped by the grouping unit 23, the determination target extraction unit 31 includes the coordinate of the pixel located at the upper end and the coordinate of the pixel located at the lower end in the transformed coordinates belonging to a common divided region. Extracts a region corresponding to the grouped pixel group as a region including a planar object. Since it is not a three-dimensional object, or even a pedestrian, it is not a three-dimensional object, and therefore an area including the extracted flat object is not extracted as a target area including the target three-dimensional object. For example, like the region including TP10 and BP10 shown in FIG. 7 (corresponding to the white line on the road in the captured image), the transformed coordinates of the pixel TP10 located at the upper end are located in the ZX transformed coordinate region 55 and located at the lower end. Since the conversion coordinates of the pixel BP10 are also located in the area 55 of the ZX conversion coordinates, the determination target extraction unit 31 extracts this area as an area including a planar object.

  In addition, the determination target extraction unit 31 determines whether the upper end coordinate conversion points RT1 to RT16 and the lower end coordinate conversion points RB1 to RB16 are divided on the ZX plane in the coordinates (Z, X) of the ZX plane divided at a predetermined interval. A three-dimensional object is extracted based on whether it belongs to a region (see FIG. 7). Here, when the captured image of the camera 10 moves up and down as the vehicle moves up and down, the area to which the coordinate conversion point belongs may change. In order to avoid this influence, the predetermined interval of the divided areas on the ZX plane is set in meter order. In this embodiment, the x-axis direction is −5.25 m> x, −5.25 ≦ x <−3.5 m, −3.5 m ≦ x <−1.75 m, −1.75 m ≦ x <0 m, 0 m ≦ x <1.75 m, 1.75 m ≦ x <3.5 m, 3.5 m ≦ x <5.25 m, 5.25 m ≦ x, and the z-axis direction is 0 ≦ z <10 m, 10 m ≦ z <20 m, 20 m ≦ z <30 m, 30 m ≦ z <40 m, 40 m ≦ z <50 m, and divided into 5 regions 11 to 15, regions 21 to 25, regions 31 to 35, regions 41 to 41 Area 45, area 51 to area 55, area 61 to area 65, area 71 to area 75, and area 81 to area 85 are set. Further, the region 11 to the region 15 are the region 10, the region 21 to the region 25 are the region 20, the region 31 to the region 35 are the region 30, the region 41 to the region 45 are the region 40, and the region 51 to the region 55 are the region 50. The region 61 to the region 65 are the region 60, the region 71 to the region 75 are the region 70, and the region 81 to the region 85 are the region 80.

  When the upper end coordinate conversion point and the lower end coordinate conversion point that are grouped at the same position in the xy coordinates belong to the same divided area on the ZX plane, the determination target extraction unit 31 determines that the upper end point and the lower end point of the same group are It is determined that the object is on the same road surface, that is, a planar object on the road surface (determined not to be a three-dimensional object). Also, if the upper end coordinate conversion point corresponding to the upper end point is located outside the specified ZX coordinate and the lower end coordinate conversion point of the lower end point of the same group as the upper end point is located within the specified ZX coordinate, the lower end point Since only is on the road surface, it is determined to be a three-dimensional object.

  Here, the movement information calculation unit 22 adds +1 to the counter value of the counter in the area where the coordinate conversion point of the lower end point is located among the counters set in each area in order to detect the road boundary described later. The position information distribution of the lower end point is calculated.

  Also, the coordinate transformation points RT1, RT3, RT5, RT6, RT8, RT9, and RT12 to RT16 of the upper end points TP1, TP3, TP5, TP6, TP8, TP9 and TP12 to TP16 are projected outside the prescribed ZX coordinates shown in FIG. Therefore, the object corresponding to the group including the upper end points TP1, TP3, TP5, TP6, TP8, TP9 and TP12 to TP16 is determined to be a three-dimensional object. That is, the edges including the upper end points TP1, TP3, TP5, TP6, TP8, TP9 and TP12 to TP16 are extracted as three-dimensional objects OB1 to OB11 on the xy plane.

  The coordinate transformation points RT2, RT4, RT7, RT10, RT11 of the upper end points TP2, TP4, TP7, TP10, TP11 are the coordinate transformation points RB2, RB4, RB7, RB10 of the lower end points BP2, BP4, BP7, BP10, BP11, Since it is located in the same region as RB11, it is determined that the group including the upper end points TP2, TP4, TP7, TP10, and TP11 is a planar object located on the road surface.

  In addition, the determination target extraction unit 31 has a pixel group corresponding to the planar object included in the extracted target area along a feature point corresponding to the target area including the extracted target three-dimensional object along a substantially vertical direction of the image. If the target area includes a plane object, the target area including the planar object and the target area including the three-dimensional object are extracted as the target area including one target three-dimensional object.

  Here, the movement information calculation unit 22 adds +1 to the counter value of the counter in the region where the coordinate conversion points RB1 to RB16 of the lower end points are located in order to detect the road boundary described later.

  Furthermore, as a second method, the determination target extraction unit 31 includes a three-dimensional object installation surface included in the image based on the coordinate value of the pixel located in the lowest position in the substantially vertical direction of the image in the group of pixels. When a reference boundary having a predetermined relationship is extracted and the position of the pixel located at the top of the group of pixels grouped by the grouping unit 23 is located above the position of the reference boundary, the grouped pixels A region corresponding to the group is extracted as a target region including a moving target three-dimensional object.

  In this processing, the grouping unit 23 groups pixels that are adjacent to each other in a substantially vertical direction in the image and whose movement speed calculated by the movement information calculation unit 22 is within a predetermined range.

  First, the determination target extraction unit 31 extracts a reference boundary that can define the installation surface of the three-dimensional object. In the present embodiment, a road boundary of a road on which a vehicle on which the pedestrian detection device 100 is mounted is extracted as a reference boundary.

  For this reason, as shown in FIG. 8, based on the obtained position distribution of the coordinate conversion points of the lower end point, an area where there is a high possibility that the reference boundary exists is extracted. That is, in the obtained position distribution of the coordinate conversion points of the lower end point, if there are counter values in a plurality of z-axis regions in the same x-axis region, the road boundary (boundary of the road on which the vehicle travels) ahead of the host vehicle Can be estimated to exist along the vehicle in a straight line, the region having the counter value is extracted as a region including the road boundary as one aspect of the reference boundary. For example, in FIG. 8, since the counter value exists in a plurality of z-axis regions (regions where the lower end points RB2, RB4, and RB7 are located) in the region 30 having the same x-axis range, these regions are used as the road boundary regions. Extract. The same applies to the region 20, the region 50, and the region 80.

  Then, it is determined whether or not a straight line indicating a road boundary exists in the extracted area (see FIG. 8). That is, based on the coordinate conversion point of the lower end point existing in the extracted region, regression analysis is performed in the xy coordinate system, and the slope of the straight line passing through the lower end point is calculated. For example, in the area 30 of FIG. 8, the regression analysis in the xy coordinate system is performed from the coordinate positions of the lower end points BP2, BP4, and BP7 in the xy coordinates corresponding to the coordinate conversion points RB2, RB4, and RB7 of the lower end points existing in the area 20. Then, the slope a3 of the straight line passing through the lower end points BP2, BP4, BP7 is calculated. Similarly, straight line inclinations a2, a5, and a8 corresponding to the projected points of the lower end points located in the regions 20, 50, and 80 are calculated. Then, if the calculated slope of the straight line is within a predetermined range that is defined in advance as a range that can be determined as a road boundary, it is determined that there is a straight line indicating the extracted road boundary. That is, the points (for example, PL1 to PL5 in FIG. 8) whose coordinates are the left end coordinates of the x-axis region where the coordinate conversion point of the lower end point is located and the representative coordinates (for example, center coordinates) of each Z-axis region are converted into the xy coordinate system. Then, the straight line slope Tn0a1 calculated by regression analysis, the right end coordinate of the x-axis area where the coordinate conversion point of the lower end point is located, and a point having the coordinates (for example, the center coordinates) of each Z-axis area as coordinates (for example, FIG. The slope of the straight line an calculated based on the lower end point is included in the range of the slope defined by the slope Tn0a2 of the straight line calculated by converting the PR1 to PR5) of 8 into the xy coordinate system and performing regression analysis. In this case, it is determined that the straight line Ln indicating the road boundary exists in the extracted area.

  For example, as shown in FIG. 9, in the region 80 of the present embodiment, the point PL1 at coordinates (x, z) = (− 3.5, 5), coordinates (x, z) = (− 3.5, 15), point PL2, coordinates (x, z) = (− 3.5, 25), point PL3, coordinates (x, z) = (− 3.5, 35), point PL4, coordinates (x, z) = (− 3.5, 45) point PL5 is converted into xy coordinates and regression analysis is performed, and the slope T30a1 of the straight line connecting the calculated points and coordinates (x, z) = (− 1.75, 5) Point PR1, coordinate (x, z) = (− 1.75,15) point PR2, coordinate (x, z) = (− 1.75,25) point PR3, coordinate (x, z) = ( The slope T30a2 of the straight line connecting the points calculated by performing regression analysis by converting the point PR4 of -1.75, 35) and the point PR5 of coordinates (x, z) = (-1.75, 45) into xy coordinates. On the other hand, since the slope a3 of the straight line connecting the lower end points BP1, BP4, and BP5 on the xy plane corresponding to the projected points RB1, RB4, and RB5 of the lower end point is within the range defined by T30a1 and T30a2, the lower end point It is determined that the straight line connecting BP2, BP4, and BP7 is a straight line L3 indicating the road boundary (see FIG. 9).

  Similarly, in the region 20, it is determined that the straight line connecting BP1 and BP6 corresponding to the projection points RB1 and RB6 of the lower end point is a straight line L2 indicating the road boundary, and in the region 50, the projection point RB10 of the lower end point The straight line connecting the lower end points BP10 and BP11 on the xy plane corresponding to RB11 is determined to be the straight line L5 indicating the road boundary, and in the region 80, the lower end point on the xy plane corresponding to the projected points RB12 to RB16 of the lower end points A straight line connecting BP12 to BP16 is determined to be a straight line L8 indicating a road boundary.

Next, as illustrated in FIG. 10, the determination target extraction unit 31 detects, as a background, a straight line composed of a three-dimensional object that continuously exists far from the vicinity of the host vehicle, out of straight lines indicating the road boundary. In general, the background in the captured image of the front is characterized in that stationary three-dimensional objects are linearly lined away from the vicinity of the host vehicle, the speed of the vicinity is high, and the speed decreases as the distance increases. Therefore, the road boundary showing such characteristics is extracted as the background. Here, when there are a plurality of road boundaries composed of three-dimensional objects, a road boundary located farther from the host vehicle is selected. In the present embodiment, the road boundary lines L2 and L8, which are the intersections of the outer wall and the road surface, are detected as the background.

Next, as shown in FIG. 11, a prediction formula for calculating the speed of the stationary object at an arbitrary position on the image is calculated from the relationship between the detected image position and speed. Since the solid object serving as the background is a stationary object, the speed of the solid object serving as the background represents the speed of the stationary object at each position of the image. For example, the speeds of the three-dimensional objects OB1 and OB4 represent the speeds of the stationary objects at the respective image positions. Therefore, by calculating a prediction formula indicating the relationship between the image position and the speed between a plurality of three-dimensional objects, the speed of a stationary object at an arbitrary position on the image can be obtained. In this embodiment, with respect to the object on the left side of the image, regression analysis is performed using the xy coordinates on the images of the three-dimensional objects OB1 and OB4 as explanatory variables and the speed as an objective variable, and a prediction formula for the speed of a stationary object is calculated. . Similarly, for the object on the right side of the image, regression analysis is performed using the xy coordinates on the three-dimensional object OB7, OB8, OB10, OB11 image as explanatory variables and speed as the objective variable, and a prediction formula for the speed of the stationary object is calculated. To do.

Then, by comparing the position of the feature point detected by the image processing unit 20 and the movement information (movement speed, movement direction) of the feature point with the movement speed of the stationary object obtained from the calculated prediction formula, Detect the body.

  That is, the determination target extraction unit 31 according to the present embodiment is configured to set the three-dimensional object installation surface included in the image based on the coordinate value of the pixel located in the lowest position in the substantially vertical direction of the image among the grouped pixel group. A reference boundary having a predetermined relationship is extracted, predicted movement information of the background of the image is calculated based on the position of the reference boundary and the movement information of the reference boundary calculated by the moving speed calculation means, and the grouping unit 23 When the difference between the movement direction included in the movement information of the grouped pixel group and the predicted movement direction included in the predicted movement information of the background is a relationship in a substantially opposite direction defined in advance, the grouped pixel group Is extracted as a target region including a target three-dimensional object that moves in the course direction of the host vehicle, that is, the direction in which the vehicle exists.

  And if it is a moving body, since it exists in front of the background, the upper end of the three-dimensional object is generally higher in the image than the lower end of the background.

Accordingly, a three-dimensional object having a speed opposite to the speed calculated from the prediction formula and having a higher upper end position than the lower end position of the background is determined to be a moving body that travels in the course direction of the host vehicle.

As shown in FIG. 11, the three-dimensional objects OB2, OB3, OB5, and OB6 have a speed direction substantially opposite to that of the stationary object calculated from the prediction formula, and the upper end position thereof is the lower end position of the background (running road boundary L2). Since it is in a higher position, it is determined that the vehicle is moving in the direction of the course of the host vehicle. Similarly, the three-dimensional object OB9 has a direction of speed substantially opposite to that of the stationary object calculated from the prediction formula, and its upper end position is higher than the lower end position of the background (running road boundary L8). It is determined that the moving body travels in the direction of the course. The contents in the substantially opposite direction are defined in advance. For example, the difference between the two movement directions is 160 ° to 200 °, or the difference between the two movement directions is 170 ° to 190 °.

On the other hand, the difference between the movement direction included in the movement information of the pixel group grouped by the grouping unit 23 and the predicted movement direction included in the predicted movement information of the background is less than a predetermined value. If the difference between the movement speed included in the movement information of the pixel group and the predicted movement speed included in the predicted movement information of the background is less than a predetermined value, the area corresponding to the grouped pixel group is set as a stationary object. Extract as

That is, the direction of the speed of the three-dimensional object is the same direction as the speed of the stationary object calculated from the prediction formula, and the difference between the value of the speed of the three-dimensional object and the value of the speed of the stationary object calculated from the prediction formula is less than a predetermined value. If there is, the solid object is determined as a stationary object, and is not determined as a solid object.

  In addition, when a plurality of target areas are extracted, the determination target extraction unit 31 has a difference in moving direction of feature points corresponding to the target three-dimensional object included in the extracted target areas that is less than a predetermined value, and the target three-dimensional object When the difference in the moving speed of the feature points corresponding to is less than a predetermined value, these target areas are extracted as one target area including one target solid object. For example, in the example illustrated in FIG. 10, the determination target extraction unit 31 groups OB2 and OB3 as one moving target three-dimensional object OB2.

Next, the pedestrian candidate extraction unit 32 will be described. The pedestrian candidate extraction unit 32 selects a target area that includes the detected three-dimensional object including the pedestrian candidate (moving three-dimensional object with a high possibility of being a pedestrian) based on the detected speed change of the moving target three-dimensional object. Extract.

Here, when the moving speed of the feature point corresponding to the pedestrian is observed in detail, when the pedestrian is walking at a constant speed, as shown in FIG. Is observed. Looking at the speed change of the leg from the start of walking, first, for example, the right leg steps in the traveling direction, and walking starts. Then, after the speed gradually increases and reaches the maximum speed, a speed change is observed in which the speed gradually decreases and then reaches the ground. At the same time as the right leg lands on the ground, this time the left leg steps in the direction of travel, and the same speed change as the right leg is observed. On the other hand, when the change in the speed of the arm is seen from the start of walking, a change in the speed opposite to the leg is observed. Thus, a pedestrian has a feature that a speed change is observed at a corresponding feature point by walking.

Therefore, by using this feature, it is possible to identify whether or not a three-dimensional object may be a pedestrian and to extract a target region including a moving target three-dimensional object that is likely to be a pedestrian. it can.

The pedestrian candidate extraction unit 32 of the present embodiment acquires the movement speed of the feature points included in the target area including the target three-dimensional object extracted by the determination target extraction unit 31 from the movement information calculation unit 22, and A variance value of the moving speed of the feature points is calculated, and when the variance value is a predetermined value or more, the target area is extracted as a target area including a pedestrian. Information regarding the extracted target area is sent to a pedestrian candidate verification unit 33 and / or a pedestrian determination unit 34 described later.

That is, for example, the variance value of the speed of the moving body is defined as a change ΔV in the moving speed, and whether ΔV is equal to or greater than a predetermined value ΔVP, that is, whether the pedestrian candidate extraction unit 32 satisfies the condition ΔV ≧ ΔVP. Thus, it is possible to identify whether or not the moving body is a pedestrian. The movement speed change ΔV can be arbitrarily defined, and may be a difference between the maximum speed and the minimum speed or may be a difference from the average speed.

When the dispersion value of the moving speed is calculated for the image information shown in FIG. 10, the dispersion values of the moving bodies of the three-dimensional objects OB2, OB5, and OB9 satisfy ΔV ≧ ΔVP. For this reason, it is determined that the three-dimensional objects OB2, OB5, and OB9 are pedestrian candidates with a high possibility of pedestrians. On the other hand, since the three-dimensional object OB6 does not satisfy the condition, it is determined as an object other than a pedestrian.

In addition, the pedestrian candidate extraction unit 32 is configured such that, even if the calculated variance value of the moving speed is less than a predetermined value, the movement amount of the feature point within the target area within the predetermined time is less than the predetermined value, and the target area If the change in the movement speed of the feature point is equal to or greater than a predetermined value, the target area is extracted as a target area including a pedestrian. As a result, even for a pedestrian who moves slowly, the possibility of being a pedestrian based on the change in the moving amount and moving speed of the feature points without immediately excluding it as a non-pedestrian, the target area including the pedestrian Can be extracted accurately.

Here, as shown in FIG. 10, OB5 and OB9 are determined as pedestrian candidates in this process even though they are not pedestrians. This is because there is a three-dimensional object having a speed change close to the characteristics of a pedestrian when the variance value is used as an index. For example, when the vanishing point is included in the area of the solid object that is the background other than the central part of the image due to the influence of the vehicle behavior, the change in the movement near the vanishing point is small, and the moving speed of the left and right is in that area. Mixed. For this reason, the dispersion value of the speed of the area becomes large, and the direction of the speed is reversed with the vanishing point as a boundary. Therefore, there is a case where a three-dimensional object serving as a background near the vanishing point is erroneously detected as a pedestrian. Further, even in a region other than the vanishing point, an incorrect speed may be measured in a region where complex edges are combined. Even in that area, the dispersion of speed is large, and there is a background with the correct speed in the surroundings, so there is a case where a solid object with a complex edge is mistakenly detected as a pedestrian even though it is a background. is there.

Therefore, in this embodiment, a pedestrian candidate verification unit 33 is provided to improve the detection accuracy of pedestrians.

As shown in FIG. 13, the pedestrian candidate verification unit 33 observes whether or not the target three-dimensional object includes a pedestrian by observing a temporal change in the distribution of the moving speed existing in the detected target area. The candidate is verified, and if it is determined that the candidate is not a pedestrian candidate, it is deleted as noise.

  The pedestrian candidate verification unit 33 verifies that the target area is a target area that does not include a pedestrian when the degree of variation in the moving direction of the feature points included in the extracted target area increases with time.

Since the target region is a region detected as a region including a moving object, the tracking process with time can be limited to a region near the horizontal position. If the target area extracted as containing a pedestrian candidate is an area that contains a stationary object such as the background, the movement speed of the target area for tracking processing will increase over time because the amount of movement of the feature points along the horizontal direction is large. Accordingly, it is measured from the edge of a different three-dimensional object. On the other hand, if the target area includes a pedestrian, the moving speed of the target area for which the tracking process is performed is measured from the edge of the same pedestrian regardless of the passage of time. Moreover, if the target area including the pedestrian candidate is an area including a vanishing point, the position of the vanishing point moves with time, so the moving speed of the target area is determined from the background (three-dimensional object) other than the vanishing point. Changes to the detected moving speed. Similarly, even when the target area includes a background having an incorrect moving speed, the target area changes so as to include a background having a correct moving speed as time elapses.

As described above, since a stationary object has a feature that a horizontal movement amount is larger than that of a moving object, by using this feature, whether or not a feature point included in the target region corresponds to a pedestrian. Can be verified.

The pedestrian candidate verification unit 33 of the present embodiment has a direction difference between the feature points included in the extracted target region and the moving direction of the feature points corresponding to the target three-dimensional object that is a pedestrian candidate being less than a predetermined value. There is a direction between the number N1 of feature points whose speed difference from the moving speed of the feature point corresponding to the pedestrian candidate is less than a predetermined value, and the moving direction of the feature point corresponding to the target three-dimensional object to be a pedestrian candidate The number N2 of feature points whose difference is greater than or equal to a predetermined value is calculated, and when N2> N1 is satisfied and the ratio R1 of N2 to the area of the target region is larger than a predetermined value Rt1, the target The region is verified as a target region that does not include pedestrians.

That is, among the moving speeds in the target area including the pedestrian candidate, the number of speed information in which the difference between the moving speed of the pedestrian candidate in the same direction as the moving speed of the pedestrian candidate is equal to or less than a predetermined value Vt1 is N1, The number of speed information in the opposite direction to the pedestrian candidate is observed for N2 for a predetermined time. When N2> N1 is satisfied and the ratio R1 of N2 to the area area of the pedestrian candidate is larger than the predetermined value Rt1, the pedestrian The candidate area does not include a pedestrian candidate, but is deleted as noise obtained from the background.

On the other hand, the three-dimensional object OB2 is a pedestrian candidate because there is no change in the moving speed in the target area even if time passes.

Further, since the three-dimensional object OB5 is a vanishing point region, the vanishing point moves to the left as time passes, and the moving speed in the target region gradually matches the leftward speed. Therefore, OB5 is determined not to be a pedestrian candidate and is deleted as noise.

Furthermore, the three-dimensional object OB9 changes to a correct moving speed with the passage of time due to erroneous detection of the background speed, and the moving speed in the target area is aligned to the rightward speed. Therefore, OB9 determines that it is other than a pedestrian candidate, and deletes it as noise.

  Thus, different characteristics can be observed in the moving speed of the target three-dimensional object according to the positional relationship between the target area and the vanishing point. For this reason, the pedestrian candidate verification unit 33 determines the time for observing a change in the variation degree of the moving direction of the feature point according to the distance between the extracted target region and the vanishing point. It is preferable that the time for observing the change in the variation degree in the moving direction of the feature point is set longer as the target region and the vanishing point are closer to each other.

  Finally, the pedestrian candidate determination unit 34 will be described.

  The pedestrian candidate determination unit 34 compares the movement information related to the target area extracted by the determination target extraction unit 31 with the movement information related to the comparison area set around the target area, and uses the movement information as a comparison result. It is determined whether or not the target three-dimensional object included in the target region extracted based on the target area is a pedestrian.

  The target area to be compared is not only the area including the moving three-dimensional object extracted by the determination target extraction unit 31, but also the area extracted by the pedestrian candidate extraction unit 32 as having a high probability of including a pedestrian, The candidate verification unit 33 includes a region verified as including a pedestrian.

  Specifically, as shown in FIG. 14, the pedestrian candidate determination unit 34 has a moving speed of a target area that may include a pedestrian and a comparison area (1 to 5 in FIG. 14) set around the moving speed. The movement speed and / or the movement direction are compared, and it is determined whether or not the pedestrian candidate in the target area is finally a pedestrian.

  This determination can prevent a non-pedestrian from being detected as a pedestrian.

That is, when the comparison object (background) existing in the target area and the surrounding area is observed, the pedestrian who moves in the direction of the course of the own vehicle is present in front of the surrounding object (background). Therefore, if the target area detected as a pedestrian candidate includes a pedestrian, a background exists around the target area including the pedestrian candidate. There is a distance difference between the target area including the pedestrian candidate and the comparison area including the background around it. The movement information (direction and speed) of the target area including the pedestrian candidate and the movement information (direction) of the surrounding comparison area And the speed) can be detected.

On the other hand, if the target area is a background (not including a pedestrian), the target area and the comparison area correspond to the same background. If the backgrounds are the same, there will be no distance difference, so no speed difference or direction difference should be detected between the movement speed of the target area and the surrounding comparison area.

Thus, when a pedestrian is included in the target area, between the movement information of the pedestrian candidate included in the target area and the movement information of the three-dimensional object constituting the background included in the comparison area set around it. It is possible to detect a characteristic that a difference occurs in

In the present invention, by using this feature, it is possible to finally determine whether or not the pedestrian candidate in the target area is a pedestrian.

Specifically, the pedestrian determination unit 34 according to the present embodiment includes the movement speed of the target three-dimensional object included in the target region processed by the determination target extraction unit 31, the pedestrian candidate extraction unit 32, and the pedestrian candidate verification unit 33. The moving speed of the comparison object included in the comparison area set around the target area is compared, and if the difference between these moving speeds is a predetermined value or more, the three-dimensional object included in the target area is a pedestrian. Is determined. On the other hand, the pedestrian determination unit 34 determines that the three-dimensional object included in the target region is a three-dimensional object other than the pedestrian when the difference between the moving speeds is less than the predetermined value.

In addition, the pedestrian determination unit 34 of the present embodiment includes the movement direction of the target three-dimensional object included in the target region processed by the determination target extraction unit 31, the pedestrian candidate extraction unit 32, and the pedestrian candidate verification unit 33 and the target thereof. Compare the movement direction of the comparison area including the comparison object set around the area, and if the difference between these movement directions is greater than or equal to a predetermined value, the target three-dimensional object included in the target area is a pedestrian judge. On the other hand, when the difference in the moving direction is less than the predetermined value, the pedestrian determination unit 34 determines that the target three-dimensional object included in the target region is a three-dimensional object other than the pedestrian.

Furthermore, the pedestrian determination unit 34 of the present embodiment is a pedestrian approaching the self in the three-dimensional object included in the target region when the two compared movement directions have a substantially opposite direction relationship defined in advance. Judge that there is. The relationship in the substantially opposite direction can be preset based on 180 degrees such that the difference between the two movement directions is 160 degrees to 200 degrees, or 175 degrees to 185 degrees.

Although not particularly limited, it is preferable that the comparison region is set for each of the divided predetermined regions and the pedestrian is determined. Thereby, the moving speed of the target three-dimensional object can be compared with the moving speed of the background surrounding the target three-dimensional object from four directions.

For example, as a comparison region, an upper comparison region (a region indicated by 1 in FIG. 14) having a predetermined length from the upper end side of the target region to the upper side, and a left side having a predetermined length from the left end side of the target region to the left side One or more of a comparison area (area indicated by 2 in FIG. 14) or a right comparison area (area indicated by 4 in FIG. 14) having a predetermined length from the right end side of the target area to the right side is set.

Furthermore, as a comparison area, another comparison area (a second upper comparison area, a second left comparison area (see FIG. 14) having a predetermined length apart from the upper edge and the left and right edges of the target area. 3), a second right comparison area (area shown by 5 in FIG. 14)). By setting the comparison area at a position separated from the target area by a predetermined distance, the movement speed of the target area and the movement speed of the comparison area can be accurately compared.

Next, an example of pedestrian determination processing when the target region is divided into five regions will be described with reference to FIG. Specifically, in the captured image shown in FIG. 14, a case where the following comparison results are obtained will be described as an example.

  First, the movement in the direction opposite to the moving speed of the detected pedestrian candidate from the upper right end of the target region including the pedestrian candidate to the upper left upper region (the region indicated by 1 in FIG. 14). There is a three-dimensional object with

  Second, a solid having a movement in a direction opposite to the moving speed of the detected pedestrian candidate in a region (a region indicated by 2 in FIG. 14) from the left end side of the target region including the pedestrian candidate to the left side of the predetermined distance D1. Things exist.

  Third, the moving speed in the same direction as the moving speed of the detected pedestrian candidate in the area up to the left predetermined value D2 to D3 (area indicated by 3 in FIG. 14) on the left end side of the target area including the pedestrian candidate. And a three-dimensional object whose speed difference from the detected moving speed is less than Vt2.

  Fourth, there is a three-dimensional object having a movement in the direction opposite to the speed of the detected pedestrian candidate in the region (the region indicated by 4 in FIG. 14) from the right end side of the target region including the pedestrian candidate to the right side of the predetermined value D1. To do.

  Fifth, the movement speed in the same direction as the movement speed of the detected pedestrian candidate in the area from the right end value D2 to D3 on the right end side of the target area including the pedestrian candidate (area indicated by 5 in FIG. 14). And a three-dimensional object whose speed difference from the detected moving speed is less than Vt2.

  When such comparison information is obtained, the pedestrian determination unit 34 determines the pedestrian candidate indicating the third comparison result and the fifth comparison result as an object other than the pedestrian, and the corresponding pedestrian candidate. Exclude from

  And the pedestrian determination part 34 finally determines the pedestrian candidate (target solid object) which shows a 1st comparison result as a pedestrian.

  Furthermore, the pedestrian determination unit 34 finally determines pedestrian candidates (target three-dimensional objects) that satisfy the second comparison result and / or the fourth comparison result as pedestrians.

  In this way, the movement information (including movement speed and movement direction) of the target three-dimensional object detected from the captured image is compared with the movement information of the surrounding comparison area, and the target three-dimensional object is determined based on the comparison result. In order to determine whether or not the person is a pedestrian, it is possible to extract a change in movement specific to the pedestrian from the movement of the target three-dimensional object, and to prevent erroneous detection of a three-dimensional object as a background as a pedestrian Can do.

  Then, the process sequence of the pedestrian detection of the pedestrian detection apparatus 100 of this embodiment is demonstrated based on the flowchart figure shown in FIG.

  FIG. 15 is a flowchart showing the processing of the pedestrian detection device 100 of the present embodiment.

  This process is executed as a program that is activated when an ignition switch (not shown) is turned on.

  First, an image in front of the host vehicle captured by the camera 10 is recorded in the image memory 11.

  In step S <b> 101, the image memory 11 outputs the image in front of the host vehicle recorded on itself to the feature point extraction unit 21 at a predetermined cycle. After this, the flow moves to step S102.

  In step S102, the feature point extraction unit 21 performs edge extraction processing of the captured image, extracts the outline of the object existing in the captured image as an edge image, and normalizes the edge image. After this, the flow moves to step S103.

  In step S103, the movement information calculation unit 22 calculates the edge speed, and calculates a moving image in which the calculated speed is represented by a predetermined gradation. After this, the flow moves to step S104.

  In step S104, the grouping unit 23 sets a strip area for object detection on the calculated moving image. After this, the flow moves to step S105.

  In step S105, the grouping unit 23 checks whether there is a pixel having speed in each strip area from the bottom to the top. When there are pixels having speed, the grouping unit 23 performs grouping assuming that these pixels represent the same object. After this, the flow moves to step S106.

  In step S106, for each group in each strip area, the coordinate conversion unit 24 sets the center coordinate of the pixel located at the top (upper end) as the upper end point and the center coordinate of the pixel located at the lowermost (lower end) as the lower end. Set to point. After this, the flow moves to step S107.

  In step S107, the coordinate conversion unit 24 converts the coordinates of the detected upper end point and lower end point to the ZX plane using (Expression 1) and (Expression 2). After this, the flow moves to step S108.

  In step S108, the determination target extraction unit 31 of the determination target extraction unit 31 defines that the position of the coordinate conversion point of the upper end point and the coordinate conversion point of the lower end point of the same group as the upper end point defines the x-axis range and the z-axis range. It is determined which region of the ZX plane corresponds, and when both the coordinate transformation point of the upper end point and the coordinate transformation point of the lower end point of the same group as the upper end point are located in the same divided region of the prescribed ZX plane, When the object including the upper end point and the lower end point is determined to be a plane object, and only the coordinate transformation point of the lower end point of the same group as the upper end point is located on the specified ZX plane, the object including the upper end point and the lower end point is determined. It is determined that the object is a three-dimensional object. Also, the counter value of the counter in the area where the lower end point is located is incremented by +1. After this, the flow proceeds to step S109.

  In step S109, the solid object extraction unit 31 determines whether the object including the upper end point and the lower end point is a planar object or a three-dimensional object (hereinafter, object attribute determination) for all detected upper and lower end points. Is called). When the object attribute determination is performed for all the detected upper and lower end points, the process proceeds to step S110. On the other hand, when the object attribute determination is not performed for all the detected upper and lower end points, the process returns to step S105.

  Subsequently, in step S110, the determination target extraction unit 31 determines an area where counter values exist in a plurality of z-axis areas in the same x-axis area from the position distribution of the coordinate conversion point of the lower end point on the ZX plane. Extract as a region where a line is highly likely to exist. After this, the flow proceeds to step S111.

  In step S111, a regression analysis in the xy coordinate system is performed on the lower end point of the xy plane corresponding to the coordinate conversion point of the lower end point existing in the region extracted by the determination target extraction unit 31, and a straight line connecting the lower end points is obtained. The slope is calculated. After this, the flow proceeds to step S112.

  In step S112, the determination target extraction unit 31 uses the left end coordinate of the x-axis region and the representative coordinate of each z-axis region as coordinates in the region where the straight line calculated in step S111 is located at the lower end point ( The straight line connecting PL1 to PL5) in FIG. 8 and the right end coordinate of the x-axis region and the range of inclination in the xy coordinate system of the point connecting the representative coordinates of each z-axis region (PR1 to PR5 in FIG. 8). If there is, it is determined that a straight line indicating the road boundary exists in the extracted region, and the straight line calculated in step S111 is detected as a straight line indicating the road boundary. After this, the flow proceeds to step S113.

  In step S113, the determination target extraction unit 31 determines whether or not straight lines indicating all the road boundaries have been detected for the region extracted in step 110. If a straight line indicating the road boundary is detected for all regions, the process proceeds to step S114. On the other hand, when the straight line indicating the road boundary is not detected for all the regions, the process returns to step S110.

In step S <b> 114, the determination target extraction unit 31 uses a straight line constituted by a three-dimensional object that continuously exists far from the vicinity of the vehicle among straight lines corresponding to the road boundary detected in step S <b> 112 as a background. Detect as. After this, the flow moves to step S115.

In step S115, the determination target extraction unit 31 performs multiple regression analysis using the moving speed of the three-dimensional object constituting the background road boundary detected in step S114 as an objective variable and the image xy coordinates at the lower end point as explanatory variables. By doing so, a prediction formula for the speed of the stationary object at an arbitrary position on the image is calculated. After this, the flow moves to step 116.

In step S116, the determination target extraction unit 31 compares the position of the road boundary serving as the background detected in step S114 with the upper edge position of the three-dimensional object, and on the image, the upper edge position of the three-dimensional object rather than the position of the road boundary. Is present as an object three-dimensional object, and an area including the object three-dimensional object is extracted as the object area. After this, the flow moves to step S117.

In step S117, the determination target extraction unit 31 compares the moving speed of the moving object candidate detected in step S116 with the moving speed of the stationary object obtained from the prediction formula at the moving object candidate position, and the absolute value of the speed difference is calculated. If it is within the predetermined range, the moving object candidate is determined as a stationary object. Then, the flow moves to step S119 . On the other hand, if the absolute value of the speed difference is outside the predetermined range, the mobile object candidate is determined as a mobile object, and the flow proceeds to step S118.

In step S118, if the direction of the speed of the three-dimensional object determined as the moving object is opposite to the direction of the moving direction of the stationary object obtained from the prediction formula, the determination target extraction unit 31 sets the moving object as the own vehicle. It is determined that the moving body is heading. The state of “the direction of movement is opposite” can be arbitrarily defined, and is defined in advance by, for example, an angle difference in the movement direction. After this, the flow moves to step S119.

In step S119, the determination target extraction unit 31 determines whether or not a moving object has been detected. If all moving objects have been detected, the flow proceeds to step S120. On the other hand, if all the moving objects have not been detected, the flow returns to step S116.

In step S120, the determination target extraction unit 31 detects a plurality of target three-dimensional objects that have a difference in speed within a predetermined value, and a plurality of targets when the difference in coordinates of the target three-dimensional objects is within a predetermined value. A three-dimensional object is grouped as one target three-dimensional object. After this, the flow moves to step S121.

Next, in step S121, the pedestrian candidate extraction unit 32 calculates the variance value ΔV of the speed of the target three-dimensional object from the speed of the feature point corresponding to the target three-dimensional object, and the calculated variance of the speed of the target three-dimensional object. If the value ΔV satisfies the condition of ΔV ≧ ΔVP, the target three-dimensional object is determined as a pedestrian candidate. After this, the flow moves to step S122.

In step S122, the pedestrian candidate verification unit 33 verifies pedestrian candidates, which will be described later, based on the transition of the velocity information distribution of the target three-dimensional object included in the detected target area. After this, the flow moves to step S123.

In step S123, the pedestrian determination unit 34 determines that the three-dimensional object included in the target region is a pedestrian based on the comparison between the detected movement speed of the pedestrian candidate and the movement speed of the background (three-dimensional object) existing around the pedestrian. A final determination is made as to whether or not. This process will be described later. After this, the flow moves to step S124.

In step S124, it is determined whether or not the detection of the pedestrian has been completed. When the detection of the pedestrian is completed, the flow proceeds to step S125. If the detection of the pedestrian has not ended, the flow returns to step S120, and the detection of the pedestrian is continued.

In step S125, it is determined whether or not the ignition switch of the host vehicle has been turned off. If the ignition switch is not turned off, the process returns to step S101 to repeat the process. On the other hand, if the ignition switch is turned off, the flow moves to step S126 and the process ends.

Next, an operation flow related to the pedestrian candidate verification process in step S122 will be described based on the flowchart shown in FIG. This operation flow is executed by the pedestrian candidate verification unit 33.

In step S1221, the pedestrian candidate verification unit 33 determines the number of speed information N1 in which the speed difference is equal to or less than a predetermined value Vt1 in the same direction as the movement speed of the pedestrian candidate among the movement information existing in the target region, The number N2 of speed information in the direction opposite to the moving speed of the candidate is calculated. After this, the flow moves to step S1222.
In step S1222, the pedestrian candidate verification unit 33 determines that the speed information numbers N1 and N2 calculated in step S1221 satisfy the relationship “N2> N1”, and the ratio R1 of N2 to the detected area area of the target region is determined in advance. When it becomes larger than the set predetermined value Rt1 (R1> Rt1), it is determined that the three-dimensional object included in the target area is noise and is deleted (S1224). On the other hand, if this condition is not satisfied, the flow moves to step S1223.
In step S1223, the pedestrian candidate verification unit 33 observes the degree of variation in the moving direction of the feature points included in the target region over time. In this process, the verification time during which the relationship of “N2> N1” and “R1> Rt1” verified in S1222 is maintained is measured. When the verification time TV1 exceeds the predetermined time T1 (TV1> T1), the pedestrian candidate verification process is terminated, and the flow proceeds to the pedestrian determination process of step S123. On the other hand, if the verification time TV1 does not exceed the predetermined time T1, verification of the pedestrian candidate is continued, and the flow proceeds to step S124.

In step S1224, the three-dimensional object determined as noise is deleted, and the process proceeds to S124.

Next, the operation flow of pedestrian determination in step S123 will be described based on the flowchart of FIG. This operation flow is executed by the pedestrian determination unit 34.

In step S1231 illustrated in FIG. 17, the pedestrian determination unit 34 includes movement information of the target area that is assumed to include the pedestrian candidate and an area that is in contact with the target area, and is located above the upper right end from the upper right end of the target area. The movement information of the three-dimensional object included in the comparison area is compared. Although the moving directions are compared here, the moving speeds may be compared.

In this step S1231, if there is a three-dimensional object having a moving direction opposite to that of the pedestrian in this comparison area, it is determined as a pedestrian. The flow moves to step S1238. On the other hand, if there is no three-dimensional object having the opposite moving direction, the flow moves to step S1232.

In step S1232, the pedestrian determination unit 34 calculates the movement information of the target area that includes the pedestrian candidate and the movement information of the left comparison area (see FIG. 14) from the left end of the pedestrian candidate area to the left side of the predetermined distance D1. Make a comparison. If there is a three-dimensional object having the opposite moving direction in the area, the flow proceeds to step S1233 in order to compare with the three-dimensional object in the right comparison area. If not, the flow moves to step S1234.
Subsequently, in step S1233, the pedestrian determination unit 34 moves information on the target area that includes the pedestrian candidate, and the right comparison area from the right end to the right side of the predetermined distance D1 (see FIG. 14). Compare the movement speeds of (see). If there is a three-dimensional object having the opposite moving speed in the region, it can be determined as a pedestrian, so the flow moves to step S1238. If not, the flow moves to step S1234.

In step S1234, the pedestrian determination unit 34 moves the movement information of the target area that is supposed to include the pedestrian candidate, and the second left-side comparison area (a left-side predetermined value D2 to D3) that is separated from the target area by a predetermined distance. The movement information of the three-dimensional object included in (see FIG. 14) is compared.

If there is a three-dimensional object that has the same direction of movement in the second left comparison area and the speed difference from the detected movement speed is less than PV2, the three-dimensional object included in the target area is determined as noise, and the flow is The process moves to step S1237. If not, the flow moves to step S1235.

In step S1235, the pedestrian determination unit 34 compares the movement information of the target area that is supposed to include the pedestrian candidate and the second right-side comparison from the right end of the target area by a predetermined distance to the right predetermined values D2 to D3. The movement information of the three-dimensional object in the area is compared. If there is a three-dimensional object that has the same direction of movement in the second right comparison area and the speed difference from the detected movement speed is less than Vt2, the three-dimensional object included in the second right target area is determined as noise. The process proceeds to step S1237. If not, the flow moves to step S124.

In step S1238, the pedestrian determination unit 34 determines that the three-dimensional object included in the target area is a pedestrian, and proceeds to step S124.

In step S1237, the pedestrian determination unit 34 determines that the three-dimensional object included in the target region is noise (not a pedestrian), deletes the three-dimensional object determined as the pedestrian candidate from the pedestrian candidate, The process proceeds to step S124.

It progresses to step S124, it is judged whether all the pedestrians were detected, and it goes to the repetition or completion | finish of a process.

Since the pedestrian detection apparatus 100 of this embodiment is comprised and operate | moved as mentioned above, there exist the following effects.

  The pedestrian detection apparatus 100 according to the present embodiment includes a target included in the extracted target area based on the movement information regarding the target area extracted from the captured image and the movement information regarding the comparison area set around the target area. It is determined whether the three-dimensional object is a pedestrian. Thereby, since the change of the relative motion of the target three-dimensional object with respect to the background existing in the vicinity can be observed, it is possible to prevent erroneous detection of the background as a pedestrian.

  The moving three-dimensional object extracted by the determination target extraction unit 31 includes a background object, but the pedestrian determination unit 32 of the present embodiment is configured so that the target three-dimensional object is relative to the three-dimensional object existing in the surrounding area. Since it is determined whether or not the target three-dimensional object is a pedestrian by observing a change in movement, the pedestrian can be accurately identified from the three-dimensional object constituting the background.

Specifically, when the difference between the movement speed related to the target area and the movement speed related to the comparison area is equal to or greater than a predetermined value, it is determined that the three-dimensional object included in the target area is a pedestrian. Therefore, it is possible to reduce pedestrian identification and erroneously detecting the background as a pedestrian.

In addition, when the difference between the movement direction related to the target area and the movement direction related to the comparison area is equal to or greater than a predetermined value, it is determined that the three-dimensional object included in the target area is a pedestrian, thereby walking from a background with a different movement direction. The person can be identified and erroneous detection of the background as a pedestrian can be reduced.

Further, when the movement direction related to the target area and the movement direction related to the comparison area have a predefined substantially opposite direction relationship, by determining that the three-dimensional object included in the target area is a pedestrian approaching the self, While identifying the pedestrian from the background, it is possible to further identify the pedestrian approaching the self (pedestrian detection device 100).

  On the other hand, if the difference in the moving speed is less than the predetermined value as a result of the comparison, it is determined that the three-dimensional object included in the target area is a three-dimensional object other than the pedestrian, and the background and the moving speed are substantially the same. A pedestrian (candidate) is identified as a three-dimensional object other than a pedestrian, and erroneous detection of the background as a pedestrian can be reduced.

  Similarly, if the difference in the moving direction is less than the predetermined value as a result of the comparison, it is determined that the target three-dimensional object included in the target region is a three-dimensional object other than a pedestrian, so that the background and the moving direction are substantially the same. It is possible to reduce pedestrians (candidates) that are identified as solid objects other than pedestrians and erroneously detecting the background as a pedestrian.

  Thus, according to the present embodiment, the movement information (movement speed and / or movement direction) is common if the same background is configured, and the movement information is different if the pedestrian is separate from the background. From the viewpoint, it is possible to accurately and easily determine whether the target three-dimensional object is a pedestrian based on comparison of movement information between the target area and the comparison area. In particular, when there are a mixture of installation objects that make up the background and pedestrians that move freely, such as a front view seen by a moving observer, the movement information of the installation is determined by the movement of the observer. It is possible to accurately determine whether or not the target three-dimensional object is a pedestrian based on the information comparison result.

  The comparison area set around the target area is an upper comparison area having a predetermined length from the upper end of the target area to the upper side, and a left comparison area having a predetermined length from the left end side to the left side of the target area. Or any one or more of the right comparison region having a predetermined length from the right end side to the right side of the target region, so that the movement information of the target three-dimensional object and the background surrounding it from various directions Comparison can be performed, and it can be accurately determined whether or not the target three-dimensional object is a pedestrian.

  Furthermore, by setting the comparison area at a position separated from the upper edge, right edge, and left edge of the target area by a predetermined distance, the movement information of the target three-dimensional object and the movement information of the background in the comparison area are accurately compared. It is possible to accurately determine whether or not the target three-dimensional object is a pedestrian. In particular, by setting the second comparison area apart from the target area, it is possible to avoid comparing the movement information at the boundary between the areas where the movement information is superimposed in a complicated manner. The movement information can be compared.

  Moreover, the pedestrian detection apparatus 100 of this embodiment is provided with the pedestrian candidate extraction part 32, calculates the dispersion | distribution value of the moving speed of the feature point in an object area | region, and pedestrians the area | region where a dispersion | distribution value is more than predetermined value. By extracting as a target area including a pedestrian, it is possible to accurately extract a target area including a pedestrian based on the characteristics of an image unique to the pedestrian.

  In addition, even if the variance value of the moving speed is less than the predetermined value, in order to extract the target area including the pedestrian based on the movement amount of the feature point in the target area and the change in the moving speed within the predetermined time, For a pedestrian with a slow movement speed, a target region including the pedestrian can be accurately extracted by a simple method.

  In particular, after extracting the target area including the pedestrian candidate based on the variance value of the moving speed of the feature points, it is included in the target area based on the comparison of the moving speed between this extracted target area and the surrounding comparison area. By determining whether or not the target three-dimensional object to be included is a pedestrian, it is possible to determine whether or not to include a pedestrian based on a different index for a target region that is likely to include a pedestrian. Can be prevented from being erroneously detected as a pedestrian.

Furthermore, the pedestrian detection device 100 according to the present embodiment includes a pedestrian candidate verification unit 33, and verifies whether the target region includes a pedestrian based on a temporal change in the degree of variation in the movement information of feature points. By doing this, it is possible to accurately verify whether or not the target region includes a pedestrian based on the characteristics of the pedestrian-specific image over time.

In particular, if the degree of variation in the movement information of feature points increases with time, whether or not the target area includes pedestrians based on the viewpoint that the feature points are not based on the same pedestrian Can be verified accurately.

In addition, when the feature points are observed for a predetermined time, the number N1 of movement information common to the movement information of the target three-dimensional object that is a pedestrian candidate is compared with the number N2 of movement information different from the movement information of the target three-dimensional object. N2> N1, and whether or not the target area includes a pedestrian is determined by determining the degree of variation in the movement information based on the ratio of N2 to the area of the target area being larger than a predetermined value. It can be verified quantitatively.

That is, the number of pieces of movement information N2 different from the pedestrian candidates is larger than the number of pieces of movement information N1 common to the pedestrian candidates (the difference is less than the predetermined number) in the target area during the predetermined time, and the ratio to the target area When there is a change that becomes larger than a predetermined value, it can be verified that the target three-dimensional object is other than a pedestrian. As a result, it is verified whether or not it is an object (noise) other than a pedestrian by observing the transition of the movement information for a predetermined time with respect to the movement information similar to the pedestrian and the background indicating the change of the movement information. can do.

In addition, in the pedestrian detection apparatus 100 of the present embodiment, the pedestrian determination unit 34 determines whether or not the target three-dimensional object is a pedestrian because the determination target extraction unit 31 extracts a target area including the moving three-dimensional object. Can be accurately determined.

In other words, adjacent pixels having the same speed on the captured image are grouped to obtain pixels located at the upper end and lower end thereof, and a plane object, a three-dimensional object, and a moving object are determined based on coordinates after the overhead conversion of the pixels. Therefore, it is possible to easily extract a target area including a three-dimensional object that moves based on a captured image obtained from one camera 10.

In addition, when the coordinates of the pixel located at the upper end in the transformed coordinates belong outside the area of the transformed coordinate having a predetermined area, and the coordinates of the pixel located at the lower end belong to one of the divided areas of the transformed coordinates, they are grouped. A region corresponding to the pixel group can be extracted as a target region including the target three-dimensional object. In other words, adjacent pixels having the same speed on the captured image are grouped to obtain pixels located at the upper end and lower end thereof, and a three-dimensional object can be determined based on the coordinates after conversion to the overhead coordinates of the pixels. Therefore, it is possible to easily extract a target area including a three-dimensional object that moves based on a captured image obtained from one camera 10.

On the other hand, when the coordinates of the pixel located at the upper end and the coordinates of the pixel existing at the lower end belong to a common divided area in the converted coordinates, the area corresponding to the grouped pixel group is set as the target area including the planar object. Can be extracted. That is, it is possible to group adjacent pixels having the same speed on the captured image to obtain pixels located at the upper end and the lower end thereof, and to determine a plane object based on the coordinates after conversion to the overhead coordinates of the pixels. Therefore, it is possible to easily extract a target area including a planar object (an object other than a moving three-dimensional object) based on a captured image obtained from one camera 10.

Furthermore, the determination target extraction unit 31 obtains a reference boundary based on the coordinate value of the upper end pixel and the coordinate value of the lower end pixel, and moves based on the positional relationship between the upper end pixel and the reference boundary. Because it is possible to extract the target area that contains a stationary object, it is possible to detect a moving object based on the speed and positional relationship of the detected three-dimensional object without estimating the observer's movement and calculating the distance to the object. Can be done easily.

Similarly, it is possible to extract a target area including a target three-dimensional object that approaches itself according to the relationship between the predicted movement direction of the background calculated from the movement information of the reference boundary and the movement direction of the target three-dimensional object.

In this way, it is assumed that the background is a stationary three-dimensional object that exists continuously from the vicinity of the host vehicle (observer), and the road boundary is assumed to be one of the background from the position of the coordinate transformation of the lower end point. It is possible to detect the presence of the background and the movement information of the background, and to extract the target area including the moving body.
In addition, since the background is a stationary object, the relationship that the speed near the own vehicle (observer) is large and the speed decreases as the distance increases. Therefore, if a prediction formula that satisfies the relationship between the image position of the background and the speed is calculated, the speed of the stationary object at an arbitrary position on the image can be estimated. Then, by comparing the estimated speed information with the speed information obtained by the detection, it is possible to efficiently determine whether the object is a stationary object or a moving object.

Furthermore, by extracting a target area of a pixel group whose difference in movement information on an image is less than a predetermined value as one target area, even when a plurality of edges are detected in one mobile object, one mobile object It can be determined that

In addition, since the feature point is a pixel or a pixel group corresponding to the edge of the object included in the captured image, a pedestrian can be easily detected by detecting the edge of the object on the captured image.

Second Embodiment
Next, the second embodiment will be described. The second embodiment is characterized by the contents of processing of the pedestrian candidate verification unit 33. Except for this point, the basic configuration and processing of the second embodiment are common to those of the first embodiment, and therefore, different points will be mainly described here.

  The pedestrian candidate verification unit 33 verifies the target region as a target region that does not include a pedestrian when the degree of variation in the moving direction of the feature point of the three-dimensional object included in the target region increases with time.

  In addition to this, in this embodiment, the pedestrian candidate verification unit 33 considers the position of the vanishing point in the captured image, and the degree of variation in the moving direction of the feature point according to the distance between the extracted target region and the vanishing point Determine the time (verification time) for observing changes.

The vanishing point in the captured image becomes one of the causes of erroneous detection as to whether or not the three-dimensional object is a pedestrian depending on the position. Further, the amount of future movement can be predicted from the position of the vanishing point on the image. If the vanishing point is at the end of the image, it is considered that the steering amount is large, so that the future movement amount becomes large. That is, it can be said that a large amount of movement can be detected in a short time. Therefore, when the vanishing point is at the edge of the image, the verification time for observing the amount of change in the moving direction can be reduced. On the other hand, as the vanishing point approaches the center of the image, the steering amount decreases, and the future movement amount decreases. Therefore, it is necessary to take a long verification time.

Based on the above, the verification time for observing the variation in the moving direction based on the position of the detected pedestrian candidate (target region) on the image, that is, the positional relationship between the vanishing point and the pedestrian candidate (target region). Can be changed.

The relationship between the vanishing point position and the verification time can be arbitrarily defined in advance. For example, the verification time can be determined based on the horizontal coordinate of the target area. In addition, an outer region along the edge of the captured image and an inner region that is inside the outer region and defined in advance from the center of the captured image are defined in advance, and verification is performed based on which region the vanishing point belongs to. The time may be determined, or the verification time may be determined according to the distance from the outer edge or center of the captured image.

Although not particularly limited, when the vanishing point belongs to the outer region (or when the vanishing point is close to the outer region or the vanishing point is far from the center), the verification time is set to be relatively short, and the vanishing point belongs to the inner region. In the case (or when the vanishing point is far from the edge or when the vanishing point is close to the center), the verification time is set relatively long.

  FIG. 18 shows a pedestrian verification process in the present embodiment. This pedestrian verification process corresponds to the process of step 122 in the flowchart shown in FIG.

In step S122A, the pedestrian candidate verification unit 33 sets the verification time according to the position of the pedestrian candidate image detected in step S121. After this, the flow moves to step S122B.
Note that steps S122B to S122E are the same processes as steps S1221 to S1244 shown in FIG. 16 described above.

  According to the pedestrian detection apparatus 100 of the present embodiment, in the pedestrian candidate verification process, the time for observing a change in the variation degree of the moving direction of the feature point according to the distance between the extracted target region and the vanishing point is set. Therefore, an appropriate verification time can be set, and the processing time can be shortened as a whole.

<Third Embodiment>
Subsequently, the third embodiment will be described. The third embodiment is characterized in the contents of processing of the determination target extraction unit 31. Except for this point, the basic configuration and processing of the third embodiment are common to those of the first embodiment, and therefore, different points will be mainly described here.

Based on the movement information of feature points, when extracting the target area including the target solid object that moves from the captured image, for example, the same three-dimensional object is extracted as a different three-dimensional object due to the result of edge detection. May end up.

The determination target extraction unit 31 according to the present embodiment reliably detects a planar object and a three-dimensional object even when the determination target extraction unit 31 cannot group the feature points of one solid object or moving object into the same group. .

  Specifically, the determination target extraction unit 31 according to the present embodiment applies a pixel group corresponding to a planar object included in the extracted target region to a pixel group corresponding to a three-dimensional object included in the extracted target region. Are extracted along the substantially vertical direction, the target area including the planar object and the target area including the three-dimensional object are extracted as the target area including the common target three-dimensional object.

Based on the image information shown in FIG. 19, the determination target extraction process of the present embodiment will be described. As shown in FIG. 19, of the _P 1 _{1 to P} 1 13 detected on the outer wall (see Figure _3), P ₁ 1~P 1 3 and _P 1 _4~P 1 13 are grouped separately of the _{P 6 1~P 2 12, P 6} 1~P 6 3 and _P 2 _4~P 2 12 are separately grouped, among _{P 8 1~P 8 11, P 8} 1~P 8 3 process when the P ₈ 4~P ₃ 11 are grouped separately, of the P ₉ 1 to P ₉ 12, in which P ₉ 1 to P ₉ 3 and _P 4 _4~P 4 12 are grouped separately Will be described.

First, an area for detecting an object is set in the calculated moving image. That is, as shown in FIG. 19, a plurality of strip-shaped areas (hereinafter referred to as strip areas) are set on the moving image, and the moving image is divided. In this embodiment, for each strip region, pixels having a speed from the lower end to the upper end of the image are sequentially scanned and detected, and the detected pixels are moved to the ZX plane based on the above-described (Equation 1) and (Equation 2). Convert coordinates.

It is determined whether or not the result of the coordinate transformation is located in the ZX plane shown in FIG. If the result of coordinate transformation is located in the ZX plane, it is determined as a candidate point for a plane object. On the other hand, if it is not located in the ZX plane, it is determined as a candidate point for a three-dimensional object. Note that the same region as that described in the above-described embodiment is set on the ZX plane, and description of the region setting method is omitted.

Specifically, in the example shown in FIG. _19, P ₁ 1~P 1 3 is determined as a candidate point of the planar articles located in the same area in the ZX _plane, P 1 _4~P 1 13 is the ZX plane Therefore, it is determined as a three-dimensional object candidate point. _{Similarly, P 2 1~P 2 3, P} 3 1~P 3 3, P 4 1~P 4 3, P 5 1~P 5 3, P 6 1~P 6 3, P 7 1~P 7 3 _{, P 8 1~P 8 3, P} 9 1~P 9 3, P 10 1~P 10 3, P 11 1~P 11 3, P 12 1~P 12 3, P 13 1~P 13 3, P _{14 1~P 14 3, P 15 1~P} 15 3, P 16 1~P 16 3 is determined to be planar object candidate _{points, P 1 4~P 1 13, P} 3 4~P 3 15, P 5 4 _{~P 5 15, P 6 3~P 6} 12, P 8 4~P 8 11, P 9 4~P 9 12, P 12 4~P 12 15, P 13 4~P 13 15, P 14 4, P _{15 4~P 15 16, P 16 4~P} 16 17 is determined to the three-dimensional object candidate points.

Next, the solid object is determined based on the determination result of the planar object candidate point and the solid object candidate point. That is, when the point determined to be a planar object candidate point and the point determined to be a three-dimensional object candidate point exist continuously in the vertical direction, a portion of the three-dimensional object that exists near the road surface is present. It can be estimated that it was erroneously determined to be a planar object candidate point. For this reason, an object in which a planar object candidate point and a three-dimensional object candidate point exist continuously in the vertical direction is determined as a three-dimensional object.

For example, in this embodiment, as shown in FIGS. 19 and 20, since the planar object candidate points _P 1 1 to P ₁ 3 and the three-dimensional object candidate points _P 1 _4~P 1 13 is continuous in the longitudinal direction, _{P 1 1} to P ₁ 13 are determined as one solid object OB1.

_Similarly, to determine the P 2 _{1 to P} 2 15 a solid object _{OB2, P} 3 1~P 3 15 as a three-dimensional object _{OB3, P} 6 1~P 6 12 one of the three-dimensional object _OB4, P 8. 1 to the P ₈ 11 determines as a three-dimensional object _OB5, determines P 9 _{1 to P} 9 12 one of the three-dimensional object _OB6, determines P _{12 1 to P} 12 15 as a three-dimensional object _{OB7, P 13} 1 the to P ₁₃ 15 determined as a three-dimensional object _OB8, determines P 14 _{1 to P 14} 4 as a single three-dimensional object _OB9, determines P _{15 1 to P} 15 16 as a three-dimensional object _{OB10, P 16} 1 to P ₁₆ 17 are determined as one solid object OB11 (see FIG. 20).

Further, out of the points determined to be a plane object and a three-dimensional object, the lower end point is extracted, and the ZX plane area counter where the coordinate conversion point to the ZX plane of the lower end point is located is the same as in the first embodiment. The counter value is added by +1 to calculate the position distribution of the lower end point. For example, in this embodiment, P ₁ 1, P ₂ 1, P ₃ 1, P ₄ 1, P ₅ 1, P ₆ 1, P ₇ 1, P ₈ 1, P ₉ 1, P ₁₀ 1, P ₁₁ 1 , P ₁₂ 1, P ₁₃ 1, P ₁₄ 1, P ₁₅ 1, P ₁₆ 1 are detected as lower end points, and the coordinate conversion points RB _{1 to} RB _{16 of the} respective lower end points P 11 to P ₁₆ 1 shown in FIG. 1 is added to the counter value of the counter of the area to be processed. Hereinafter, by performing the same process as that of the above-described embodiment, it is possible to detect the road boundary, the three-dimensional object, and the moving object in the three-dimensional object.

After detecting the moving body, a pedestrian can be detected by performing the same operation as in the above-described embodiment.

FIG. 21 is a flowchart showing the processing of the pedestrian detection device 100 in the present embodiment. The process shown in FIG. 20 is executed as a program to be started when the ignition switch is turned on. In FIG. 20, the same processing contents as those in the flowchart of the embodiment of the present invention shown in FIG. 15 will be described with a focus on different processing given the same step number and surrounded by a two-dot chain line.

In step S131, scanning is performed from the lower end to the upper end of the image within the strip area for object detection set in step S104, and the coordinates of pixels having speed are expressed by ZX using (Equation 1) and (Equation 2). Transform coordinates on a plane. After this, the flow moves to step S132.

In step S132, if the coordinate conversion point of the feature point pixel is located in the ZX plane, the determination target extraction unit 31 determines that the feature point is a plane object candidate point, and the feature point pixel coordinate conversion point is the ZX plane. If it is not located within, it is determined as a solid object candidate point. After this, the flow proceeds to step S133.

In step S133, all the pixels for which the velocity has been calculated are subjected to coordinate conversion on the ZX plane, and it is determined whether or not the detection of the planar object candidate point and the three-dimensional object candidate point has been completed. When the detection of the planar object candidate point and the three-dimensional object candidate point is completed, the flow proceeds to step S134. On the other hand, when the detection of the planar object candidate point and the three-dimensional object candidate point is not completed, the flow returns to step S131, and the detection of the planar object candidate point and the three-dimensional object candidate point is continued.

In step S134, when the planar object candidate point and the solid object candidate point are continuously present in the vertical direction in each strip region, the determination target extraction unit 31 selects an object including the planar object candidate point and the solid object candidate point. Judge as one solid object. After this, the flow proceeds to step S135.

In step S135, the determination target extraction unit 31 detects the lowest point among the planar object candidate points as the lower end point, and the counter of the area where the coordinate conversion point on the ZX plane of the detected lower end point is located. Add +1 to the counter value. After this, the flow proceeds to step S136.

In step S136, the determination target extraction unit 31 determines whether the calculation of the position distribution information of the lower end point of each area is completed based on the counter value in each area. When the calculation of the position distribution information of the lower end point of each region is completed, the flow proceeds to step S110. On the other hand, if the calculation of the position distribution information of the lower end point of each region has not been completed, the flow returns to step S134, and the calculation of the position distribution information of the lower end point is continued.

The processing after step 110 detects a pedestrian by performing the same processing as the processing shown in FIG. 15 described above.

  As described above, according to the pedestrian detection apparatus 100 of the present embodiment, a feature is observed in which a pixel group of a planar object is connected to a pixel group of a three-dimensional object and is observed in a single three-dimensional object in a substantially vertical direction of the image. Thus, even if one solid object is determined separately as a planar object and a three-dimensional object, it can be determined as one target three-dimensional object and a corresponding target region can be extracted.

As a result, in image processing including feature point extraction, even when pixels are included in the same three-dimensional object and the equivalent movement speed is not calculated due to some influence, the three-dimensional object can be identified with high accuracy. it can.

  The embodiment described above is described for facilitating the understanding of the present invention, and is not described for limiting the present invention. Therefore, each element disclosed in the above embodiment is intended to include all design changes and equivalents belonging to the technical scope of the present invention.

  That is, in this specification, although the vehicle-mounted apparatus 1000 containing the pedestrian detection apparatus 100 which concerns on this invention is demonstrated as the one aspect | mode, this invention is not limited to this.

  In the present specification, as one aspect of the pedestrian detection device 100, a feature point extraction unit 21 as an example of a feature point extraction unit, a movement information calculation unit 22 as an example of a movement information calculation unit, and a determination target extraction Although the pedestrian detection apparatus 100 provided with the determination object extraction part 24 as an example of a means and the pedestrian determination part 34 as a pedestrian determination means was demonstrated, it is not limited to this.

  Further, as described in this specification, the pedestrian detection apparatus 100 includes a pedestrian candidate extraction unit 32 as an example of a pedestrian candidate extraction unit and a pedestrian candidate verification unit 33 as an example of a pedestrian candidate verification unit. And can be provided. In addition, a grouping unit as an example of a grouping unit and a coordinate conversion unit 24 as an example of a coordinate conversion unit can be provided.

  Although the embodiment of the present invention has been described in detail with reference to the drawings, the embodiment is merely an example of the present invention, and the present invention is not limited only to the configuration of the embodiment. Accordingly, it is a matter of course that the present invention includes any design change within a range not departing from the gist of the present invention.

For example, the block diagram is not limited to that shown in the above-described embodiment, and may have other configurations having equivalent functions.

  The camera mounting position is not limited to the position described in the embodiment. The optical axis of the camera faces the front front direction (Z direction) of the vehicle, and the horizontal axis and the vertical axis of the imaging surface are substantially the same as the road surface, respectively. What is necessary is just to set so that it may become parallel and substantially perpendicular | vertical.

In addition, in normalizing the detected edge width, the edge width is not limited to three pixels, and an arbitrary number of pixels can be set. In this case, since the pixel at the center of the edge is used in the subsequent processing, the number of pixels with the edge width is desirably an odd number.

Further, the number of strip regions set by dividing the xy plane is not limited to that shown in the above embodiment, and can be set by dividing it into an arbitrary number.

Further, the number of regions set by dividing the ZX plane is not limited to that shown in the above embodiment, and can be set by dividing it into an arbitrary number.

Further, the vertical and horizontal ranges of the ZX plane can be set to arbitrary values.

Moreover, although the said embodiment demonstrated the example which mounts the pedestrian detection apparatus 10 in the vehicle which drive | works a road, you may mount in another mobile body.

Furthermore, in the above-described embodiment, the example of the curbstone, the white line, and the contact point between the outer wall and the road surface has been described as the road boundary to be detected, but is not limited thereto, for example, the guard rail, the boundary between the parked vehicle and the road surface, You may detect the boundary with areas (fields, fields, etc.) other than a road surface.

It is a figure which shows an example of the block configuration of the pedestrian detection apparatus of this embodiment. (A) And (B) is a figure which shows the example of mounting of the camera 10. FIG. It is an example of the image ahead of the vehicle imaged with the camera 10 mounted on the vehicle. (A)-(f) is a figure for demonstrating the calculation process example of a moving speed. It is a figure which shows an example of a moving image. It is a figure for demonstrating the example of a grouping process. It is a figure for demonstrating the example of a process which extracts the area | region containing a solid object as an object of determination. It is a figure for demonstrating the example of a method which extracts a track boundary. It is a figure for demonstrating the example of the method of extracting the runway boundary used as a background. It is a figure for demonstrating the example of a method of extracting the target solid object which moves based on the extracted track boundary. It is a figure for demonstrating the process example which extracts the moving solid object as a target solid object. It is a figure which shows the example of a characteristic of the speed change of a pedestrian's leg part and an arm part. It is a figure for demonstrating the example of a verification process of a pedestrian candidate. It is a figure for demonstrating the determination process of a pedestrian. It is a flowchart figure which shows an example of a pedestrian detection process. It is a flowchart figure which shows an example of a pedestrian candidate verification process. It is a flowchart figure which shows an example of a pedestrian determination process. It is a flowchart figure which shows the other example of a pedestrian candidate verification process based on 2nd Embodiment. It is a figure for demonstrating the example of a process for extracting the object solid object to move. It is another figure for demonstrating the process example for extracting the object solid object to move. It is a flowchart figure which shows the other example of the determination target extraction process based on 3rd Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1000 ... In-vehicle apparatus 100 ... Pedestrian detection apparatus 10 ... Camera 11 ... Image memory 20 ... Image processing part 21 ... Feature point extraction part 22 ... Movement information calculation part 23 ... Grouping part 24 ... Coordinate conversion part 30 ... Determination processing part 31 ... Determination object extraction unit 32 ... Pedestrian candidate extraction unit 33 ... Pedestrian candidate verification unit 34 ... Pedestrian determination unit 200 ... Vehicle controller 300 ... Alarm device 400 ... Driving support device 500 ... Output device

Claims (20)

Imaging means mounted on the vehicle ;
Feature point extracting means for extracting feature points from an image of the front of the vehicle imaged by the imaging means;
Movement information calculation means for calculating movement information including movement speed and movement direction of the feature points extracted by the feature point extraction means;
Based on the movement information of the feature points calculated by the moving information calculating unit, with including a pixel adjacent to Oite longitudinally to the captured image, the object comprising a target three-dimensional object which moves out of the image A determination target extraction means for extracting a region;
When the variation degree of the movement information along the horizontal direction of the feature points included in the extracted target area increases with time, the target area is verified not to include a pedestrian and does not include the pedestrian. Pedestrian candidate verification means for deleting the verified target area,
The movement information related to the target area subjected to the verification process is compared with the movement information related to the comparison area set around the target area, and the movement speed related to the target area is set around the target area. When the difference between the movement speed relating to the comparison area is equal to or greater than a predetermined value and the difference between the movement direction relating to the target area and the movement direction relating to the comparison area set around the target area is equal to or greater than a predetermined value A pedestrian detection device comprising: a pedestrian determination unit that determines that a target three-dimensional object included in the target region is a pedestrian . In the pedestrian detection device according to claim 1 ,
The pedestrian candidate verification means observes the feature points included in the extracted target area for a predetermined time, and the direction difference in the moving direction of the feature points corresponding to the target three-dimensional object to be a pedestrian candidate is less than a predetermined value. The number N1 of feature points in which the speed difference of the feature points corresponding to the pedestrian candidate is less than a predetermined value, and the direction of the direction of movement of the feature points corresponding to the target three-dimensional object to be a pedestrian candidate When the number N2 of feature points having a difference equal to or larger than a predetermined value is calculated, N2> N1 is satisfied, and the ratio R1 of N2 to the area of the target region is larger than a predetermined value Rt1, A pedestrian detection device that verifies a target area as a target area that does not include a pedestrian. In the pedestrian detection device according to claim 1 or 2 ,
The said pedestrian candidate verification means is a pedestrian detection apparatus which determines the time which observes the variation of the dispersion | variation degree of the moving direction of the said feature point according to the distance of the said extracted object area | region and vanishing point. In the pedestrian detection apparatus as described in any one of Claims 1-3 ,
The pedestrian determination means, when the movement direction related to the target area and the movement direction related to the comparison area are in a substantially opposite direction relationship defined in advance, the target three-dimensional object included in the target area walks close to itself. A pedestrian detection device that determines that a person is a person. In the pedestrian detection apparatus as described in any one of Claims 1-4 ,
The pedestrian determination unit determines that the target three-dimensional object included in the target region is a three-dimensional object other than a pedestrian when the difference in movement speed is less than a predetermined value as a result of the comparison. . In the pedestrian detection device according to any one of claims 1 to 5 ,
The pedestrian determination device determines that the target three-dimensional object included in the target region is a three-dimensional object other than a pedestrian when the difference in the movement direction is less than a predetermined value as a result of the comparison. . In the pedestrian detection device according to any one of claims 1 to 6,
The comparison region set around the target region is an upper comparison region having a predetermined length from the upper end side to the upper side of the target region, and a left side having a predetermined length from the left end side to the left side of the target region. A pedestrian detection device including any one or more of a comparison region or a right comparison region having a predetermined length from the right end side to the right side of the target region. In the pedestrian detection device according to any one of claims 1 to 7,
The comparison region set around the target region is spaced apart from the upper end side of the target region by a predetermined distance and a second upper comparison region having a predetermined length width upward from the left end side of the target region. A second left side comparison area that is separated by a predetermined distance on the left side and has a width of a predetermined length on the left side, or a second right side comparison that is separated by a predetermined distance from the right edge of the target area and has a width of a predetermined length on the right side A pedestrian detection device including any one or more of regions. In the pedestrian detection device according to any one of claims 1 to 8,
The moving speed of the feature point included in the target area extracted by the determination target extracting unit is acquired from the moving information calculating unit, the variance value of the moving speed of the feature point in the target area is calculated, and the variance value is A pedestrian detection device further comprising pedestrian candidate extraction means for extracting the target area as a target area including a pedestrian when the target area is equal to or greater than a predetermined value. In the pedestrian detection device according to claim 9,
The pedestrian candidate extracting means is configured such that, even if the calculated variance value of the moving speed is less than a predetermined value, the amount of movement of the feature point within the target region within a predetermined time is less than the predetermined value, and the target A pedestrian detection device that extracts the target region as a target region including a pedestrian when a change in the moving speed of a feature point in the region is a predetermined value or more. In the pedestrian detection device according to any one of claims 1 to 10 ,
Grouping means for grouping pixel groups adjacent to each other in a substantially vertical direction in the image, the moving speed calculated by the movement information calculating means being within a predetermined range;
Coordinate conversion means for converting the pixels included in the image into conversion coordinates in an overhead view seen from a predetermined viewpoint; and
The determination target extraction unit is configured to extract the captured image based on the coordinates of the pixel located at the upper end and the coordinate of the pixel located at the lower end among the pixel groups grouped by the grouping unit. A pedestrian detection device that extracts a target area including a target three-dimensional object from inside. The pedestrian detection device according to claim 11 ,
The coordinate conversion means is an overhead view seen from a predetermined viewpoint, and converts the pixel into conversion coordinates divided into predetermined divided regions,
The determination target extraction unit is configured such that, out of the pixel group grouped by the grouping unit, the coordinate of the pixel located at the upper end in the transformed coordinate belongs outside the area of the transformed coordinate and the coordinate of the pixel located at the lower end Is a pedestrian detection device that extracts a region corresponding to the grouped pixel group as a target region including the target three-dimensional object. The pedestrian detection device according to claim 12 ,
In the case where the determination target extraction unit includes, in the pixel group grouped by the grouping unit, the coordinate of the pixel located at the upper end and the coordinate of the pixel existing at the lower end in the converted coordinate belong to a common divided region. A pedestrian detection apparatus that extracts a region corresponding to the grouped pixel group as a target region including a planar object. The pedestrian detection device according to claim 13 ,
The determination target extraction unit includes a pixel group corresponding to a planar object included in the extracted target region in a pixel group corresponding to a three-dimensional object included in the extracted target region in a substantially vertical direction of the image. A pedestrian detection device that extracts a target area including the planar object and a target area including the three-dimensional object as a target area including one target three-dimensional object when connected along. In the pedestrian detection device according to any one of claims 1 to 14 ,
Grouping means for grouping pixel groups adjacent to each other in a substantially vertical direction in the image and having a movement speed calculated by the movement information calculation means within a predetermined range,
The determination target extraction unit is a reference having a predetermined relationship with the installation surface of the three-dimensional object included in the image, based on the coordinate value of a pixel located at the lower end in the substantially vertical direction of the image in the group of pixels. When the boundary is extracted and the position of the pixel located at the upper end of the group of pixels grouped by the grouping unit is located above the position of the reference boundary, it corresponds to the grouped pixel group A pedestrian detection apparatus that extracts a region as a target region including the moving target three-dimensional object. The pedestrian detection device according to claim 15 ,
The determination target extraction unit calculates predicted movement information of the background of the image from the movement information of the reference boundary calculated by the movement speed calculation unit based on the position of the reference boundary, and is grouped by the grouping unit. If the difference between the movement direction included in the movement information of the pixel group and the predicted movement direction included in the predicted movement information of the background is a relationship in a substantially opposite direction defined in advance, the grouped pixels The pedestrian detection apparatus which extracts the area | region corresponding to a group as an object area | region containing the object solid object which moves to the said self direction. The pedestrian detection device according to claim 16 ,
The determination target extraction unit has a difference between a movement direction included in the movement information of the pixel group grouped by the grouping unit and a predicted movement direction included in the predicted movement information of the background is less than a predetermined value, When the difference between the movement speed included in the movement information of the pixel group grouped by the grouping unit and the predicted movement speed included in the predicted movement information of the background is less than a predetermined value, the grouped pixel group The pedestrian detection apparatus which extracts the area | region corresponding to 1 as an area | region containing the said stationary object. The pedestrian detection device according to claim 15 or 16 ,
When the plurality of target areas are extracted, the determination target extraction unit has a difference in moving direction of feature points corresponding to the target three-dimensional object included in the extracted target areas that is less than a predetermined value, and the target solid A pedestrian detection device that extracts these target areas as one target area including a common target three-dimensional object when a difference in moving speed of feature points corresponding to the object is less than a predetermined value. In the pedestrian detection device according to any one of claims 1 to 18 ,
The pedestrian detection device, wherein the feature point is a pixel or a pixel group corresponding to an edge of an object included in the captured image. Calculating each of movement information including the moving speed and moving direction of the feature points extracted from the picked-up image in which the front of the vehicle is picked up by the image pickup means mounted on the vehicle;
Extracting a target region including a target three-dimensional object that moves from the captured image based on movement information of the feature point, and including pixels adjacent in the vertical direction in the image;
Verifying whether or not the target three-dimensional object included in the extracted target region includes a pedestrian candidate based on a temporal transition of movement information present in the detected target region;
The movement information related to the extracted target area is compared with the movement information related to the comparison area set around the target area, and the target three-dimensional object included in the extracted target area is walked based on the comparison result. Determining whether or not a person is a person,
The step of verifying whether or not the target three-dimensional object included in the target area includes a pedestrian candidate is based on the fact that the variation degree of the movement information along the horizontal direction of the feature points included in the extracted target area If the area becomes larger, the target area is verified not to include a pedestrian, the target area verified not to include the pedestrian is deleted,
The step of determining whether or not the target three-dimensional object included in the target area is a pedestrian relates to a moving speed related to the target area where the verification process has been performed and a comparison area set around the target area. When the difference between the moving speed is a predetermined value or more and the difference between the moving direction related to the target area and the moving direction related to the comparison area set around the target area is a predetermined value or more, the target A pedestrian detection method for determining that a target three-dimensional object included in a region is a pedestrian.

Images