JP5141603B2 - Moving object detection apparatus and moving object detection method - Google Patents

Description

  The present invention relates to a moving object detection apparatus and a moving object detection method for detecting a moving object based on a captured image.

  The optical flow of the photographed image is calculated, the comparison target pixel existing in the vertical direction or the horizontal direction of the determination target pixel is assumed to be a point of a stationary object, the prediction speed vector of the determination target pixel is calculated, and the determination target pixel If the difference between the horizontal component of the speed vector and the horizontal component of the predicted speed vector is in the direction of the vehicle and the speed difference is greater than or equal to a predetermined threshold value, the point is determined as a point on the moving object and the position A technique is known in which moving objects are extracted by grouping points where the difference between the difference and the velocity vector is equal to or less than a predetermined threshold (see Patent Document 1).

JP 2005-209019 A

  However, the conventional technique has a problem in that the determination accuracy of the region corresponding to the moving object is lowered when the velocity vector (optical flow) value of each part of the moving object is different.

  The present invention relates to a moving object candidate region including at least a part of moving object candidates from an image, a moving object candidate area having a lower end on the moving direction side, a moving object candidate existing in a lower direction than the lower end, and a leading end on the moving direction side. By detecting the rear end of the moving object candidate existing on the opposite side in the traveling direction, and determining the area including the moving object candidate area and the area to which the lower end, the front end, and the rear end belong as one moving object area, Solve.

  According to the present invention, the front and rear ends of the moving object can be detected even when different velocity vectors are detected depending on the part even though they are the same moving object. Can be determined.

FIG. 1 is a diagram illustrating an example of a block configuration of an in-vehicle device 1000 including a moving object detection system 100 according to the present embodiment. 2A and 2B are diagrams illustrating an example of mounting the camera 20. FIG. 2A is a view of the host vehicle on which the camera is mounted as viewed from the side, and FIG. 2B is a view of the host vehicle on which the camera is mounted as viewed from above. FIG. 3 is an example of an image in front of the host vehicle captured by the camera 20. FIGS. 4A to 4F are diagrams for explaining the movement speed calculation process. That is, it is a diagram for explaining a method of acquiring an edge image obtained by normalizing extracted edges and calculating a moving direction and a moving speed from an edge counter value (residence time). FIG. 5 is a diagram illustrating an example of a moving image. FIG. 6 is a diagram for explaining a process of extracting a three-dimensional object from speed information and extracting moving object candidates by comparing three-dimensional objects moving back and forth in time. 6A shows a three-dimensional object extraction result at time t, and FIG. 6B shows a three-dimensional object extraction result at time t + 1. FIG. 7A shows a case where the moving object is a pedestrian, detects the lowermost position of the moving object from the moving object candidate area, determines whether there is a movable space on the traveling direction side of the lowermost position, It is a figure for demonstrating the process which detects the front-end | tip position of a moving object if it exists. FIG. 7B is a diagram for explaining a process of detecting a rear end position by grouping regions from detected front end positions when the moving object is a pedestrian. FIG. 7C is a diagram for explaining the rear end position determination process when the moving object is a pedestrian. FIG. 8A shows a case where the moving object is a two-wheeled vehicle, detects the lowermost position of the moving object from the moving object candidate area, determines whether there is a movable space on the traveling direction side of the lowermost position, It is a figure for demonstrating the process which detects the front-end | tip position of a moving object if it does. FIG. 8B is a diagram for describing a process of grouping regions from the detected front end position and detecting the rear end position when the moving object is a two-wheeled vehicle. FIG. 8C is a diagram for explaining the determination process of the rear end position when the moving object is a two-wheeled vehicle. FIG. 9A shows a case where the moving object is an automobile, detects the lowermost position of the moving object from the moving object candidate area, determines whether there is a movable space on the traveling direction side of the lowermost position, and It is a figure for demonstrating the process which detects the front-end | tip position of a moving object if it does. FIG. 9B is a diagram for explaining a process of grouping areas from the detected front end position and detecting the rear end position when the moving object is an automobile. FIG. 9C is a diagram for explaining the rear end position determination process when the moving object is an automobile. FIG. 10 is a flowchart for explaining the moving object detection process of the present embodiment.

  A moving object detection system 100 according to the present embodiment is an apparatus that is mounted on a vehicle and detects a moving object that exists in front of the traveling direction of the vehicle from image information obtained by imaging the front of the vehicle.

  An in-vehicle device 1000 including a moving object detection system 100 according to the present embodiment will be described based on the drawings.

  FIG. 1 is a diagram illustrating an example of a block configuration of an in-vehicle device 1000 including a moving object detection system 100 according to the present embodiment.

  As shown in FIG. 1, the in-vehicle device 1000 includes a moving object detection system 100, a vehicle controller 200, an alarm device 300, a travel support device 400, and an output device 500, and these devices are CAN (Controller Area). Network) Connected by other in-vehicle LAN and exchanges information with each other. The alarm device 300 issues an alarm to the driver according to the detection result of the moving object detection system 100. The driving support device 400 executes a driving support function according to the detection result of the moving object detection system 100. Further, the output device 500 presents the detection result of the moving object detection system 100 to the driver via a display and / or a speaker.

  As illustrated in FIG. 1, the moving object detection system 100 includes a camera 20, an image memory 21, and a control device 10.

  Hereinafter, each structure with which the moving object detection system 100 is provided is demonstrated.

  First, the camera 20 and the image memory 21 will be described. The camera 20 is mounted on the vehicle and captures an image in front of the vehicle. The camera 20 is a camera having an image sensor such as a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS). The camera 20 of the present embodiment captures the front of the vehicle in the traveling direction at a predetermined cycle, and sequentially outputs the captured images to the image memory 21 for each frame. The image memory 21 stores an image captured by the camera 20 in an accessible state.

  FIG. 2 shows an installation example of the camera 20. FIG. 2A is a view of the host vehicle on which the camera is mounted as viewed from the side, and FIG. 2B is a view of the host vehicle on which the camera is mounted as viewed from above. In the present embodiment, the camera 20 is installed in the upper part of the vehicle interior facing the front of the vehicle. Then, the optical axis LS of the camera 20 is adjusted so as to be directed in the Z direction of the vehicle traveling direction (driver front direction), the horizontal axis X of the imaging surface is adjusted to be parallel to the road surface, and the imaging surface The vertical axis Y is adjusted to be perpendicular to the road surface.

  FIG. 3 is an example of an image captured in front of the vehicle using the camera 20 of the present embodiment. The image captured by the camera 20 is represented by an xy coordinate system with the vertex at the upper left of the image as the origin. An axis extending rightward from the origin is taken as the x-axis, and an axis extending downward from the origin is taken as the y-axis. Note that the captured image illustrated in FIG. 3 includes an outer wall that is a three-dimensional stationary object, a pedestrian that moves from the right side to the left side of the image, and a white line on the road that is a stationary object (including lines other than white).

  Further, as shown in FIG. 1, the control device 10 of the moving object detection system 100 includes a ROM (Read Only Memory) 12 storing a program for executing the moving object detection processing according to the present embodiment, and the ROM 12. By executing the stored program, a CPU (Central Processing Unit) 11 as an operation circuit that functions as the moving object detection system 100 and a RAM (Random Access Memory) 13 that functions as an accessible storage device are provided. . As an operation circuit, an MPU (Micro Processing Unit), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field Programmable Gate Array) is used instead of or together with the CPU (Central Processing Unit) 11. Etc. can be used.

  Next, processing functions provided in the control device 10 of the moving object detection system 100 will be described. The control device 10 includes a speed information calculation function, a moving object candidate area extraction function, a road surface boundary detection function, a first moving object area determination function, a moving object front end detection function, a moving object rear end function, It has a moving object area determination function and a moving object determination function. And each function is performed by cooperation of the software for implement | achieving each function, and the hardware containing the control apparatus 10 mentioned above.

  Hereinafter, functions realized by the control device 10 of the above-described moving object detection system 100 will be described.

  First, the speed information calculation function of the moving object detection system 100 will be described.

  The control device 10 extracts feature points corresponding to the object in the image based on the information of the image captured by the camera 20, and obtains speed information including the movement direction and / or movement speed of the extracted feature points. calculate. That is, this speed information calculation function executes a feature extraction process and a speed information calculation process.

  First, in order to observe the movement of the imaged object on the image from each image data (frame data) captured by the camera 20, the control device 10 includes a feature part including an extension of the object and a characteristic part of the object. Extract. The control device 10 according to the present embodiment reads an image captured by the camera 20 from the image memory 21, binarizes the read captured image using a predetermined threshold, and extracts an edge of an object present in the image.

  The control device 10 calculates the velocity of the pixel of the characteristic portion obtained from the extracted edge. The control device 10 stores the obtained moving speed and moving direction of the characteristic portion in the RAM 13 in association with the imaging timing identifier or the frame identifier. The pixel speed information includes “pixel moving speed” and “pixel moving direction” together with specifying information for specifying the pixel. Note that if there are a plurality of feature portions in one image data, the speed is calculated for all the feature portions.

  Then, the control device 10 counts up the count value of the pixel at the position where the edge corresponding to the extension of the object is detected based on the information of the image of the object imaged by the camera 20, and sets the inclination of the count value. Based on this, the moving speed and moving direction of the edge are calculated.

  Furthermore, the control apparatus 10 updates the counter value of the pixel counter of the pixel corresponding to the edge contained in each image data with respect to image data having different imaging timings by a predetermined method. Here, the pixel counter is a counter set for each pixel. When the pixel corresponds to the edge, the counter value of the pixel counter is incremented by +1. When the pixel does not correspond to the edge, the counter value of the pixel counter is increased. This is a counter that is set to 0 (initialized). This counter value update process is performed for each frame that is repeatedly imaged by the camera 20 in a predetermined cycle. When this operation is performed, the counter value of the corresponding pixel counter increases for a pixel having a long time corresponding to an edge, while the counter value of the corresponding pixel counter decreases for a pixel having a short time corresponding to an edge.

  The change in the counter value of the pixel counter represents the moving direction and moving amount of the edge. Based on the counter value, the control device 10 calculates the moving direction and moving speed of the edge on the captured image. Since the coordinate system of the image represents the azimuth, the moving direction and moving speed of the edge and the feature corresponding to the edge can be obtained.

  Hereinafter, the speed information calculation method will be described in detail with reference to FIG.

  FIG. 4 is a diagram for explaining speed information calculation processing. A method of acquiring edge images obtained by normalizing extracted edges and calculating a moving direction and a moving speed from an edge counter value (staying time). It is a figure for demonstrating.

  The control device 10 performs binarization processing on the edge image. The binarization process is a process in which a pixel at a position where an edge is detected is set to 1 and a pixel at a position where no edge is detected is set to 0. FIG. 4A shows an example of the binarized image of the extracted vertical edge.

  Next, as shown in FIG. 4B, thinning processing is performed on the generated binary image. The thinning process is a process of reducing the edge width of the detected edge until a predetermined pixel width is reached. That is, the edge width is narrowed by performing thinning processing on each extracted edge. In this example, as shown in FIG. 4B, the edge width of the edge is thinned until the predetermined pixel width becomes one pixel. In this way, the edge is thinned to a predetermined pixel width, thereby setting the center position as the center of the edge. Note that, in this example, an example in which one pixel is thinned is shown, but the number of pixels to be thinned is not particularly limited.

  Next, an expansion process is performed to expand the edge width of the thinned edge. The expansion process is performed so that the edge width is constant from the center position set by thinning toward the edge movement direction, and the edge width is also changed from the center position to the direction opposite to the edge movement direction. It is a process of expanding. In this example, the edge is expanded in the horizontal direction so that the edge width of the thinned edge becomes a width corresponding to three pixels. By this process, the extracted edges are normalized, and an edge image having a uniform width is obtained. Specifically, as shown in FIG. 4C, one pixel is expanded from the edge center position x0 in the edge movement direction (the positive direction of the x-axis) and opposite to the edge movement direction from the edge center position x0. The edge width is expanded to 3 pixels by expanding one pixel in the direction (negative direction of the x axis).

  By performing the thinning process and the expansion process in this way, the edge width of the extracted edge image is standardized by standardizing to a predetermined width in the edge moving direction.

  Subsequently, a count-up process in calculating speed information will be described. The count-up process mentioned here is a process for counting up the value of the memory address corresponding to the position of the pixel where the edge is detected and initializing the value of the memory address corresponding to the position of the pixel where the edge is not detected. It is.

  Hereinafter, the edge count-up processing by the calculation unit 22 will be described with reference to FIGS. For convenience of explanation, here, a case where the edge moves in the positive direction of the x-axis will be described as an example. Even when the edge moves in the negative x-axis direction, the y-axis direction, or two-dimensionally, the basic processing method is common.

  As shown in FIG. 4C, the edge has a center position of the edge at a position x0 in a certain frame. Then, the pixel is expanded from the center position to the position x0 + 1 of one pixel in the edge moving direction, and similarly expanded from the center position to the position x0-1 of one pixel in the direction opposite to the edge moving direction.

  The count value of the memory address corresponding to the position where such an edge is detected, “x0-1”, “x0”, “x0 + 1” is incremented by “+1”. On the other hand, the count value of the memory address corresponding to the position where the edge is not detected is reset.

  For example, as shown in FIG. 4D, edges are detected at positions “x0-1”, “x0”, and “x0 + 1” at time t. Therefore, the count value of the memory address corresponding to each position is incremented by “1”. As a result, the count value at the position “x0 + 1” is “1”, the count value at the position “x0” is “3”, and the count value at the position “x0-1” is “5”.

  Next, as shown in FIG. 4E, since the edge does not move even at time t + 1, the edge is detected at each of the positions “x0-1”, “x0”, and “x0 + 1”. . Therefore, the count values at the positions “x0-1”, “x0”, and “x0 + 1” are further incremented by one. As a result, the count value at the position “x0 + 1” is 2, the count value at the position “x0” is 4, and the count value at the position “x0-1” is 6.

  Further, as shown in FIG. 4F, at time t + 2, the edge is shifted by one pixel in the positive direction of the x-axis, and the edge is detected at positions “x0”, “x0 + 1”, and “x0 + 2”. Therefore, the count value of the memory address corresponding to the positions “x0”, “x0 + 1”, and “x0 + 2” where the edge is detected is counted up. On the other hand, the count value at the position “x0-1” where no edge is detected is reset to “zero”. As a result, as shown in FIG. 4F, the count value at the position “x0 + 2” is 1, the count value at the position “x0 + 1” is 3, and the count value at the position “x0” is 5. Further, the count value at the position “x0-1” where no edge is detected is reset to “0”.

  The control device 10 repeats the process of counting up the count value of the memory address corresponding to the position where the edge is detected and resetting the count value of the memory address corresponding to the position where the edge is not detected. In the description based on FIG. 4, the center position “x0” of the edge as the position for detecting the count value, the position “x0 + 1” of one pixel in the moving direction of the edge from the center position, and the movement of the edge from the center position Although the count value is detected at three positions “x0-1” of one pixel in the direction opposite to the direction, if the slope of the count value to be described later is obtained, the arrangement and number of points for detecting the count value are not limited. That is, as long as the count value can be detected at two or more locations in the edge moving direction, the count value may be detected in any number.

  The count value correlates with the time (number of frames, dwell time) during which an edge is detected at that position. If the frame rate is set sufficiently higher than the moving speed of the edge, the edge is detected a plurality of times at the same position between consecutive frames. For example, in the example of FIG. 4, two edges at time t and time t + 1 are detected at the position x0. Therefore, when the count value at the position where the edge is detected is counted up, the count value becomes equal to the time (number of frames) during which the edge is detected at that position. In particular, the smallest count value h of the edge count values represents how many frames the edge has moved from the same position.

  Next, an edge moving speed, moving direction, and position calculation method in this embodiment will be described. In this embodiment, the count value of the pixel corresponding to the extracted feature position is counted up at a predetermined period, and the speed information of the pixel corresponding to the feature section is calculated based on the slope of the count value.

  For example, in the case of FIG. 4E, the count values at the positions “x0-1”, “x0”, and “x0 + 1” are “6”, “4”, and “2”, respectively. When the count value “2” of “x0 + 1” is subtracted from the count value “6” of the position “x0-1”, the slope H of the count value can be calculated as H = (6-2) / 2 = 2.

  This means H = {(time from the edge moving to the position x0-1 to the present) − (time after the edge moves to the position x0 + 1)} / (2 pixels). That is, by calculating the slope H, the time (number of frames) required for the edge to pass through one pixel at the position x0 is calculated.

  Therefore, the slope H of the count value corresponds to how many frames it takes for the edge to move by one pixel, and the edge moving speed 1 / H is calculated based on the slope H of the count value. In FIG. 4E, since 2 frames are required to move 1 pixel, the edge moving speed is calculated as 1/2 (pixel / frame).

  Next, a method for determining the edge moving direction based on the magnitude of the count value will be described. Since the edge moves to a position where there is no edge and the count value at the position where the edge is newly detected is 1, the count value at each position is the smallest value. Therefore, the count value in the direction in which the edge moves is small, and the count value in the direction opposite to the direction in which the edge moves is large. By using this tendency, the moving direction of the edge can be determined.

  The smallest count value h among the count values at the current position represents the time when the edge is detected at the position, that is, how many frames the edge has moved to stay at the same position. . For this reason, the position of the edge can be obtained from the position of the edge = x0 + h / H where the center position of the edge is x0. For example, in FIG. 4F, the edge speed is ½ (pixel / frame), and the edge is detected at the same position continuously for one frame at time t + 2, so the position of the edge at time t + 2 Can be calculated as moving from the position x0 by “1 (frame) × {1/2 (pixel / frame)} = 0.5 pixel”.

  From the above, it is possible to count up the count value of the memory address corresponding to the position where the edge is detected, and to calculate the moving speed and moving direction of the edge based on the slope of the counted up count value.

  Note that the speed information calculation unit 20 of the present embodiment calculates speed information of the x direction (lateral direction) component of the pixel corresponding to each edge.

  In addition, the control device 10 classifies edge speed information existing on the captured image into a predetermined class value, and generates a moving image that represents the characteristics of the speed information. FIG. 5 shows an example of the moving image. As shown in FIG. 5, in the moving image of the present embodiment, the pixel of the edge where the speed information is detected is indicated by a circle, and the pixel having the higher moving speed is indicated by a larger circle, thereby indicating the speed information of the pixel. Express. In addition, the moving direction of the pixel is represented by a solid black mark that represents a pixel that moves to the right, that is, the right direction, and the moving direction is represented by a white mark that represents a pixel that moves to the left, that is, the left direction. Express. Thus, the moving image represents speed information including the moving speed and the moving direction.

  That is, in FIG. 5, the speed toward the right side of the image is detected in the area corresponding to the outer wall and the white line on the right side of the traveling path of the host vehicle, and the speed toward the left side of the image in the area corresponding to the outer wall on the left side of the traveling path. Is detected. Also, if a pedestrian moving from the right side of the road to the left side has a possibility of a collision on the right side of the optical axis of the camera 20, the central part near the pedestrian's torso is moved toward the right side of the image, The leg is detected as moving toward the left side.

  Next, the moving object candidate area extraction function of the moving object detection system 100 will be described. The control device 10 extracts a region including a feature point whose speed information is less than a predetermined value set in advance as a moving object candidate region including at least a part of the moving object candidate. Specifically, the control device 10 extracts a three-dimensional object (its image area) included in the captured image, and among the image areas extracted as the three-dimensional object, feature points whose speed information is less than a predetermined value set in advance. Is extracted as a moving object candidate region of a moving three-dimensional object that is likely to approach the host vehicle.

  The control device 10 extracts a three-dimensional object to be detected. Based on the velocity information of each pixel of the feature unit, the control device 10 extracts a region in which the pixels having the same velocity information (the difference between the values of the velocity information is less than a predetermined value, and the same hereinafter) are continuous in the vertical direction. . The feature that “pixels having common speed information continue in the vertical direction” is a three-dimensional feature on the image. The control device 10 uses speed information calculated based on the inclination of the count value of the pixel detected in the feature portion in the image when extracting the image area including the three-dimensional object.

  FIG. 6 is a diagram for explaining a process of extracting a three-dimensional object from speed information and extracting moving object candidates by comparing three-dimensional objects moving back and forth in time. 6A shows a three-dimensional object extraction result at time t, and FIG. 6B shows a three-dimensional object extraction result at time t + 1.

  Specifically, the control device 10 searches the moving image shown in FIG. 6A in the vertical direction (y-axis direction). However, when extracting a pixel row having vertically continuous speed information, the upper end pixel and the lower end pixel of the pixel row are not subjected to the processing for determining the commonality of the speed information. When a moving image is searched in the vertical direction and a pixel A (pixel code not shown) having speed information is found, a pixel B adjacent to the pixel A in the vertical direction is searched. When the pixel B has speed information, and the pixel C adjacent to the pixel B in the vertical direction also has speed information, the difference between the moving directions of the pixel B and the pixel C (E3-E2) is within the threshold value Re, and If the difference (V3−V2) in the moving speed between the pixel B and the pixel C is within the threshold value Tv, it is determined that the common pixels are vertically continuous. Next, the presence / absence of speed information is similarly determined for the pixel D adjacent to the pixel C in the vertical direction, and the movement direction of the pixel B and the pixel D (whether E4-E2 is within the threshold value Tv) is within the threshold value Re, or movement It is determined whether the difference in speed magnitude (V4−V2) is within the threshold value Tv. Thereafter, the process is repeated until one of the conditions is not satisfied.

  When the condition is not satisfied with the pixel N, for example, when the pixel N has no moving speed, the number of pixels from the pixel A to the pixel N-1 is counted. If the pixel N has a moving speed but the difference (Vn−Vn−1) from the moving speed of the pixel (N−1) exceeds the threshold value Tv, the number of pixels from the pixel A to the pixel N is counted. To do. Furthermore, when the difference between the speed direction of the pixel N and the moving direction of the pixel N-1 is within a predetermined value, the number of pixels from the pixel A to the pixel N is counted. Thus, the number of pixels whose speed information satisfies a predetermined condition is counted. If the count value is equal to or greater than the threshold value TH, the pixel row is extracted as a partial image region of a three-dimensional object having a feature that speed information is common in the vertical direction.

  With this technique, the control device 10 of the present embodiment extracts three-dimensional objects OB1 (t) to OB11 (t) as shown in FIG.

  Next, the control apparatus 10 extracts a moving object that may approach the host vehicle from the three-dimensional object.

  The control device 10 compares the position and speed information of the three-dimensional object obtained at different timings shown in FIGS. 6A and 6B with different imaging timings. When there is a possibility that the three-dimensional object approaches the own vehicle, the angle between the own vehicle (the camera 10 thereof) and the three-dimensional object is almost constant over time, and therefore the speed information is compared (the vehicle does not approach the own vehicle). It shows a small value (compared to the speed information of the object). That is, in the moving image shown in FIG. 6, the size of the circle corresponding to the moving speed of the approaching three-dimensional object is relatively small. Therefore, there is a high possibility that a three-dimensional object corresponding to a region including pixels with small speed information will approach the host vehicle.

  The control device 10 of the present embodiment compares the speed information of the three-dimensional object detected at different times, and approaches a region including a feature point whose speed information is less than a predetermined value A set in advance to the host vehicle. A moving object candidate area including at least a part of the moving object candidates that can be extracted is extracted. The predetermined value A, which is a threshold value for extracting a moving object (three-dimensional object) that is likely to approach the host vehicle, is arbitrarily set in advance through experiments or the like.

  Furthermore, when there is a possibility that the three-dimensional object approaches the host vehicle, the change in the horizontal position of the screen is small, and the lower end of the image area corresponding to the three-dimensional object is the lower direction of the image (the direction in which y is a larger value). Move to. That is, the lower end of the image area corresponding to the three-dimensional object moves downward in the image, and the three-dimensional object corresponding to the area where the moving amount of the speed information is small is highly likely to be a three-dimensional object approaching the host vehicle.

  In the control device 10 of the present embodiment, the speed information is less than the predetermined value A set in advance, and the moving direction of the feature point located at the lower end of the region including the feature point detected continuously in time series is If it is the direction approaching the host vehicle, that is, the downward direction of the image (the direction in which y is a large value), the moving object candidate including at least a part of the moving object candidate that may approach the host vehicle in that region Extract as a region.

  In other words, the control device 10 is a moving object that has a small moving amount of the horizontal position of the three-dimensional object detected at different times and has a possibility of approaching the host vehicle with the three-dimensional object whose lower end position has moved downward. to decide.

  And the control apparatus 10 extracts the image area | region corresponding to this moving object as a moving object candidate area | region including a part of moving object candidate.

  In the example illustrated in FIG. 6, the control device 10 includes the speed information of the three-dimensional objects OB1 (t) to OB11 (t) extracted at time t1 and the three-dimensional objects OB1 (t + 1) to OB9 (t + 1) extracted at time t + 1. Are compared, and the three-dimensional objects OB8 (t) and OB6 (t + 1) that are in a correspondence relationship are extracted as moving object candidate regions that include at least a part of moving object candidates that may approach the host vehicle.

  Next, the road surface boundary detection function of the moving object detection system 100 will be described. The control device 10 detects the boundary between the moving object candidate region and the road surface region.

  Specifically, the control device 10 scans the speed information in the lower direction of the image (the direction in which y becomes a larger value) from the lower end of the moving object candidate in the moving object candidate region in the moving direction side. The lowermost pixel when not detected in the adjacent pixels in the downward direction is detected as a boundary between the moving object candidate and the road surface area. By this process, the lowest end of the moving object can be obtained.

  Further, the “downward direction” as the scanning direction is a vertically downward direction (a direction in which the X coordinate of the image is the same and the Y coordinate increases), and an oblique downward direction (the X coordinate of the image is different and the Y coordinate increases). Direction) (hereinafter the same). In this process, the control device 10 determines a contact point (a pixel in which no speed information is detected in the adjacent pixel in the lower direction of the image) with the road surface area having the shortest distance from the lower end on the moving direction side of the moving object candidate. Detect as the bottom end.

  Moreover, it replaces with the presence or absence of speed information, and the control apparatus 10 detects the lowest end based on the value of speed information. That is, the control apparatus 10 scans further downward speed information from the lower end of the moving object candidate on the traveling direction side, and the difference from the speed information of the pixel corresponding to the lower end of the moving object candidate on the traveling direction side is a predetermined value B. If the lower speed information is not detected in the downward direction, the lowermost pixel is detected as the boundary between the moving object candidate and the road surface area. The predetermined value B in the process of obtaining the lowermost end of the moving object is set in advance from the viewpoint of determining whether different areas belong to a common object.

  Next, the first moving object region determination function of the moving object detection system 100 will be described. The control device 10 scans the presence / absence of speed information from the boundary between the previously obtained moving object candidate and the road surface area in the traveling direction of the moving object candidate, and if there is a movable area in which no speed information is detected, The obtained moving object candidate area (OB6 (t + 1) in the example of FIG. 6B) is determined as the first moving object area.

  That is, the moving object detection system 100 according to the present embodiment determines whether or not there is a moving space that can move on the traveling direction side of the moving object candidate, and if there is a moving space on the traveling direction side of the moving object candidate, Since it can be confirmed that the moving object candidate is likely to approach the host vehicle, the moving object candidate area is set as the first moving object area. This first moving object region forms part of the corresponding region of the moving object detected by the moving object detection system 100 of the present embodiment.

  Next, the moving object tip detection function of the moving object detection system 100 will be described. The control device 10 detects a pixel located at the boundary (corner portion) between the movable region and the road surface region as the tip of the moving object candidate on the traveling direction side.

  Moreover, the control apparatus 10 detects a front-end | tip as follows, when a feature point is extracted between this road surface area | region and a movable area | region. That is, the control device 10 first extracts a three-dimensional object region in which the speed information common to the vertical direction of the image continues in the region between the boundary with the detected road surface region and the movable region. Then, the control device 10 positions the lowermost pixel of the extracted three-dimensional object region in a predetermined positional relationship with the lowermost pixel detected as the boundary with the road surface region, and also sets the lowermost pixel of the three-dimensional object region. When the difference between the velocity information of the pixel and the velocity information of the lowermost pixel detected as the boundary between the road surface area is less than a predetermined value C, the boundary between the three-dimensional object area, the moving object candidate, and the road surface area is grouped. To do. Further, the control device 10 detects the pixel at the lower end of the moving object candidate in the traveling direction of the grouped region as the tip of the moving object candidate on the traveling direction side. Note that the predetermined value C in the process for obtaining the tip of the moving object is set in advance from the viewpoint of determining whether different areas belong to a common object.

  Specifically, the control device 10 according to the present embodiment is configured such that the image area of the three-dimensional object detected between the road surface area and the movable area is the tip detected earlier (the lowest end detected as the boundary with the road surface area). ) Within a predetermined distance range, and the difference between the speed information of the image area of the three-dimensional object and the speed information of the tip detected earlier (the lowermost end detected as the boundary with the road surface area) is small, and the predetermined value C If it is less than that, it is determined that the three-dimensional object is a part of the moving object candidate connected to the tip detected earlier, and the pixel at the lower end of the moving direction side including this three-dimensional object is determined as the tip of the moving object. Detect as.

  Next, the moving object rear end detection function of the moving object detection system 100 will be described. The control device 10 obtains the rear end of the moving object corresponding to the front end in the traveling direction. The control device 10 scans the velocity information in the direction opposite to the traveling direction of the moving object candidate from the previously detected tip (opposite direction to opposite direction, 180 ° ± α with respect to the traveling direction, hereinafter the same), and obtains in advance. A pixel when the speed information of the predetermined threshold value is not detected in the adjacent pixel in the direction opposite to the traveling direction is detected as the rear end of the moving object candidate in the traveling direction.

  Further, the control device 10 scans the velocity information in the direction opposite to the moving direction of the moving object candidate from the detected tip, and groups the pixels of the velocity information in a predetermined range obtained in advance with the tip pixels. The pixel at the lower end on the opposite direction side of the moving object candidate in the converted region is detected as the trailing end of the moving object candidate. Although not particularly limited, the predetermined value range is obtained in advance from the viewpoint of determining whether the front end and the rear end are the same moving object. In the present embodiment, the predetermined value range is calculated based on tip speed information.

  In other words, the control device 10 at least temporarily stores a pixel that is located on the side opposite to the traveling direction with respect to the tip pixel, and that has velocity information in a predetermined range calculated based on the tip velocity information. Grouping is performed, and among the grouped pixels, the lower end pixel on the side opposite to the moving direction of the moving object candidate is set as the rear end.

  In this process, the pixels to be grouped do not necessarily have to be adjacent to each other. This is because, like pedestrians' legs, they are detected as separate three-dimensional objects, and there is a portion without speed information between them. In the rear end detection process of the present embodiment, a pixel interval (a pixel interval corresponding to the opening of both feet of the pedestrian) that allows grouping with the front end is set in advance. If it is within this pixel interval, the control device 10 determines that the velocity information in the predetermined value range belongs to the rear end detection group even if it is not continuous.

  In addition, the control device 10 determines a predetermined value range used when grouping with the leading pixel as a difference between the estimated speed of the stationary object included in the image and the representative value of the speed information of the pixels belonging to the first moving object area. Obtained based on the absolute value of.

  First, the control device 10 obtains an average value of the speed information in the first moving object region as a representative value of the speed information of the pixel to which the first moving object region belongs. Further, the control device 10 calculates the estimated speed of the stationary object in the first moving object region (moving object candidate region). And the control apparatus 10 sets the value range based on the absolute value of the difference of the average value of the speed information of a 1st moving object area | region and the estimated speed of a stationary object as a predetermined value range in a grouping process.

  Incidentally, the estimated speed of a stationary object is calculated as follows. If the vehicle speed V (km / h) and the position a (Z, X (m)) of the object A are given, the speed information S (pixels / second) of the stationary object at the position a where the object A exists is expressed by Equation 1 and It can be calculated from Equation 2. Here, F is the frame rate (fps), θ is the angle (Rad) between the host vehicle and the object, and PXr is the angular resolution (Rad) per pixel in the horizontal direction. Necessary information such as the speed, frame rate, and the like of the host vehicle is acquired from the vehicle controller 200. The vehicle controller 200 sends necessary information in response to a request from the object detection device 100.

θ = ATAN (X / Z) ... Equation 1
S = F * (ATAN (X / (Z + V / (3.6 * F)) − θ) / PXr) Equation 2
Further, since the positions Z and X of the object A are calculated using the camera parameters from the x and y coordinates at the lower end of the image as shown in equations 3 and 4 described later, the lower end position of the object, that is, the speed on the road surface. Will be calculated.

  However, as the y coordinate, the y coordinate of the pixel located at the tip (corner portion) of the moving object candidate region is used instead of the lower end of the moving object candidate region. As the x coordinate, the horizontal center coordinate of the moving object candidate region is used. Here, Ch (m) represents the camera height, Tr (Rad) represents the camera depression angle, Ih represents the image vertical size, Iw represents the image horizontal size, and PYr (Rad) represents the angular resolution per pixel in the height direction.

Z = (Ch) / (TAN (Tr + (y−Ih / 2) * PYr))...
X = x * TAN ((Z-Iw / 2) * PXr) ... Formula 4
Further, in the above-described detection processing of the trailing edge, the control device 10 groups the pixels located below the predetermined height in the image upper direction with respect to the detected leading edge pixels with the leading edge pixels. If they are the same moving object, the position of the pixel at the rear end is located above the position of the pixel at the front end (the direction in which y decreases). In addition, the height of the moving object can be set to an upper limit according to the size of the detection target. For example, an upper limit corresponding to the length of a general human leg can be set for the length of the leg of the pedestrian. For this reason, the control device 10 groups only the pixels located below the predetermined height in the image upper direction than the detected leading edge pixel as the trailing edge candidates. Thereby, the rear end of the moving object (for example, a leg part of a pedestrian) to be detected can be extracted with high accuracy.

  In addition, in this rear end detection process, the control device 10 groups pixels located within a predetermined distance in the direction opposite to the moving direction of the moving object candidate from the detected front end pixels with the front end pixels. The upper limit of the width of the moving object can be set according to the length and size of the detection countermeasure. For example, an upper limit can be set for the width of the pedestrian, the width of the two-wheeled vehicle, and the width of the vehicle. For this reason, the control apparatus 10 groups only pixels located within a predetermined distance in the horizontal direction of the image from the detected leading edge pixel as a trailing edge candidate. Thereby, the rear end of the moving object (for example, a pedestrian, a two-wheeled vehicle, a motor vehicle) to be detected can be extracted with high accuracy.

  Furthermore, in this rear end detection process, the control device 10 once detects the rear end, and then scans for the presence or absence of speed information from the rear end in the direction opposite to the traveling direction of the moving object candidate. If there is a movable area that is not detected, the rear end detected first is determined. Similarly to the front end, if the rear end is adjacent to the movable region, it can be confirmed that the rear end exists on the extension of the image region corresponding to the moving object candidate. For this reason, in this embodiment, the speed information is further scanned in the direction opposite to the movement direction from the rear end, and if the rear end and the movable region are adjacent to each other, the first detected rear end is determined.

  Next, the second moving object area determination function of the moving object detection system 100 will be described. The control device 10 determines an area including the lower end, the front end, and the rear end of the first moving object area as the second moving object area. Taking a pedestrian as an example, the leg of the pedestrian is determined as the second moving object region. As described above, the second moving object region can be specified by the three points of the upper end position, the right end position, and the left end position.

  Finally, the moving object determination function of the moving object detection system 100 will be described. The control device 10 determines an area including the first moving object area and the second moving object area as the moving object area. Taking a pedestrian as an example, the first moving object region corresponding to the trunk part of the pedestrian and the second moving object region corresponding to the leg part of the pedestrian are combined and determined as a moving object region.

  The first moving object area determination process, the second moving object area determination process, and the moving object area determination process described above will be described with reference to FIGS. 7A to 7C, FIGS. 8A to 8C, and FIGS. 9A to 9C. This will be described in detail.

  FIG. 7A shows a case where the moving object is a pedestrian, detects the lowermost position of the moving object from the moving object candidate area, determines whether there is a movable space on the traveling direction side of the lowermost position, FIG. 7B is a diagram for explaining processing for detecting the tip position of a moving object if present, and FIG. 7B shows a case where the moving object is a pedestrian, and groups the areas based on the detected tip position, and the rear end position. The figure for demonstrating the process which detects this, It is a figure for the case where a moving object is a pedestrian, Comprising: The figure for demonstrating the rear end position determination process.

  8A to 8C illustrate a case where the moving object is a two-wheeled vehicle and corresponds to FIGS. 7A to 7C, respectively. Similarly, FIGS. 9A to 9C are diagrams corresponding to FIGS. 7A to 7C, respectively, when the moving object is an automobile.

  First, based on FIG. 7A, an example of a tip detection method will be described using a case where the moving object candidate is a pedestrian as an example.

  As shown in FIG. 7A, the control device 10 further moves diagonally downward from the lower end (xl, yl) on the traveling direction (arrow direction in the figure) side of the moving object candidate (the image downward direction, the x coordinate value does not change, and the y coordinate Scans for the presence or absence of velocity information in the direction in which the value increases or in the direction in which the x coordinate value changes and the y coordinate value increases (the same applies hereinafter), and the velocity information is not detected in the adjacent pixels in the diagonally downward direction of the image. The pixel at the lower end (xl−1, yl + n) is detected. This lowest end (xl-1, yl + n) is a boundary between the moving object candidate and the road surface area.

  In this example, when a region (movable region) where there is no speed information of the three-dimensional object is detected on the moving direction side of the moving object, the control device 10 has a high possibility of approaching the host vehicle. The region is determined to be a moving object candidate to be detected (moving object candidate approaching the host vehicle). On the other hand, when the movable area is not detected in the traveling direction side of the moving object, the control device 10 has a low possibility of approaching the host vehicle. It is determined that the object is a moving object candidate that does not approach the vehicle.

  In the example shown in FIG. 7A, the control device 10 detects a movable area (ellipse in the figure) from which velocity information is not detected in the traveling direction (left side in the figure) of the moving object candidate from the boundary (xl−1, yl + n) with the road surface area. Therefore, the moving object candidate area OB6 (t + 1) is determined as the first moving object area.

  Furthermore, the control device 10 refers to a three-dimensional object existing between the lowest end (xl-1, yl + n) of the moving object candidate and the movable region, and the lower end position of the three-dimensional object is the lowest end position of the moving object candidate. If the speed difference with the speed information of the three-dimensional object that is within the predetermined range above the moving object candidate and constitutes the lowest end position of the moving object candidate is within the predetermined value, the same moving object is grouped and grouped. The pixel at the lower end on the moving direction side of the moving object candidate in the area is detected as the tip on the moving direction side of the moving object candidate.

  However, in the example of FIG. 7A, since there is no solid object between the lowermost end (xl−1, yl + n) and the movable region, the position of the tip is not changed, and the lowermost end (xl−1) of the moving object candidate. , Yl + n) as a tip.

  Further, based on FIG. 8A, an example of the tip detection method will be described by taking as an example the case where the moving object candidate is a two-wheeled vehicle.

  In the example shown in FIG. 8A, the moving object candidate region is OB7 (t + 1). The lower end of the moving object candidate area OB7 (t + 1) on the traveling direction side is (xl, yl). The lower end of this moving object candidate region scans the speed information from (xl, yl) further downward or diagonally downward (in this example, the direction in which the x coordinate value decreases and the y coordinate value increases). The lowermost pixel (xl-5, yl + n) when not detected in the adjacent pixel in the lower direction of the image is detected. The lowermost pixel (xl-5, yl + n) is the boundary between the moving object candidate and the road surface area.

  Further, in the example shown in FIG. 8A, the control device 10 has a movable region (FIG. 8) in which speed information is not detected in the traveling direction (left direction in the drawing) of the moving object candidate from the boundary (xl−5, yl + n) with the road surface region. Therefore, the moving object candidate area OB7 (t + 1) is determined as the first moving object area. In the example of FIG. 8A, since there is no solid object between the lowermost end (xl-5, yl + n) and the movable region, the position of the tip is not changed, and the lowermost end (xl-5 of the moving object candidate). , Yl + n) as a tip.

  Furthermore, based on FIG. 9A, an example of a tip detection method will be described by taking as an example the case where the moving object candidate is an automobile (four-wheeled vehicle).

  In the example shown in FIG. 9A, the moving object candidate region is OB8 (t + 1). The lower end of the moving object candidate region OB8 (t + 1) on the traveling direction side is (xl, yl). The lower end of this moving object candidate region scans the speed information from (xl, yl) further downward or diagonally downward (in this example, the direction in which the x coordinate value decreases and the y coordinate value increases). The lowermost pixel (xl−2, yl + n) when not detected in the adjacent pixel in the lower direction of the image is detected. The lowermost pixel (xl-2, yl + n) is the boundary between the moving object candidate and the road surface area.

  Further, in the example shown in FIG. 9A, the control device 10 detects a movable region (FIG. 9) in which speed information is not detected in the traveling direction (left direction in the drawing) of the moving object candidate from the boundary (xl−2, yl + n) with the road surface region. Therefore, the moving object candidate area OB8 (t + 1) is determined as the first moving object area.

  In the example of FIG. 9A, a three-dimensional object OB4 (t + 1) exists between the lowermost pixel (xl−2, yl + n) of the moving object candidate and the movable region. The lower end position (xl-4, yl + n−m) of the three-dimensional object OB4 (t + 1) is located above the lowermost pixel (xl-2, yl + n) position (in the direction in which y decreases), It exists within a predetermined distance range from the pixel (xl−2, yl + n). Further, the speed difference from the three-dimensional object OB6 (t + 1) to which the lowermost end (xl-2, yl + n) of the moving object candidate belongs is also within a predetermined range. For this reason, grouping is performed as the same object, and the lower end (xl−4, yl + n−m) on the traveling direction side of the three-dimensional object OB4 (t + 1) is detected as the tip of the moving object candidate.

  By the way, since the moving two-wheeled vehicle shown in FIGS. 8A to 8C and the automobile (four-wheeled vehicle) shown in FIGS. 9A to 9C have a long moving object along the traveling direction, the moving objects have different positions in the length direction. Different velocity vectors (optical flow) may be detected at the site. This is due to the following reason. When the moving object crosses the traveling road of the own vehicle included in the imaging region from right to left, the speed of the closest approach position where the moving object and the own vehicle are closest to each other is zero. A leftward speed is detected on the left side of the closest approach position, and a rightward speed is detected on the right side of the approach position. For this reason, if the object width (length along the moving direction) of the moving object is large, the area of the different velocity vector (optical flow) is detected and divided as the area of the different object even though it is the same object. There is. However, this is a movement on the image, and is always generated because it is affected by the positional relationship between the own vehicle (camera 10) and the moving object, the moving speed and moving direction of the own vehicle, and the moving speed and moving direction of the moving object. However, in such a situation, there is a possibility of being divided into different objects even though they are the same moving object. According to the moving object detection system 100 of the present embodiment, it is possible to determine with high accuracy a region of a moving object in which different velocity vector (optical flow) regions are detected while being the same moving object.

  Here, referring back to FIG. 7B, an example of the detection method of the rear end will be described based on FIG. 7B, taking as an example the case where the moving object candidate is a pedestrian.

  As shown in FIG. 7B, the velocity information is scanned in the direction opposite to the moving direction of the moving object candidate from the tip (xl−1, yl + n), and the pixel of the velocity information in the predetermined value range obtained in advance is changed to the tip pixel ( xl-1, yl + n). Then, the pixel (xl−1 + m, yl + n) at the lower end on the side opposite to the traveling direction of the moving object candidate in the grouped region G7B is detected as the trailing end of the moving object candidate.

  Note that the predetermined range set in advance used in the rear end detection process is, as described above, representative of the estimated speed of the stationary object included in the image and the speed information of the pixel to which the determined first moving object area belongs. Obtained based on the absolute value of the difference from the value. In the example shown in FIG. 7B, the control device 10 uses the average value of the speed information of the pixels in the OB6 (t + 1) area as the first moving object area to represent the speed information of the pixels to which the first moving object area belongs. Find the value. Moreover, the control apparatus 10 calculates the estimated speed of the stationary object contained in an image using Formula 1-Formula 4 mentioned above. At that time, the horizontal center coordinate xc of OB6 (t + 1) is used as the x coordinate, and the y position (y1 + n) at the lowest end of the body scanned further down, not the lower end of the first moving object region, is used as the y coordinate. Is used to calculate the estimated speed of a stationary object.

  Further, as illustrated in FIG. 7B, the control device 10 is a pixel located less than a predetermined distance (less than the object width upper limit in the figure) in the direction opposite to the moving object candidate in the traveling direction from the tip (xl−1, yl + n) pixel. Group only. The predetermined distance is an upper limit of a preset object width. In this example, the predetermined distance is determined based on the average pedestrian stride.

  Thus, when performing grouping for detecting the trailing edge in this example, the control device 10 scans the speed information from the tip in the direction opposite to the traveling direction, and the speed information is divided into a movable area and a road surface area. The height from the corner (the height in the y direction) vertically above the corner that touches is less than a predetermined value, the speed difference from the representative value is less than a set threshold value, and the object width from the tip is less than a predetermined value Pixels that satisfy the condition are grouped.

  In the example shown in FIG. 7B, the control device 10 groups pixels belonging to the three-dimensional object OB5 (t + 1) and the three-dimensional object OB7 (t + 1), and sets the lower end (xl-1 + m, yl + n) in the direction opposite to the traveling direction of the group. , Detected as the trailing edge of the moving object.

  Next, based on FIG. 8B, an example of the detection method of the rear end will be described using a case where the moving object candidate is a two-wheeled vehicle as an example.

  As shown in FIG. 8B, the speed information is scanned from the tip (xl-5, yl + n) in the direction opposite to the traveling direction of the moving object candidate, and the pixel of the speed information in the predetermined range obtained in advance is changed to the tip pixel ( xl-5, yl + n). Then, the pixel (xl−5 + m, yl + n) at the lower end on the side opposite to the traveling direction of the moving object candidate in the grouped region G8B is detected as the trailing end of the moving object candidate.

  Further, as illustrated in FIG. 8B, the control device 10 groups only pixels that are located within a predetermined distance in the direction opposite to the moving direction of the moving object candidate from the pixel at the tip (xl−5, yl + n). The predetermined distance is an upper limit of a preset object width. In this example, the predetermined distance is determined based on the length of the average motorcycle (the width along the traveling direction).

  The grouping method in this example uses the method described in FIG. 7B. In the example shown in FIG. 8B, the control device 10 groups pixels belonging to the three-dimensional object OB4 (t + 1), the three-dimensional object OB5 (t + 1), and the three-dimensional object OB7 (t + 1), and the lower end ( xl-5 + m, yl + n) is detected as the rear end of the moving object.

  Further, based on FIG. 9B, an example of the rear end detection method will be described using a case where the moving object candidate is an automobile as an example.

  As shown in FIG. 9B, the velocity information is scanned from the tip (xl−4, yl + n−m) in the direction opposite to the moving direction of the moving object candidate, and the pixel of the velocity information in the predetermined range obtained in advance is Grouped sequentially with pixels (xl−4, yl + n−m). Then, the lowermost pixel (xl−4 + p, yl + ns) on the opposite side of the moving direction of the moving object candidate in the grouped region G8B is detected as the trailing end of the moving object candidate.

  Further, as illustrated in FIG. 9B, the control device 10 groups only pixels that are located within a predetermined distance in the direction opposite to the moving direction of the moving object candidate from the pixel at the tip (xl−4, yl + n−m). The predetermined distance is an upper limit of a preset object width. In this example, the predetermined distance is determined based on the length of the average motorcycle (the width along the traveling direction).

  The grouping method in this example uses the method described in FIG. 7B. In the example illustrated in FIG. 9B, the control device 10 groups pixels belonging to the three-dimensional object OB4 (t + 1), the three-dimensional object OB6 (t + 1), and the three-dimensional object OB8 (t + 1), and the lower end ( xl−4 + p, yl + n−s) is detected as the rear end of the moving object.

  Here, referring back to FIG. 7C, an example of a method for determining the rear end will be described based on FIG. 7C, taking as an example the case where the moving object candidate is a pedestrian.

  From the viewpoint of improving accuracy, the control device 10 scans for the presence or absence of speed information from the rear end in the direction opposite to the traveling direction of the moving object candidate (pedestrian), and when there is a movable region where the speed information is not detected, The rear end detected first is determined.

  As described above, it is assumed that there is a moving space on the front end side of the moving object, and if the boundary of the moving object is detected based on the presence of the movable region, detection with high accuracy is possible. On the other hand, the rear end has many overlapping portions with the background, and it is difficult to detect with high accuracy. However, as the vehicle travels, the rear end position also moves and a moving space may appear. The appearance of the moving space on the rear end side indicates that there is a space that can avoid the moving object. Therefore, if there is a moving space on the rear end side and a movable region can be detected, the extension can be determined as a boundary with the moving object, so that the rear end can also be detected with high accuracy.

  When a movable region is detected further on the opposite side of the advancing direction at the rear end, the control device 10 scans the speed information from the rear end in a downward direction or a diagonally downward direction, and the difference between the movement speed and the speed of the rear end is detected. A pixel within a predetermined range (moving speed approximates) is detected as the trailing end of the moving object candidate.

  In the example shown in FIG. 7C, pixels having a common (approximate) moving speed are scanned further diagonally downward from the rear end (xr, yr), and positions (xr + 1, yr + n + p) where no speed information exists in adjacent pixels are scanned. Explore. Furthermore, if there is a region (movable region) where the velocity information of the three-dimensional object does not exist in the direction opposite to the traveling direction at this position, the position (xr + 1, yr + n + p) is finally determined as the rear end.

  The control device 10 determines the region G7C including the lower end, the front end, and the rear end of the first moving object region as the second moving object region. Then, the control device 10 determines an area including the first moving object area (OB6 (t + 1)) and the second moving object area (G7C) as the moving object area.

  According to the above processing, as for a pedestrian, a moving body whose torso part moves constantly but whose leg fluctuates due to an opening leg movement is not divided into one moving object. It can be detected as a moving object with high accuracy.

  Next, based on FIG. 8C, an example of a method for determining the rear end will be described using a case where the moving object candidate is a two-wheeled vehicle as an example.

  The method described in FIG. 7C is used as the method for determining the rear end in this example.

  In the example shown in FIG. 8C, pixels having a common (approximate) moving speed are scanned further diagonally downward from the rear end (xr, yr), and positions (xr + 2, yr + p) where no speed information exists in adjacent pixels are scanned. Explore. Furthermore, if there is an area (movable area) where the velocity information of the three-dimensional object does not exist in the direction opposite to the traveling direction at this position, the position (xr + 2, yr + p) is finally determined as the rear end.

  The control device 10 determines the region G8C including the lower end, the front end, and the rear end of the first moving object region as the second moving object region. Then, the control device 10 determines an area including the first moving object area (OB7 (t + 1)) and the second moving object area (G8C) as the moving object area.

  According to the above processing, one moving object is divided even for a moving object that has a large width along the moving direction and may detect a different velocity vector (optical flow) depending on the part, such as a two-wheeled vehicle. And can be detected with high accuracy as one moving object.

  Furthermore, based on FIG. 9C, an example of a method for determining the rear end will be described using a case where the moving object candidate is an automobile as an example.

  The method described in FIG. 7C is used as the method for determining the rear end in this example.

  In the example shown in FIG. 9C, pixels having a common (approximate) moving speed are scanned obliquely downward from the rear end (xr, yr), and positions (xr + v, yr + u) where no speed information exists in adjacent pixels are scanned. Explore. Furthermore, if there is a region (movable region) where the velocity information of the three-dimensional object does not exist in the direction opposite to the traveling direction at this position, the position (xr + v, yr + u) is finally determined as the rear end.

  The control device 10 determines the region G9C including the lower end, the front end, and the rear end of the first moving object region as the second moving object region. Then, the control device 10 determines an area including the first moving object area (OB8 (t + 1)) and the second moving object area (G9C) as the moving object area.

  According to the above processing, one moving object is divided even for a moving object that has a large width along the moving direction and may detect a different velocity vector (optical flow) depending on the part, such as an automobile. And can be detected with high accuracy as one moving object.

  Subsequently, a control procedure of the moving object detection system 100 of the present embodiment will be described based on the flowchart of FIG.

  The process shown in FIG. 10 is executed by the control device 10 that is activated in response to an ignition switch (not shown) being turned on.

  First, in step S <b> 101, the control device 10 acquires an image ahead of the host vehicle that is captured by the camera 10 at a predetermined cycle and stored in the image memory 21.

  Next, in step S <b> 102, the control device 10 performs edge extraction processing on the captured image, extracts the outline of the object present in the captured image as an edge image, and normalizes the edge image.

  In step S103, the control device 10 calculates edge speed information, and creates a moving image (see FIG. 5) in which the calculated movement information is represented by a predetermined gradation.

  Next, in step S104, the control device 10 sequentially scans the calculated moving image from the lower part of the image to the upper part of the image, compares the speed information, and is continuously common in the vertical direction (y direction) of the image. A region in which approximate speed information exists is extracted as a three-dimensional object. Further, adjacent three-dimensional objects having speed information that is common or approximate are extracted as the same three-dimensional object.

  In subsequent step S105, the control device 10 compares the position information and speed information of the three-dimensional object detected before in time, and the three-dimensional object in which the speed difference and the horizontal position difference are small and the lower end position is moved vertically downward. As a three-dimensional object that may approach the host vehicle, the region is extracted as a moving object candidate region.

  Furthermore, in step S106, the control device 10 determines whether all moving object candidate regions have been extracted. If all moving object candidate regions have been extracted, the process proceeds to step S107. If it is determined that all the moving object candidate areas have not been extracted, the process returns to step S104, and extraction of the three-dimensional object is continued.

  Subsequently, in step S107, the control device 10 scans the commonality of the speed information downward or obliquely downward from the lower end position on the traveling direction side of the extracted moving object candidate. The control device 10 detects a pixel as a boundary between the moving object candidate and the road surface area, at the position of the lowest end where there is no speed information that is common or approximated downward or obliquely downward.

  Subsequently, in step S108, the control device 10 determines whether or not the speed information of the three-dimensional object exists on the traveling direction side of the lowest end position. If there is a movable area where there is no speed information of the three-dimensional object on the traveling direction side, it can be determined that there is a movable space on the traveling direction side of the moving object candidate, so there is a possibility that the moving object candidate corresponds to the moving object. If it is high, the moving object candidate area is determined as the first moving object area. On the other hand, if the speed information of the three-dimensional object exists on the moving direction side of the moving object candidate, it is determined that there is no movable space, the moving object candidate is excluded, and the process proceeds to step S116.

  Next, in step S109, the control device 10 searches for a three-dimensional object between the lowest end position and the moving space, and performs a grouping process with the lowest end based on the velocity information of the lower end position of the three-dimensional object. Then, the control device 10 detects the pixel at the lower end on the traveling direction side of the moving object candidate in the grouped area as the tip on the traveling direction side of the moving object candidate.

  Further, in step S110, the control device 10 scans the speed information from the tip position in the direction opposite to the traveling direction, and the same based on the speed difference from the representative value of the moving object candidate area, the object width, and the object height. A grouping process as an object is performed.

  In step S111, the control device 10 determines whether the grouping process as the same object has been performed. If it is determined that the grouping process has been completed, the process proceeds to step S112. If it is determined that the grouping process has not been completed, the process returns to step S110 to continue the grouping process.

  In step S112, the end on the opposite side of the traveling direction after grouping as the same object is detected as the trailing end of the moving object.

  In step S113, the control device 10 determines whether or not the rear end can be determined. If it is determined that it can be confirmed, the process proceeds to step S114. If it cannot be determined, the process proceeds to step S116.

  In step S <b> 114, the control device 10 scans the commonness of the speed information downward from the detected rear end position, and detects pixels in which the common or approximate speed information does not exist in the adjacent pixels in the downward direction. If there is an area (movable area) for the speed information of the three-dimensional object at the position opposite to the traveling direction of the moving object, the position is finally determined as the trailing end of the moving object.

  Finally, in step S115, the control device 10 determines an area including the lower end, the front end, and the rear end of the first moving object area as the second moving object area. Furthermore, the control device 10 determines an area including the first moving object area and the second moving object area as the moving object area.

  After this, the flow moves to step S116. In step S116, the control device 10 determines whether or not the ignition switch of the host vehicle is turned off. If it is determined that the ignition switch is not turned off, the control device 10 returns to step S1 and repeats the process. On the other hand, if it is determined that the ignition switch of the host vehicle has been turned off, the process is terminated.

  The detection information of the moving object is sent to the alarm device 300, the travel support device 400, and the output device 500. The alarm device 300 outputs alarm information based on the detection information of the moving object to the driver via the output device 500. The driving support device 400 executes driving support control based on the detection information of the moving object.

  The moving object detection system 100 of the present embodiment is configured and operates as described above, and thus provides the following effects.

  According to the moving object detection system 100 of the present embodiment, it is possible to determine a region of a moving object with different moving speeds of each part with high accuracy.

  That is, the moving object detection system 100 according to the present embodiment detects the lower end of the moving object candidate area including at least a part of the moving object candidates from the image, and detects the presence or absence of speed information further downward from the lower end. , The pixel located at the boundary between the road surface area including the lowermost edge when the speed information is not detected in the downward adjacent pixel and the movable area where the speed information on the moving direction of the moving object candidate is not detected is the moving object. A pixel that is detected as a tip of the candidate in the direction of travel and when speed information of a predetermined range value is not detected when speed information is scanned from this tip to the opposite side of the direction of travel is defined as the rear end of the moving object candidate in the direction of travel The area including the area to which the lower end, the front end, and the rear end belong and the moving object candidate area is determined as the moving object area. It is possible to determine the area of a moving object including the parts having different speeds with high accuracy.

  When the optical flow is calculated when the moving object has different speeds for each part such as a pedestrian, the value of the optical flow differs depending on the part of the pedestrian. When a pedestrian walks, even if the body part moves at a constant speed, the leg part varies in a range where the speed is small due to the leg opening operation. As described above, when the detected speed difference is small and the moving objects are grouped based on the speed difference, the existence area of the entire pedestrian may not be correctly grouped. Since other tip positions and rear end positions that are moving objects but move at different speeds can be detected, it is possible to determine the areas of moving objects with different moving speeds of the respective parts with high accuracy.

  In addition, for two-wheeled vehicles and automobiles (four-wheeled vehicles) having relatively long moving objects along the traveling direction, different velocity vectors (optical flows) are detected in the respective parts where the positions of the moving objects in the length direction are different. There is a case. As described above, when the object width (length along the moving direction) of the moving object is large, areas of different velocity vectors (optical flow) are detected even though they are the same object, and are divided as areas of different objects. There is a case. The moving object detection system 100 according to the present embodiment appropriately detects the lower end, the front end, and the rear end, and determines the area including the area to which these belong and the moving object candidate area as the moving object area. It is possible to determine with high accuracy the area of the moving object in which areas of different velocity vectors (optical flow) are detected.

  Further, the moving object detection system 100 according to the present embodiment extracts an area including a feature point whose speed information is less than a predetermined value set in advance as a moving object candidate area including at least a part of the moving object candidate. Thus, it is possible to extract a moving object candidate that may approach the host vehicle from the three-dimensional object. Some moving objects that may approach the host vehicle have the same moving direction as the background, but in this embodiment, the determination is not based on the difference in the moving direction of the background. Separately, a three-dimensional object (moving object) approaching the host vehicle can be reliably detected.

  Further, the moving object detection system 100 according to the present embodiment scans for the presence or absence of speed information in the traveling direction of the moving object candidate from the boundary with the road surface area, and when there is a movable area where the speed information is not detected, the moving object Since the candidate area is determined as the first moving object area, if the position where the speed information is switched from the area with the speed information is detected, the traveling direction of the moving object can be accurately determined without using the grouping method based on the speed information difference. The end (tip) of the side object can be detected.

  Furthermore, the moving object detection system 100 according to the present embodiment determines a region including the lower end, the front end, and the rear end of the first moving object region as the second moving object region, and the first moving object region and the second moving object region. Is determined as a moving object area, so moving objects with different moving speeds in each part, for example, the body part moves constantly, but the legs move like a pedestrian whose speed changes due to the leg opening action. An object can also be detected as a single moving object with high accuracy without dividing the single moving object.

  Further, the moving object detection system 100 according to the present embodiment scans the moving object from the lowermost end of the moving object for the presence or absence of speed information, and determines that there is a moving space if the speed information of the three-dimensional object does not exist. The moving object candidate is determined as the first moving object candidate. Thus, by detecting the moving space, it is possible to reliably detect a moving object that is likely to approach the host vehicle. On the other hand, if the moving space is not detected, it can be determined that the moving object cannot move and the possibility of approaching the host vehicle becomes low.

  According to the present embodiment, it is possible to detect a moving object that may approach the host vehicle based on the presence of the movable space. In other words, by detecting the presence of a movable area in which no speed information exists in the area on the traveling direction side from the tip of the moving object candidate, the process of grouping the same objects by comparing the speed information is performed. Since the tip can be detected, the detection accuracy of the tip of the moving object can be improved.

  In addition, according to the moving object system 100 of the embodiment of the present invention, the predetermined range used when grouping with the tip pixel is the estimated speed of the stationary object included in the image, and the determined first moving object Since it is obtained based on the absolute value of the difference from the representative value of the velocity information of the pixel to which the region belongs, it is possible to identify the moving object from the background and accurately detect the rear end of the moving object.

  In addition, the moving object detection system 100 according to the present embodiment moves the region when the moving direction of the feature point located at the lower end of the region including the feature point continuously detected in time series is the downward direction of the image. Since the object candidate area is extracted, an area approaching the host vehicle can be extracted as a moving object candidate area that may approach the host vehicle.

  In addition, the moving object detection system 100 according to the present embodiment has a speed information value whose speed information value is less than a predetermined value set in a further downward direction from the lower end of the moving object candidate in the moving direction in the moving object candidate area. Scan the presence / absence and detect the velocity information of the pixel corresponding to the lower end and the velocity information below the predetermined value as the boundary with the road image when the lowermost pixel is not detected in the downward direction. The tip of the lower part of the different object (the part in contact with the road surface) can be detected.

  In addition, the moving object detection system 100 according to the present embodiment includes a three-dimensional object region in a region between the boundary between the road surface region detected by the road surface boundary detection unit and the movable region detected by the first moving object region determination unit. When the lowermost pixel of the three-dimensional object region is positioned in a predetermined positional relationship with the lowermost pixel and the difference between the velocity information of the lowermost pixel is less than a predetermined value, the road surface region The boundary of the object and the three-dimensional object region are grouped, and the lower pixel on the moving direction side of the moving object candidate in the grouped region is detected as the tip on the moving direction side of the moving object candidate. It is possible to detect the tip of the lower part (the part in contact with the road surface) of an object with a different.

  In particular, when the moving object is a pedestrian, the speed information is different between the trunk and the leg. Since the speed information of the legs has diagonal continuity, the commonality of the speed information is scanned downward, and the positions where there is no speed information that is common or approximate to the downward direction are detected. In addition, it is possible to accurately detect the tip position of the lower end of a pedestrian having a different speed for each part.

  Further, the moving object detection system 100 of the present embodiment scans the detected tip for the presence or absence of speed information in a direction opposite to the traveling direction of the moving object candidate, and sets pixels of speed information in a predetermined range of preset values. Objects that are grouped with the leading edge pixels, and the lower end pixel on the opposite side of the moving direction of the moving object candidate in the grouped area is detected as the trailing end of the moving object candidate. The tip of the lower part (the part in contact with the road surface) can be detected.

  In addition, the moving object detection system 100 according to the present embodiment groups the pixels located below the predetermined height in the image upper direction with respect to the detected leading end pixel with the leading end pixel, and moves the moving object in the grouped region. Since the pixel at the lower end on the side opposite to the direction of travel of the candidate is detected as the rear end on the side opposite to the direction of travel of the moving object candidate, the lower part of the object whose moving speed is different for each of the left and right end parts The tip of (part) can be detected accurately.

  In addition, the moving object detection system 100 according to the present embodiment groups pixels that are located within a predetermined distance in the direction opposite to the moving direction of the moving object candidate from the detected front end pixel and grouped with the front end pixel. The lower end pixel on the opposite side of the moving object candidate in the area is detected as the rear end of the moving object candidate in the opposite direction to the moving direction of the moving object candidate. can do. That is, by setting the predetermined distance based on the width of the moving object to be detected, the rear end of the target to be detected can be detected appropriately. For example, the rear end of the pedestrian can be detected appropriately by setting a predetermined distance according to the maximum value of the pedestrian (human) stride.

  Further, according to the moving object detection system 100 of the embodiment of the present invention, after detecting the rear end, the speed information is scanned from the rear end in the direction opposite to the traveling direction of the moving object candidate. When there is a movable area that is not detected, the rear end detected first is determined, so that the rear end can be accurately detected.

  In other words, the object edge on the opposite side of the traveling direction has many parts that overlap with the background, making it difficult to distinguish it from the background, but since the object edge on one side can be accurately detected, grouping to determine the same object is performed. Even if the threshold value is not strict, grouping errors with the background can be reduced by limiting the object width. If the vehicle and the moving object continue to move, if the traveling direction is continued, the overlap with the background is eliminated, and a moving space appears on the opposite side of the traveling direction. In that case, it is possible to detect accurately by applying the object end detection method on the traveling direction side.

  Further, according to the moving object detection system 100 of the embodiment of the present invention, the count value of the pixel corresponding to the extracted position of the feature portion is counted up at a predetermined period, and the feature portion is based on the slope of the count value. Since the pixel velocity information corresponding to is calculated, the pixel movement information can be acquired by a simple method. Further, by such image processing, it is possible to easily derive a three-dimensional feature point in which pixels having common speed information are continuous in the vertical direction and easily extract a three-dimensional object region from the captured image.

  The embodiment described above is described for facilitating the understanding of the present invention, and is not described for limiting the present invention. Therefore, each element disclosed in the above embodiment is intended to include all design changes and equivalents belonging to the technical scope of the present invention.

  That is, in this specification, the moving object detection system 100 will be described as an example of the moving object detection device according to the present invention, but the present invention is not limited to this.

  In this specification, as an example of the moving object detection device according to the present invention, the moving object detection system 100 including the control device 10 including the CPU 11, the ROM 12, and the RAM 13 will be described as an example. However, the present invention is not limited thereto. It is not a thing.

  Further, in this specification, the imaging means, speed information calculation means, moving object candidate area extraction means, road surface boundary detection means, first moving object area determination means, moving object tip detection means, As one aspect of the moving object detection device having the moving object rear end detection unit, the second moving object region determination unit, and the moving object determination unit, a camera 20, a speed information calculation function, and a moving object candidate region extraction function , A road surface boundary detection function, a first moving object area determination function, a moving object tip detection function, a moving object rear end function, a second moving object area determination function, and a moving object determination function However, the present invention is not limited to this.

DESCRIPTION OF SYMBOLS 100 ... Moving object detection system 10 ... Control apparatus 11 ... CPU
12 ... ROM
13 ... RAM
DESCRIPTION OF SYMBOLS 20 ... Camera 21 ... Image memory 200 ... Vehicle controller 300 ... Alarm apparatus 400 ... Driving assistance apparatus 500 ... Output device

Claims (11)

An imaging means mounted on a vehicle for capturing an image in front of the vehicle;
Speed information calculating means for extracting feature points from the image captured by the imaging means and calculating speed information including the moving speed and / or moving direction of the feature points;
Moving object candidate area extracting means for extracting an area including a feature point whose speed information is less than a preset predetermined value as a moving object candidate area including at least a part of the moving object candidate;
In the moving object candidate area, the speed information in the lower direction is scanned from the lower end on the moving direction side of the moving object candidate, and the lowermost pixel is moved when the speed information is not detected in the adjacent pixels in the lower direction of the image. Road surface boundary detecting means for detecting the boundary between the object candidate and the road surface area;
The presence or absence of the speed information is scanned in the traveling direction of the moving object candidate from the boundary with the road surface area, and when there is a movable area where the speed information is not detected, the moving object candidate area is defined as the first moving object area. First moving object region determination means for determining
Moving object tip detection means for detecting a pixel located at a boundary between the road surface region and the movable region as a tip on the traveling direction side of the moving object candidate;
The pixel when the speed information is scanned from the tip in the direction opposite to the moving direction of the moving object candidate, and the speed information within a predetermined threshold value obtained in advance is not detected in the adjacent pixel in the direction opposite to the moving direction Moving object rear end detection means for detecting the moving object candidate as a rear end on the moving direction side,
Second moving object region determination means for determining a region including the lower end, the front end, and the rear end of the first moving object region as a second moving object region;
A moving object detection apparatus comprising: a moving object determination unit that determines an area including the first moving object area and the second moving object area as a moving object area. The moving object detection device according to claim 1,
The moving object candidate region extraction means further moves the region when the moving direction of the feature point located at the lower end of the region including the feature point detected continuously in time series is the downward direction of the image. A moving object detection apparatus that extracts an object candidate area. In the moving object detection device according to claim 1 or 2,
The road surface boundary detection means scans further downward velocity information from the lower end of the moving object candidate in the moving direction in the moving object candidate region, and the velocity of the pixel corresponding to the lower end of the moving object candidate on the moving direction side A moving object detection apparatus that detects a lowermost pixel as a boundary between a moving object candidate and a road surface area when speed information whose difference from information is less than a predetermined value is not detected in a downward direction. In the moving object detection device according to any one of claims 1 to 3,
The moving object front end detection means is a vertical portion of the image located in a region between a boundary between the road surface area detected by the road surface boundary detection means and a movable area detected by the first moving object area determination means. A three-dimensional object region in which velocity information common to the direction is continuous is extracted, and the lowermost pixel of the three-dimensional object region is positioned in a predetermined positional relationship with the lowermost pixel detected by the road surface boundary detection unit; When the difference between the velocity information of the lowermost pixel of the three-dimensional object area is less than a predetermined value, the lowermost pixel and the three-dimensional object area are grouped, and the moving object candidate in the grouped area is grouped. A moving object detection apparatus that detects a pixel at a lower end on a traveling direction side as a tip on a traveling direction side of the moving object candidate. In the moving object detection device according to any one of claims 1 to 4,
The moving object rear end detection means scans speed information in a direction opposite to the moving direction of the moving object candidate from the detected front end, and sets pixels of speed information within a predetermined range value with respect to the speed information of the front end. A moving object detection apparatus that groups with a front end pixel and detects a lower end pixel of the grouped region on a side opposite to a moving direction of the moving object candidate as a rear end of the moving object candidate. The moving object detection device according to claim 5,
The moving object rear end detection unit further groups a pixel located below a predetermined height in the image upper direction with respect to the detected front end pixel with the front end pixel. The moving object detection device according to claim 6,
The moving object rear end detection means further groups a pixel located within a predetermined distance in a direction opposite to the traveling direction of the moving object candidate from the detected front end pixel with the front end pixel. . In the moving object detection device according to any one of claims 1 to 7,
The moving object trailing edge detecting means uses a predetermined range used when grouping with the leading pixel, an estimated speed of a stationary object included in the image, and a speed of a pixel belonging to the determined first moving object area. A moving object detection device obtained based on an absolute value of a difference from a representative value of information. In the moving object detection device according to any one of claims 1 to 8,
The moving object rear end detection means detects the rear end, and then scans for the presence or absence of the speed information in a direction opposite to the moving direction of the moving object candidate from the rear end, and the movable area where the speed information is not detected. A moving object detection device that determines the rear end detected first. In the moving object detection device according to any one of claims 1 to 9,
The speed information calculation means counts up the count value of the pixel corresponding to the extracted feature position at a predetermined period, and calculates the speed information of the pixel corresponding to the feature section based on the slope of the count value. A moving object detection device to calculate. An area where speed information including the moving speed and / or moving direction of the feature point extracted from the captured image in front of the vehicle is less than a predetermined value is set as a moving object candidate area including at least a part of the moving object candidates. Extract and
In the moving object candidate area, the speed information in the lower direction is scanned from the lower end on the moving direction side of the moving object candidate, and the lowermost pixel is detected when the speed information is not detected in the adjacent pixels in the lower direction. Detect as the boundary between the candidate and the road surface area,
Detecting a pixel located at the boundary between the movable area where the speed information is not detected in the traveling direction of the moving object candidate from the boundary with the road surface area and the road surface area as a tip on the traveling direction side of the moving object candidate;
The speed information is scanned in the direction opposite to the moving direction of the moving object candidate from the front end, and the pixel in the case where speed information of a predetermined predetermined value is not detected is determined as the rear end of the moving object candidate in the moving direction. Detect as
A moving object detection method for determining that the first moving object area and an area including a lower end, a front end, and a rear end of the first moving object area are areas corresponding to the same object.

Images