JP2010021633A - Mobile object detector and detection method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a mobile object detector having high mobile object detection accuracy. <P>SOLUTION: The mobile object detector includes: a reduced image generation unit 22 for generating the reduced images of different reduction rates from captured images photographed by a camera 10; a feature point extraction unit 23 for respectively extracting the feature point of the generated reduced image and the feature point of the captured image of the camera 10; a movement information calculation unit 24 for calculating movement information including the moving speed and/or moving direction of each feature point; a composition unit 25 for generating composite information from the movement information of the feature point of the reduced image and the movement information of the feature point of the captured image, that are calculated; and a mobile object detection unit 30 for detecting a mobile object on the basis of the generated composite information. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a moving body detection apparatus and a moving body detection method for detecting a moving body based on a captured image.

Obtain the optical flow of the captured image and the assumed background optical flow obtained from the estimated motion of the moving body and the estimated spatial model, and compare the optical flow of the entire captured image with the optical flow of the background An apparatus for detecting a moving body around a vehicle is known (see Patent Document 1).

As a method for calculating an optical flow, a method is known in which a position where the degree of similarity with a preset template is maximized is identified in a captured image.

Japanese Patent Laid-Open No. 2004-56763

  However, in the technique of detecting a moving body using the optical flow of the background obtained from the estimated movement of the moving body and the estimated spatial model as in the conventional technique, the accuracy of the optical flow is included in the template. The detection accuracy of moving objects observed in a minute time is reduced in areas where there are large image changes such as changes in shape and shade intensity, such as around the host vehicle where the device is mounted. There was a problem that there was a possibility.

  The problem to be solved by the present invention is to improve the detection accuracy of a moving body in a portion where an image change is large.

  The present invention detects a moving object based on composite information generated from movement information of feature points of a captured image and movement information of feature points of reduced images having different reduction ratios generated from the captured image. Solve the above problems.

  According to the present invention, it is possible to expand a region for calculating feature point movement information by generating reduced images with different reduction ratios from captured images. Therefore, it is possible to detect a moving object with high accuracy.

<First Embodiment>
FIG. 1 is a diagram illustrating an example of a block configuration of an in-vehicle device 1000 including the moving body detection device 100. The moving body detection device 100 according to the present embodiment is a device that is mounted on a vehicle and detects a moving body such as a pedestrian, another vehicle, or a bicycle based on a captured image in front of the vehicle (forward along the traveling direction). .

  Moreover, as shown in FIG. 1, the vehicle-mounted apparatus 1000 of this embodiment acquires the mobile body detection information from the mobile body detection apparatus 100, this mobile body detection apparatus 100, and the information based on the acquired mobile body detection information. And an external device such as a vehicle controller 200 that outputs to the user or the vehicle side. As shown in FIG. 1, as an external device, in addition to a vehicle controller 200 that controls driving of a vehicle, an alarm device 300 that notifies the presence of a pedestrian, a travel support device 400 that provides travel support according to the presence of a pedestrian, and a display And / or one or more output devices 500 having speakers and presenting information on the presence of pedestrians. The arithmetic processing functions of the moving object detection device 100, the vehicle controller 200, the alarm device 300, the travel support device 400, and the output device 500 of the present embodiment are configured by combining operation circuits such as a CPU, MPU, DSP, and FPGA. The processing means executes. For example, the image processing unit 20 and the mobile unit detection unit 30 of the mobile unit detection apparatus 100 according to the present embodiment include programs that operate by being configured by a microcomputer and a memory, and ASICs and FPGAs that incorporate the respective processes as circuits. To perform computations necessary for moving object detection.

  In addition, the structure which performs the arithmetic processing of these apparatuses is connected by vehicle-mounted LAN, such as CAN (Controller Area Network).

  Hereinafter, the moving body detection apparatus 100 of the present embodiment will be described in detail based on the drawings. As illustrated in FIG. 1, the moving body detection device 100 includes a camera 10, an image memory 11 that stores captured data of the camera 10, an image processing unit 20 that processes captured images, and a moving body detection unit 30. Is provided.

  Next, each configuration included in the image processing apparatus 100 will be described.

  First, the camera 10 is a camera having an image sensor such as a CCD (Charge-Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor), for example, and continuously around the vehicle at a predetermined period (vehicle front, vehicle rear, vehicle side). And the like are output to the image memory 11. The image memory 11 stores the image data in an accessible state. As the image memory 11, an HDD, CD, MD, DVD, optical disk, or other recording medium is used.

  FIG. 2 shows an installation example of the camera 10. As shown in FIG. 2, the camera 10 is installed in front of the upper part of the vehicle interior. As shown in the figure, in the present embodiment, one camera 10 is installed in the vehicle. That is, in this embodiment, the surroundings of the vehicle are imaged by the monocular camera 10.

  Further, the camera 10 has its optical axis LS oriented in the front front direction (Z direction) of the vehicle, the horizontal axis X (not shown) of the imaging surface is parallel to the road surface, and the vertical axis Y (not shown) of the imaging surface. (Omitted) is set to be approximately perpendicular to the road surface.

  An example of a captured image (image in front of the host vehicle) captured by the camera 10 is shown in FIG. The captured image captured by the camera 10 is represented by an xy coordinate system defined by an x axis extending from left to right and an y axis extending from top to bottom with the upper left vertex of the image as the origin. The example of the captured image illustrated in FIG. 3 includes a left and right white line, a boundary line of a road such as an outer wall, and a pedestrian moving from the left side to the right side of the road.

  Next, the image processing unit 20 will be described. The image processing unit 20 includes a captured image acquisition unit 21, a reduced image generation unit 22, a feature point extraction unit 23, a movement information calculation unit 24, and a synthesis unit 25, and the captured image captured by the camera 10. Process.

  First, the captured image acquisition unit 21 will be described. The captured image acquisition unit 21 acquires captured images captured by the camera 10 from the image memory 11 (or the camera 10) at a predetermined cycle. Then, the captured image acquisition unit 21 sends the captured image to the reduced image generation unit 22 and the feature point extraction unit 23. Of course, the reduced image generation unit 22, the feature point extraction unit 23, the movement information calculation unit 24, and the synthesis unit 25 directly access the image memory 11 without using the captured image acquisition unit 21 and acquire a captured image or a reduced image. Can do.

  Next, the reduced image generation unit 22 will be described. The reduced image generation unit 22 generates reduced images having different reduction ratios from the captured images captured by the camera 10 and acquired by the captured image acquisition unit 21. As the reduced image, an image in which a plurality of unit pixels constituting the captured image are combined is generated. That is, the unit pixel in the reduced image is expanded more than the unit pixel in the original captured image. In other words, the reduced image generation unit 22 enlarges the size of the unit pixel at a different rate, or combines the unit pixels by a different number to obtain a plurality of reduced images having different reduction rates.

  An example of a reduced image generation method will be described with reference to FIG. As shown in FIG. 4, the captured image that is the source of the reduced image is shown in FIG. 4 (a), the 1/2 reduced image is shown in FIG. 4 (b), and the 1/4 reduced image is the highest. It is shown in FIG. The captured image in FIG. 4A corresponds to the captured image shown in FIG.

  The captured image, the 1/2 reduced image, and the 1/4 reduced image have different ranges (also referred to as speed detection ranges) in which movement information, which will be described later, can be calculated, and the speed detection range can be expanded more than the captured images.

The 1/2 reduced image shown in FIG. 4B is a reduced image generated by combining two pixels arranged in the horizontal direction in the captured image into one pixel. That is, the 1/2 reduced image is an image obtained by reducing the captured image by 1/2 in the X-axis direction. Also, the 1/4 reduced image shown in FIG. 4C is a reduced image generated by combining four pixels arranged in the horizontal direction in the captured image into one pixel. That is, the 1/4 reduced image is an image obtained by reducing the captured image to 1/4 in the X-axis direction. The 1/4 reduced image may be generated by combining two pixels arranged in the horizontal direction of the 1/2 reduced image into one pixel. The reduction ratio of the reduced image is not particularly limited, and can be generated at an arbitrary reduction ratio. The reduced image generation unit 22 temporarily stores the generated reduced image in the image memory 11. Further, the correspondence relationship between the pixel in the captured image and the position (coordinate value) of the pixel in the reduced image on the image is stored in association with the information of the captured image and / or each reduced image.

As described above, in the present embodiment, by generating reduced images with different reduction ratios from captured images, it is possible to expand a region (speed calculation range) where feature point movement information can be calculated. Accurate movement information can be calculated even when the change is large.

  Next, the feature point extraction unit 23 will be described. The feature point extraction unit 23 extracts a feature point in the captured image acquired by the captured image acquisition unit 21 and a feature point in the reduced image generated by the reduced image generation unit 22, respectively. In the present processing example, the feature points extracted by the feature point extraction unit 23 are pixels or pixel groups corresponding to the edges of the object included in the captured image.

  Specifically, when extracting feature points from a captured image, the feature point extraction unit 23 acquires an image captured by the camera 10 from the image memory 11 and binarizes the acquired captured image using a predetermined threshold. By doing so, the edge of the object existing in the image is extracted. FIG. 5A shows an example of the extracted vertical edge. Next, thinning processing is performed on each extracted edge to narrow the edge width, and the center of the edge is accurately set (see FIG. 5B). Further, the edge is extended in the horizontal direction so that the edge width of the thinned edge becomes a constant width, for example, a width corresponding to three pixels (see FIG. 5C). By this operation, the extracted edges are normalized, and an edge image having a uniform width for each edge is obtained.

  Further, the feature point extraction unit 23 extracts feature points of the reduced image by the same method. The extracted feature point information of the captured image and the feature point information of the reduced image are sent to the movement information calculation unit 24.

  The movement information calculation unit 24 calculates movement information including the movement direction and / or movement speed of the feature points extracted by the feature point extraction unit 23 and the pixels corresponding to the periphery of the feature points. The obtained moving speed and moving direction of the characteristic part are stored in association with the imaging timing identifier or the frame identifier. This pixel movement information includes “pixel movement speed” and “pixel movement direction” along with pixel identification information. When a plurality of feature parts exist in one image data, the movement information is calculated for all the feature parts.

  Hereinafter, an example of the movement information calculation method will be described. The movement information calculation unit 24 counts up the count value of the pixel at the position where the edge corresponding to the extension of the object is detected based on the information of the image of the object imaged by the camera 10, and sets the inclination of the count value. Based on this, the moving speed and moving direction of the edge are calculated.

  In addition, the movement information calculation unit 24 updates the counter value of the pixel counter of the pixel corresponding to the edge included in each image data by using a predetermined method for the captured image data having different imaging timings. Here, the pixel counter is a counter set for each pixel. When the pixel corresponds to the edge, the counter value of the pixel counter is incremented by +1. When the pixel does not correspond to the edge, the counter value of the pixel counter is increased. This is a counter that is set to 0 (initialized). This counter value update process is performed for each frame that is repeatedly imaged by the camera 10 in a predetermined cycle. When this operation is performed, the counter value of the corresponding pixel counter increases for a pixel having a long time corresponding to an edge, while the counter value of the corresponding pixel counter decreases for a pixel having a short time corresponding to an edge.

  This change in the counter value of the pixel counter represents the moving direction and moving amount of the edge. Therefore, based on this counter value, the edge moving direction and moving speed on the captured image or reduced image are calculated. Since the coordinate system of the image represents the azimuth, the moving direction and moving speed of the edge and the feature corresponding to the edge can be obtained.

  Furthermore, a movement information calculation method performed by the movement information calculation unit 24 will be described with reference to FIG. FIG. 5 is a diagram for explaining the movement information calculation process, that is, a process for obtaining an edge image in which the extracted edge is normalized and calculating a movement direction and a movement speed from the edge counter value (residence time). It is a figure for demonstrating concretely.

  First, the feature point extraction unit 23 performs binarization processing on the edge image. The binarization process is a process in which a pixel at a position where an edge is detected is set to 1 and a pixel at a position where no edge is detected is set to 0. FIG. 5A shows an example of the binarized image of the extracted vertical edge.

  Next, as shown in FIG. 5B, thinning processing is performed on the generated binary image. The thinning process is a process of reducing the edge width of the detected edge until a predetermined pixel width is reached. That is, the edge width is narrowed by performing thinning processing on each extracted edge. In this example, as shown in FIG. 5B, the edge width of the edge is thinned until the predetermined pixel width becomes one pixel. In this way, the edge is thinned to a predetermined pixel width, thereby setting the center position as the center of the edge. Note that, in this example, an example in which one pixel is thinned is shown, but the number of pixels to be thinned is not particularly limited.

  Next, an expansion process is performed to expand the edge width of the thinned edge. The expansion process is performed so that the edge width is constant from the center position set by thinning toward the edge movement direction, and the edge width is also changed from the center position to the direction opposite to the edge movement direction. It is a process of expanding. In this example, the edge is expanded in the horizontal direction so that the edge width of the thinned edge becomes a width corresponding to three pixels. By this process, the extracted edges are normalized, and an edge image having a uniform width is obtained.

  Specifically, as shown in FIG. 5C, the pixel is expanded by one pixel from the edge center position x0 to the edge movement direction (the positive direction of the x axis), and opposite to the edge movement direction from the edge center position x0. The edge width is expanded to 3 pixels by expanding one pixel in the direction (negative direction of the x axis).

  By performing the thinning process and the expansion process in this way, the edge width of the extracted edge image is standardized by standardizing to a predetermined width in the edge moving direction.

  Next, a count-up process performed by the movement information calculation unit 24 to calculate movement information will be described. The count-up process mentioned here is a process for counting up the value of the memory address corresponding to the position of the pixel where the edge is detected and initializing the value of the memory address corresponding to the position of the pixel where the edge is not detected. It is.

  Hereinafter, the edge count-up process by the movement information calculation unit 24 will be described with reference to FIGS. For convenience of explanation, here, a case where the edge moves in the positive direction of the x-axis will be described as an example. Even when the edge moves in the negative x-axis direction, the y-axis direction, or two-dimensionally, the basic processing method is common.

  As shown in FIG. 5C, the edge has the center position of the edge at a position x0 in a certain frame. Then, the pixel is expanded from the center position to the position x0 + 1 of one pixel in the edge moving direction, and similarly expanded from the center position to the position x0-1 of one pixel in the direction opposite to the edge moving direction.

  The count value of the memory address corresponding to the position where such an edge is detected, “x0-1”, “x0”, “x0 + 1” is incremented by +1. On the other hand, the count value of the memory address corresponding to the position where the edge is not detected is reset.

  For example, in FIG. 5D, edges are detected at positions “x0-1”, “x0”, and “x0 + 1” at time t. Therefore, the count value of the memory address corresponding to each position is incremented by one. As a result, the count value at the position “x0 + 1” is “1”, the count value at the position “x0” is “3”, and the count value at the position “x0-1” is “5”.

  Next, as shown in FIG. 5E, since the edge does not move even at time t + 1, the edge is detected at each of the positions “x0-1”, “x0”, and “x0 + 1”. . Therefore, the count values at the positions “x0-1”, “x0”, and “x0 + 1” are further incremented by one. As a result, the count value at position “x0 + 1” is 2, the count value at position “x0” is 4, and the count value at position “x0-1” is 6.

  Further, as shown in FIG. 5 (f), at time t + 2, the edge is shifted by one pixel in the positive direction of the x-axis, and the edge is detected at positions “x0”, “x0 + 1”, and “x0 + 2”. Therefore, the count value of the memory address corresponding to the positions “x0”, “x0 + 1”, and “x0 + 2” where the edge is detected is counted up. On the other hand, the count value at the position “x0-1” where no edge is detected is reset to “zero”. As a result, the count value at the position “x0 + 2” is 1, the count value at the position “x0 + 1” is 3, and the count value at the position “x0” is 5, as shown in FIG. Further, the count value at the position “x0-1” where no edge is detected is reset to “0”.

  As described above, the movement information calculation unit 24 counts up the count value of the memory address corresponding to the position where the edge is detected, and resets the count value of the memory address corresponding to the position where the edge is not detected.

  In the description based on FIG. 5, as the position for detecting the count value, the center position “x0” of the edge, the position “x0 + 1” of one pixel in the moving direction of the edge from the center position, and the edge position from the center position The count value is detected at three positions of the position “x0-1” of one pixel in the direction opposite to the moving direction. However, if the slope of the count value described later is obtained, the arrangement and number of points for detecting the count value are not limited. . That is, as long as the count value can be detected at two or more locations in the edge moving direction, the count value may be detected in any number.

  If the frame rate is set sufficiently higher than the speed at which the edge moves, the edge is detected a plurality of times at the same position between consecutive frames. For example, in the example of FIG. 5, the edge is detected twice at the position x0 in the continuous frame at time t and frame at time t + 1. Therefore, when the count value of the memory address corresponding to the position where the edge is detected is counted up, the count value correlates with the time (number of frames, dwell time) at which the edge is detected at that position. In particular, the smallest count value h of the edge count values represents how many frames the edge has been in the same position after moving.

  Next, an edge moving speed, moving direction, and position calculation method in this embodiment will be described. In this embodiment, the inclination of the count value is calculated, and the moving speed, moving direction, and position of the edge are calculated based on this inclination.

  For example, in the case of FIG. 5E, the count values at the positions “x0-1”, “x0”, and “x0 + 1” are “6”, “4”, and “2”, respectively. When the count value “2” of “x0 + 1” is subtracted from the count value “6” of the position “x0-1”, the slope H of the count value can be calculated as H = (6-2) / 2 = 2.

  This means H = {(time from the edge moving to the position x0-1 to the present) − (time after the edge moves to the position x0 + 1)} / (2 pixels). That is, by calculating the inclination H, the time (number of frames) required for the edge to pass through one pixel at the position x0 is calculated.

  Therefore, the slope H of the count value corresponds to how many frames it takes for the edge to move by one pixel, and the edge moving speed 1 / H is calculated based on the slope H of the count value. In FIG. 5E, since 2 frames are required to move 1 pixel, the edge moving speed is calculated as 1/2 (pixel / frame). Similarly, in FIG. 5F, H = (5-1) / 2 = 2, so that the edge moving speed is ½ (pixel / frame).

  Next, a method for determining the edge moving direction based on the magnitude of the count value will be described. Since the edge moves to a position where there is no edge and the count value at the position where the edge is newly detected is 1, the count value at each position is the smallest value. Therefore, the count value in the direction in which the edge moves is small, and the count value in the direction opposite to the direction in which the edge moves is large. By using this tendency, the moving direction of the edge can be determined.

  Furthermore, if the frame rate is set sufficiently higher than the speed at which the edge moves, it can be assumed that the detection target object moves at a constant speed. In addition, the smallest count value h among the count values at the current position represents the time when the edge is detected at the position, that is, how many frames the edge has moved to stay at the same position. ing. Thus, the edge position can be obtained by “edge position = x0 + h / H” where x0 is the center position of the edge. For example, in FIG. 5F, the edge speed is ½ (pixel / frame), and the edge is detected at the same position for one frame continuously at time t + 2, so the position of the edge at time t + 2 Can be calculated as moving from the position x0 by “1 (frame) × {1/2 (pixel / frame)} = 0.5 pixel”.

  From the above, it is possible to count up the count value of the memory address corresponding to the position where the edge is detected, and to calculate the moving speed and moving direction of the edge based on the slope of the counted up count value.

  Similarly, the same processing is performed on the reduced image, and movement information including the movement speed and movement direction of the feature point (edge) is calculated. In this example, in addition to the movement information of the captured image, the movement information of the reduced image obtained by reducing this by 1/2 in the horizontal direction and the movement information of the reduced image obtained by reducing the captured image by 1/4 in the horizontal direction are calculated.

However, since the moving speed calculated for the feature point of the reduced image is the moving speed in each reduced image, when performing the process of detecting the moving body based on the movement information, the moving speed in the reduced image is determined with respect to the captured image. It needs to be converted into movement information.

In this way, by calculating movement information for a reduced image having a predetermined reduction ratio that is one pixel larger than that of the captured image, the movement of an edge that moves one pixel or more per frame in the captured image is determined by one frame. Since the movement speed can be calculated by converting the movement to less than one pixel per hit, the speed detection range can be expanded.

Specifically, for a reduced image obtained by enlarging the size of one pixel in a captured image, the edge movement information is obtained by counting up according to edge detection and calculating the time required for the edge to pass one pixel. In order to calculate, the movement speed can be calculated by converting the movement of the edge that moves by one pixel or more per frame into the movement of one pixel or less per frame.

For this reason, the moving speed can be calculated in a relatively wide range without setting the frame rate high. In other words, since accurate movement information can be calculated over a wide range, the movement speed of the background to be compared can be calculated accurately, and as a result, the comparison of the movement information between the background and the object can be performed with high accuracy. It can be carried out.

  The movement speed calculation unit 24 sends the calculated movement information to the synthesis unit 25.

  Next, the synthesis unit 25 will be described. The synthesis unit 25 generates synthesis information from the movement information of the feature points of the reduced image calculated by the movement information calculation unit 24 and the movement information of the feature points of the captured image.

  Although the mode of the composite information is not particularly limited, the information on the feature point movement information of the captured image or the feature point movement information of any reduced image is associated with the pixel of each feature point that is the target of the moving object detection. It can be. In the synthesis information of the present embodiment, information on feature points (edges) in a predetermined visual field image (for example, coordinate information) is associated with movement information of feature points of a captured image or feature point movement information of any reduced image. Information. In this example, a captured image that has not undergone reduction processing is used as the predetermined field-of-view image, and the position information of the feature point (edge) and the movement information of the feature point of the captured image or the characteristics of any reduced image are included in the captured image. The image information on which the point movement information is superimposed is used as composite information.

  That is, for each pixel corresponding to the feature point detected in the captured image, the synthesis unit 25 selects either the captured image movement information or the reduced image movement information, and uses the selected movement information as the feature point pixel. Compositing information is generated in association with (pixel ID).

  As described above, the synthesis unit 25 and each reduced image (1/2 reduced image, half-reduced image, different size of the unit region (size of the imaging region corresponding to the pixel) at the time of calculating the feature point movement information). The movement information obtained from the (1/4 reduced image) is combined to generate one piece of combined information suitable for detection of the moving object.

  Furthermore, since the movement information calculated in the captured image and each reduced image has different unit areas (pixels) in which the movement information is detected, the synthesis unit 25 aligns the image sizes of the unit areas. . In this example, in order to generate the composite information based on the captured image captured by the camera 10, it is aligned with the pixel size of the captured image. In other words, a 1/2 reduced image reduced by 1/2 in the horizontal direction is enlarged 2 times along the horizontal direction, and a 1/4 reduced image reduced by 1/4 in the horizontal direction is enlarged 4 times along the horizontal direction. To do.

  Furthermore, the composition unit 25 aligns the units of the moving speed. Since the unit of moving speed calculated in the present embodiment is pixels / frame, by multiplying the calculated moving speed (pixel / frame) by the frame rate of the camera 10, these moving speeds are expressed in units of pixels / second. Convert to system.

  The unit of moving speed of the reduced image obtained by reducing the captured image by 1/2 in the horizontal direction is also converted into a unit system of pixels / second. Here, in the 1/2 reduced image, the detected unit area (pixel) is twice (2 pixels) the unit area (pixel) of the captured image, so the amount of movement per frame in the captured image is 2 Doubled. Therefore, for the 1/2 reduced image, the moving speed (pixel / second) corresponding to the captured image can be obtained by doubling the calculated moving speed and multiplying by the frame rate. Similarly, for a ¼ reduced image, the moving speed (pixel / second) corresponding to imaging can be obtained by multiplying the calculated moving speed by four and multiplying by the frame rate.

Incidentally, since the reduced image is generated from the captured image, the position (coordinate value) of the pixel and / or pixel group in the reduced image and the pixel and / or pixel group in the captured image can be associated with each other. . Information about the correspondence relationship may be attached to the reduced image, or may be stored in the image memory 11 so that it can be referred to.

As described above, after aligning the magnitude and unit of the moving speed, composite information is generated from the moving information of the feature points of the reduced image and the moving information of the feature points of the captured image.

The synthesizing unit 25 selects appropriate information from the movement information of the reduced image and the movement information of the captured image, and configures the synthesis information based on the selected movement information. Hereinafter, an example of a synthesis information generation method for selecting what movement information from which reduced image and how to generate synthesis information will be described.

The synthesizing unit 25 of the present embodiment generates synthesis information from the movement information of each feature point detected in the captured image and the movement information of the feature point of the reduced image corresponding to the feature point of the captured image. Specifically, the synthesis unit 25 determines whether or not a feature point has been extracted from the captured image, and when the feature point is extracted, using the feature point movement information in the reduced image corresponding to the feature point. Generate synthesis information. In other words, the movement information combining process when generating the combined information is limited to pixels in which feature points (edges) are detected in the captured image.

This is because the influence of noise generated by the image processing is eliminated, and the synthesis processing is performed only for the feature points derived from the presence of the moving object. That is, movement information may be calculated in a portion where a feature point (edge) does not exist in the captured image (a portion where no object actually exists) due to the influence of image processing such as image reduction or image enlargement, This is because such movement information may cause a false detection, and the influence is eliminated.

As described above, in this embodiment, in order to eliminate the influence of the movement information generated in the image processing instead of the movement information due to the presence of the moving object, the pixel to be subjected to the synthesis processing by the synthesis unit 25 is captured image. The pixel is included in the pixel and the feature point (edge) is detected. In this way, noise generated in image processing such as generation of a reduced image and enlargement of a reduced image is performed only by combining pixels with other reduced images and movement information only for pixels in which feature points are detected in the captured image. By eliminating the influence of the above, it is possible to prevent a decrease in detection accuracy.

The synthesizing unit 25 specifies the pixel or the pixel group (the pixel or the pixel group corresponding to the feature point extracted in the captured image) to be synthesized in the captured image, and then the movement information of the pixel where the edge exists The movement information (movement speed) is synthesized.

The synthesis unit 25 generates synthesis information based on the movement information of the feature points of the reduced image having the highest reduction ratio among the movement speeds of the feature points of the reduced images respectively corresponding to the feature points extracted in the captured image. . That is, when a feature point is extracted from the captured image (that is, when the presence of an object is detected), the corresponding movement information is selected in the reduced image with the highest reduction ratio as the moving speed of the feature point. By doing so, it is possible to generate composite information based on movement information having a relatively high S / N. In other words, an apparently slow speed is detected for a reduced image with a high reduction ratio, but when the moving speed is calculated based on the accumulated amount of edges, the slower speed has a larger amount of accumulated edges and the S / N is higher. . For this reason, by selecting movement information with a high reduction ratio, it is possible to generate composite information based on movement information with a high S / N for feature points where an object exists.

When the moving speed of the feature point extracted from the captured image is not calculated, the synthesizing unit 25 uses the feature of the reduced image having the highest reduction ratio among the moving speeds of the feature points of the reduced images corresponding to the feature points. Synthesis information is generated based on the point movement information. That is, when feature points are extracted from the captured image and the presence of the object can be detected, but the moving speed of the object is not detected, the combined information is generated using the movement information of the reduced image.

In the synthesis information generation process, the synthesis unit 25 first specifies a pixel from which a feature point has been extracted in the captured image as a synthesis target. Of the movement information of each reduced image corresponding to the feature point (edge) to be synthesized, the movement information of the reduced image having the highest reduction ratio is selected. For example, when a 1/2 reduced image and a 1/4 reduced image are generated, a feature to be synthesized from movement information in a 1/4 reduced image with the highest degree of reduction (reduction rate). Select movement information corresponding to a point. Since the reduced image is generated based on the captured image, the correspondence between the pixel of the captured image and the pixel of the reduced image can be determined.

The synthesizing unit 25 acquires the movement information of the selected reduced image from the movement information calculating unit 24 or the image memory 11. Then, composite information is generated using the selected movement information. In other words, the movement information calculated based on the captured image is replaced with the movement information of the selected reduced image. This process is performed on feature points (edges) extracted from the captured image to generate composite information.

Thus, by replacing the movement information of the reduced image with a high reduction ratio with the movement information of the captured image, it is possible to obtain highly accurate composite information.

In the present embodiment, the moving speed of the feature point (edge) is calculated based on the accumulated amount of the feature point. Therefore, when a plurality of pieces of movement information are calculated for each pixel corresponding to the feature point, the reduction rate In the case of a high-reduced image, the movement information of feature points that are apparently slower in speed is detected. In other words, since the movement information is calculated based on the accumulation amount of the feature amount (edge), the S / N is higher because the accumulation amount of the movement information is larger in the feature amount having the slower movement. For this reason, when movement information is calculated for reduced images with different reduction ratios, it is possible to generate high-precision composite information by selecting high speed information with a high S / N and high reduction ratio to generate composite information. Obtainable.

Further, when the moving speed of the feature point extracted from the captured image is not calculated, the presence of the object is detected by generating composite information based on the moving information of the feature point of the reduced image having the highest reduction ratio. Even when the moving speed is not known, it is possible to obtain highly accurate composite information.

  Further, the combining unit 25 has a feature point in which a moving direction and a direction difference of the feature point of the captured image are less than a predetermined value out of the moving speed of the feature point of each reduced image corresponding to the feature point detected in the captured image. In addition, the composite information is generated based on the movement information of the feature points of the reduced image having the highest reduction ratio.

With regard to the moving direction, when the moving speed of the feature point is in a low speed range or a very low speed range, the moving direction varies, so that a reduced image with a low reduction rate (the image with the lowest reduction rate is a captured image). The accuracy is higher. For this reason, in this embodiment, when the direction difference between the moving direction of the feature point of the captured image and the moving direction of the feature point of the corresponding reduced image is less than a predetermined value, the reduced image with the highest reduction ratio is displayed. Select movement information of feature points, and generate synthesis information. As a result, the feature point of the reduced image with the highest reduction ratio is obtained only when the moving direction of the pixel or pixel group corresponding to one feature point is aligned to some extent, that is, when the moving speed is fast enough to converge the moving direction. This movement information can be selected and used to generate highly accurate composite information.

An example of a method for generating synthesis information will be described with reference to FIG. FIG. 6 shows pixels whose movement information is calculated from the captured image, 1/2 reduced image, and 1/4 reduced image when the frame rate is 30 fps. In FIG. 6, pixels with light ink are pixels from which feature points (edges) have been extracted in the captured image, and pixels whose movement speed is not calculated (not detected) in the captured image, and / or These pixels satisfy the condition that the difference between the moving direction and the moving direction of the reduced image is less than a predetermined value. In this example, composite information is generated that preferentially uses the movement information of the pixels to which light ink is applied. Note that when the same condition is satisfied, composite information using the movement information of a reduced image with a high reduction ratio in preference is generated.

As described above, when the movement information is calculated in the reduced image for the feature points extracted in the captured image, the synthesis unit 25 does not calculate the movement speed in the captured image and / or the movement direction is substantially the same. When the movement direction difference is within a predetermined value, the movement information of the image having the highest (or relatively high) reduction ratio is selected as the movement information of the pixel corresponding to the captured image, and the selected movement information is used. To generate synthesis information.

One aspect of the synthesis information generated in this way will be described with reference to FIG.

The synthesizing unit 25 according to the present embodiment synthesizes the movement information selected for each pixel in which an edge exists on the captured image, and generates a synthesized movement image that is expressed by classifying the synthesized movement information into predetermined class values. To do. An example of this combined movement information is shown in FIG. The synthesized moving image shown in FIG. 7 classifies movement information associated with feature points (edges) existing on a predetermined visual field image (captured image) of the observer into predetermined class values, and features of the movement information This is image information expressed in the form in which is represented.

In this composite moving image, the pixel value of the moving speed component is represented by a circle, the pixel of the edge where the moving information is detected is indicated by a circle, and the pixel having the higher moving speed is indicated by a larger circle, thereby indicating the movement information of the pixel. Express.

In addition, the moving direction of the pixel is represented by a solid black mark that represents a pixel that moves to the right, that is, the right direction, and the moving direction is represented by a white mark that represents a pixel that moves to the left, that is, the left direction. Express.

That is, in the situation indicated by the composite moving image in FIG. 7, the moving information (moving speed) from the outer wall on the right side of the traveling path of the host vehicle and the white line toward the right side of the image is shown, and from the outer wall on the left side of the traveling path to the left side of the image. The moving information (moving speed) is shown. Further, movement information (moving speed) from the pedestrian moving from the left side to the right side of the traveling path toward the right side of the image is shown. As described above, the synthesized moving image can express movement information including the moving speed and the moving direction on the image. The synthesis unit 25 provides the generated synthesis information or the synthesized moving image to the moving object detection unit 30.

  Next, the moving body detection unit 30 will be described.

The moving body detection unit 30 includes a grouping unit 31, a coordinate conversion unit 32, and a determination unit 33, and detects a moving body based on the combination information (for example, a combined moving image) generated by the combining unit 25. Since the moving object is a moving three-dimensional object, the moving object detection unit 30 according to the present embodiment is a three-dimensional object in which a feature point in the synthesized moving image or a pixel group corresponding to the feature point can be a candidate for the moving object. It is determined whether or not the three-dimensional object is a moving object.

  Hereinafter, each structure with which the mobile body detection part 30 is provided is demonstrated.

  First, the grouping unit 31 will be described. The grouping unit 31 groups the pixel groups adjacent to each other in the substantially vertical direction in the synthesized moving image, in which the movement speed calculated by the movement information calculation unit 24 is within a predetermined range. For this reason, the grouping unit 31 sets an area in the composite moving image. For example, as shown in FIG. 8, a plurality of strip-shaped areas are set on the combined moving image, and the combined moving image is divided by each of the plurality of strip-shaped areas. Then, the grouping unit 31 detects the moving object, and the moving speed calculated by the movement information calculating unit 24 is within a predetermined range for each divided region, and is adjacent in the vertical direction in the image (vertical direction). Group of pixels). The grouping unit 31 sequentially scans each strip region from the lower part to the upper part of the image, and there is a pixel having a speed in a certain attention area, and the speed with a pixel having a speed existing in the upper part of the attention area. The difference is compared, and when the speed difference is equal to or less than the threshold value T1, it is determined that they are the same object, and those pixels are grouped.

Specifically, the synthesized moving image (see FIG. 7) is searched in the vertical direction (y-axis direction). When a pixel A having movement information is found, a pixel B adjacent to the pixel A is searched. Same if the pixel B has movement information, the difference in the speed direction between the pixel A and the pixel B is within the threshold value Re, and the difference in the speed magnitude between the pixel A and the pixel B is within the threshold value Tv Judged to be continuous vertically at the moving speed.

Next, the presence / absence of movement information is similarly determined for the pixel C adjacent to the pixel B, and whether the difference in speed direction between the pixel A and the pixel C is within the threshold value Re or whether the difference in speed magnitude is within the threshold value Tv. Judging. Thereafter, the process is repeated until one of the conditions is not satisfied. Here, the feature that “pixels having common movement information in the vertical direction are continuous” is a feature on a stereoscopic image. That is, each area is scanned from the lower part to the upper part of the image, and when there is a pixel having speed in the area, the speed difference with the pixel having speed adjacent to the pixel is compared. When the speed difference is equal to or less than the threshold value T1, it can be estimated that the objects move at the same speed with respect to the vehicle.

  Then, the upper end position and the lower end position of the grouped area are detected. In the synthesized moving image shown in FIG. 8, vertical positions BL1 to BL15 on the image having the lower end point in the strip width and vertical positions TL1 to TL15 on the image having the upper end point in the strip width are detected.

  Next, the coordinate conversion unit 32 will be described. The coordinate conversion unit 32 converts the pixels included in the composite moving image into converted coordinates in an overhead view viewed from a predetermined viewpoint. The conversion coordinates are conversion coordinates that are given a predetermined area (area) and are divided into divided areas.

  First, the coordinate conversion unit 32 detects the upper end position (the pixel existing at the highest position) and the lower end position (the pixel existing at the lowest position) of the pixels grouped by the grouping unit 31. By this process, lower end positions BL1 to BL15 and upper end positions TL1 to TL15 shown in FIG. 8 are detected.

  As shown in FIG. 9, the coordinate conversion unit 32 uses the x coordinate of the upper end point and the lower end point as the median of the strip width in the area where the upper end position and the lower end position of each strip area where the object exists is present. Based on the coordinate values TP <b> 1 to TP <b> 15 and BP <b> 1 to BP <b> 15 and the camera parameters of the camera 101, the upper end point and the lower end point are projected onto the real space.

  That is, the coordinate conversion unit 32 has the lower end points BP1 to BP15 (not shown) and the upper end points TP1 to TP15 (not shown) whose coordinates are the center positions of the lower ends BL1 to BL15 and the upper ends TL1 to TL15 extracted on the xy plane. Are converted into three-dimensional position coordinates (three-dimensional position coordinates in real space) as points on a ZX plane (hereinafter referred to as a prescribed ZX plane) having a prescribed area.

  Here, the coordinates of each point are (x, y), the height from the road surface of the camera is Ch (m), the depression angle of the camera is Tr (rad), the vertical size of the image is Ih, and the horizontal size of the image is Iw. If the angular resolution per pixel in the height direction is PYr (rad) and the angular resolution per pixel in the horizontal direction is PXr (rad), the points TP1 to TP15 and BP1 to BP15 on the xy plane are Is converted into coordinates (Z, X) on the ZX plane.

(Formula 1) Z = (Ch) / (TAN (Tr + (y−Ih / 2) × PYr))
(Formula 2) X = x × TAN ((Z-Iw / 2) × PXr)
The converted points of the upper end points TP1 to TP15 are referred to as upper end coordinate conversion points RT1 to RT15, and the converted points of the lower end points BP1 to BP15 are referred to as lower end coordinate conversion points RB1 to RB15. A part of the correspondence is shown in FIG.

  Then, it is determined which of the two-dimensional regions in which the projected positions RT1 to RT15, RB1 to RB15 define the X-axis range and the Z-axis range as shown in FIG. In this embodiment, the X-axis direction is less than −5.25 m, −5.25 to less than −3.5 m, −3.5 m to less than −1.75 m, −1.75 m to less than 0 m, 0 m to 1 8 regions of less than .75m, 1.75m or more and less than 3.5m, 3.5m or more and less than 5.25m, and 5.25m or more are set in the Z-axis direction, 20m to 25m, 25m to 30m, 30m to 35m , 35 m to 45 m and 45 m to 55 m are set, and a total of 8 × 5 two-dimensional areas are set. Then, the information obtained by the coordinate conversion unit 32 is sent to the determination unit 33.

  Next, the determination unit 33 will be described. The determination unit 33 determines whether or not the object detected from the captured image is a moving body based on information obtained from composite information such as a composite moving image.

By the way, since the moving body is a moving object itself, it is not a two-dimensional plane object but a three-dimensional object. The determination unit 33 of the present embodiment first determines whether or not the detected object (pixel group) is a three-dimensional object that can be a candidate for a moving object, and further, the three-dimensional object (candidate) is a moving object. It is determined whether or not there is.

  Based on the coordinates of the pixel present at the highest position (upper end) and the coordinates of the pixel present at the lowest position (lower end), the determination unit 33 determines whether the target object included in the captured image is a planar object. It is determined whether it is a three-dimensional object.

  The determination unit 33 according to the present embodiment includes a case where the coordinates of the pixel located at the upper end and the coordinate of the pixel located at the lower end of the pixel group grouped by the grouping unit 31 belong to a common divided region. Determines that the area corresponding to the grouped pixel group is a planar object. On the other hand, in the pixel group grouped by the grouping unit 31, the determination unit 33 has the coordinates of the pixel located at the upper end in the transformation coordinates outside the region of the transformation coordinate, and the coordinates of the pixels located at the lower end are any. If it belongs to such a divided region, the region corresponding to the grouped pixel group is determined as the target three-dimensional object. For example, as in OB1 shown in FIG. 9, the conversion coordinate of the pixel TP1 located at the upper end is located outside the area of the ZX conversion coordinate, and the conversion coordinate of the pixel BP1 located at the lower end is located within the ZX conversion coordinate. The determination unit 33 determines that the region to which OB1 belongs is a region including a three-dimensional object.

  Further, the determination unit 33 determines that the upper end coordinate conversion points RT1 to RT15 and the lower end coordinate conversion points RB1 to RB15 are in any divided region of the ZX plane in the coordinates (Z, X) of the ZX plane divided at a predetermined interval. Based on whether it belongs, a three-dimensional object is extracted (see FIG. 9). Here, when the captured image of the camera 10 moves up and down as the vehicle moves up and down, the area to which the coordinate conversion point belongs may change. In order to avoid this influence, the predetermined interval of the divided areas on the ZX plane is set in meter order. In this embodiment, the x-axis direction is −5.25 m> x, −5.25 ≦ x <−3.5 m, −3.5 m ≦ x <−1.75 m, −1.75 m ≦ x <0 m, 0 m ≦ x <1.75 m, 1.75 m ≦ x <3.5 m, 3.5 m ≦ x <5.25 m, 5.25 m ≦ x, and the z-axis direction is 0 ≦ z <10 m, 10 m ≦ z <20 m, 20 m ≦ z <30 m, 30 m ≦ z <40 m, 40 m ≦ z <50 m, and divided into 5 regions 11 to 15, regions 21 to 25, regions 31 to 35, regions 41 to 41 Region 45, region 51 to region 55, region 61 to region 65, region 71 to region 75, and region 81 to region 85 are set. Further, the region 11 to the region 15 are the region 10, the region 21 to the region 25 are the region 20, the region 31 to the region 35 are the region 30, the region 41 to the region 45 are the region 40, and the region 51 to the region 55 are the region 50. The region 61 to the region 65 are the region 60, the region 71 to the region 75 are the region 70, and the region 81 to the region 85 are the region 80.

  The determination unit 33 determines that the upper end point and the lower end point of the same group are the same when the upper end coordinate conversion point and the lower end coordinate conversion point that are grouped at the same position in the xy coordinates belong to the same divided area on the ZX plane. It is determined that the object is on the road surface, that is, a planar object on the road surface (determined not to be a three-dimensional object). Also, if the upper end coordinate conversion point corresponding to the upper end point is located outside the specified ZX coordinate and the lower end coordinate conversion point of the lower end point of the same group as the upper end point is located within the specified ZX coordinate, the lower end point Since only is on the road surface, it is determined to be a three-dimensional object.

  For example, as shown in FIG. 9, coordinate transformation points RT1, RT3, RT5, RT6, RT8, RT9 and RT12 to RT15 of the upper end points TP1, TP3, TP5, TP6, TP8, TP9 and TP12 to TP15 are shown in FIG. Since the projection is performed outside the region of the prescribed ZX conversion coordinates, the group including the upper end points TP1, TP3, TP5, TP6, TP8, TP9 and TP12 to TP15 is determined to be a three-dimensional object. The edges including the upper end points TP1, TP3, TP5, TP6, TP8, TP9 and TP12 to TP15 are determined as the three-dimensional objects OB1 to OB10 on the xy plane.

  The coordinate transformation points RT2, RT4, RT7, RT10, RT11 of the upper end points TP2, TP4, TP7, TP10, TP11 are coordinate transformation points RB2, RB4, RB7, RB10 of the lower end points BP2, BP4, BP7, BP10, BP11. , RB11, the group including the upper end points TP2, TP4, TP7, TP10, TP11 is determined to be a plane object located on the road surface.

  Here, the movement information calculation unit 24 adds +1 to the counter value of the counter in the area where the coordinate conversion point of the lower end point is located among the counters set in each area in order to detect the road boundary described later. The position information distribution of points is calculated.

  Note that in the determination unit 33, a pixel group corresponding to a planar object included in the extracted target area is connected to a feature point corresponding to the target area including the extracted target three-dimensional object along a substantially vertical direction of the image. In this case, the region including the planar object and the region including the three-dimensional object are determined as the region including one solid object.

  Furthermore, the determination unit 33 determines whether the three-dimensional object is a moving object, that is, whether it is a moving body.

  The determination unit 33 of the present embodiment extracts a stationary object such as a road surface included in the captured image as a background, compares the movement information of the background with the movement information of the pixel group to be determined, and the result of the comparison Based on the above, it is determined whether or not the region corresponding to the pixel group includes a moving body.

  Specifically, the determination unit 33 is a reference having a predetermined relationship with the installation surface of the three-dimensional object included in the image, based on the coordinate value of the pixel located in the lowest position in the substantially vertical direction of the image among the group of pixels. The movement direction of the pixel group that is extracted by the boundary and grouped by the grouping unit 31 is substantially opposite to the movement direction of the reference boundary in which the pixel group and the y coordinate value in the vertical direction in the captured image are common. In the case of a directional relationship, the grouped pixel group is determined to be a moving object.

  The moving body determination method will be described. First, the determination part 33 extracts the reference | standard boundary corresponding to the installation surface of a solid object as one aspect | mode of a background. In the present embodiment, a road boundary of a road on which a vehicle on which the mobile body detection device 100 is mounted is extracted as a reference boundary.

  For this reason, as shown in FIG. 10, based on the obtained position distribution of the coordinate conversion points of the lower end point, an area where there is a high possibility that the reference boundary exists is extracted. That is, in the obtained position distribution of the coordinate conversion points of the lower end point, if there are counter values in a plurality of z-axis regions in the same x-axis region, the road boundary (boundary of the road on which the vehicle travels) ahead of the host vehicle Can be estimated to exist along the vehicle in a straight line, the region having the counter value is extracted as a region including the road boundary as one aspect of the reference boundary. For example, in FIG. 10, in the region 30 where the x-axis is the same, there are counter values in a plurality of z-axis regions (regions where the lower end points RB2, RB4, and RB7 are located). Extract. The same applies to the region 20, the region 50, and the region 80.

  Then, as shown in FIG. 10, an area where there is a high possibility that a road boundary exists is extracted from the obtained position distribution of the coordinate conversion points of the lower end point. In other words, in the obtained position distribution of the coordinate transformation point of the lower end point, if there are counter values in a plurality of z-axis areas in the same x-axis area, it is estimated that the road boundary exists linearly along the vehicle in front of the host vehicle. Therefore, an area with a counter value is extracted as a road boundary area. For example, in FIG. 10, since there are counter values in a plurality of z-axis regions (regions where the lower end points RB2, RB4, and RB7 are located) in the region 30 where the x-axis is the same, these regions are used as runway boundary regions. Extract. The same applies to the region 20, the region 50, and the region 80.

  Then, it is determined whether or not a straight line indicating a road boundary exists in the extracted area (see FIG. 10). That is, based on the coordinate conversion point of the lower end point existing in the extracted region, regression analysis is performed in the xy coordinate system, and the slope of the straight line passing through the lower end point is calculated. For example, in the area 30 of FIG. 10, the regression analysis in the xy coordinate system is performed from the coordinate positions of the lower end points BP2, BP4, and BP7 in the xy coordinates corresponding to the coordinate conversion points RB2, RB4, and RB7 of the lower end points existing in the area 20. Then, the slope a3 of the straight line passing through the lower end points BP2, BP4, BP7 is calculated. Similarly, straight line inclinations a2, a5, and a8 corresponding to the projected points of the lower end points located in the regions 20, 50, and 80 are calculated. Then, if the calculated slope of the straight line is within a predetermined range that is defined in advance as a range that can be determined as a road boundary, it is determined that there is a straight line indicating the extracted road boundary. That is, the points (for example, PL1 to PL5 in FIG. 10) whose coordinates are the left end coordinates of the x-axis area where the coordinate conversion point of the lower end point and the representative coordinates (for example, center coordinates) of each Z-axis area are converted into the xy coordinate system. Then, the straight line slope Tn0a1 calculated by regression analysis, the right end coordinate of the x-axis area where the coordinate conversion point of the lower end point is located, and a point having the coordinates (for example, the center coordinates) of each Z-axis area as coordinates (for example, FIG. The slope of the straight line “an” calculated based on the lower end point is within the range of the slope defined by the slope of the straight line Tn0a2 calculated by performing regression analysis by converting 10 PR1 to PR5) into the xy coordinate system. In this case, it is determined that a straight line Ln indicating a road boundary exists in the extracted area.

  For example, as shown in FIGS. 10 and 11, in the region 80, the point PL <b> 1 with coordinates (x, z) = (− 3.5, 5), coordinates (x, z) = (− 3.5, 15). Point PL2, coordinate (x, z) = (-3.5,25) point PL3, coordinate (x, z) = (-3.5,35) point PL4, coordinate (x, z) = ( -3.5, 45) point PL5 is converted into xy coordinates and regression analysis is performed to calculate the slope T30a1 of the straight line connecting the points and the point of coordinates (x, z) = (-1.75, 5) PR1, point PR2 with coordinates (x, z) = (− 1.75, 15), point PR3 with coordinates (x, z) = (− 1.75, 25), coordinates (x, z) = (− 1 .75, 35), the point PR4 and the coordinate PR (x, z) = (− 1.75, 45) are converted into xy coordinates, and regression analysis is performed to obtain the slope T30a2 of the straight line connecting the calculated points. The slope a3 of the straight line connecting the lower end points BP1, BP4, and BP5 on the xy plane corresponding to the projected points RB1, RB4, and RB5 of the lower end point is within the range defined by T30a1 and T30a2. It is determined that the straight line connecting BP2, BP4, and BP7 is a straight line L3 indicating the road boundary.

  Similarly, in the region 20, it is determined that the straight line connecting BP1 and BP6 corresponding to the projection points RB1 and RB6 of the lower end point is a straight line L2 indicating the road boundary, and in the region 50, the projection point RB10 of the lower end point The straight line connecting the lower end points BP10 and BP11 on the xy plane corresponding to RB11 is determined to be the straight line L5 indicating the road boundary, and in the region 80, the lower end point on the xy plane corresponding to the projected points RB12 to RB15 of the lower end points A straight line connecting BP12 to BP15 is determined to be a straight line L8 indicating a road boundary.

Next, as illustrated in FIG. 12, the determination unit 33 detects, as a background, a straight line formed of a three-dimensional object that continuously exists far from the vicinity of the host vehicle, out of straight lines indicating a road boundary. In general, the background in the captured image of the front is characterized in that stationary three-dimensional objects are linearly lined away from the vicinity of the host vehicle, the speed of the vicinity is high, and the speed decreases as the distance increases. Therefore, the road boundary showing such characteristics is extracted as the background. Here, when there are a plurality of road boundaries composed of three-dimensional objects, a road boundary located farther from the host vehicle is selected. In this embodiment, the road boundary lines L2 and L8, which are the intersections of the outer wall and the road surface, are detected as the background. However, among the three-dimensional objects constituting L2, OB2 and OB3 that do not match the background characteristics are excluded.

And as shown in FIG. 13, the graph which shows the relationship between the vertical direction position (y coordinate value) of the image between the solid objects detected as a background and a moving speed is calculated | required. That is, in the three-dimensional object existing in the image, if the object is a stationary object such as a background, the speed decreases as the object moves away from the own vehicle as it goes to the upper part of the image. That is, if you set the speed on the vertical axis (plus the speed to move to the left, minus the speed to move to the right), and set the y-coordinate of the image on the horizontal axis, the relationship between the speed and y-coordinate will be a straight line . Moreover, the speed of a stationary solid object existing at the same y coordinate (same distance) decreases as it goes from both ends of the image to the center.

An object that exists in an area (shaded area) delimited by the runway boundary L2 that is the background and the center of the image, and that moves laterally in the direction of moving away from the background in the same manner as the background, has the same y coordinate ( A speed slower than the background is detected in the same direction as the speed of the background existing at the same distance.

  Further, as shown in FIG. 13, if a speed opposite to the speed of the background (for example, the reference boundary) is detected, it can be determined that there is a moving body that moves laterally away from the background.

  In the present embodiment, the moving object is detected using L2 as the background (reference boundary). For the three-dimensional objects OB2 and OB3 corresponding to pedestrians existing in a region separated by the background and the center of the image, a movement direction substantially opposite to the movement direction of the background is detected. Therefore, these are determined as a moving body. Based on the determination result, the moving body detection device 100 detects the moving body.

That is, the determination unit 33 according to the present embodiment installs the three-dimensional object included in the combined moving image based on the coordinate value of the pixel located at the lower end in the substantially vertical direction of the combined moving image in the grouped pixel group. A reference boundary having a predetermined relationship with the surface is extracted, and a moving direction of the pixel group grouped by the grouping unit 31 and a reference boundary having a common y-coordinate value in the vertical direction in the pixel group and the combined moving image are shared. If the moving direction is in a substantially opposite relationship defined in advance, it is determined that the grouped pixel group is a moving body.

In addition, a process in the case where there is a three-dimensional object in which the moving direction is the same as the background speed and a speed slower than the background exists will be described. In such a case, the first detection cannot determine whether the object is a stationary object or a moving object, but it is possible to determine whether the object is a moving object by observing movement information (movement speed) over time. That is, if it is a stationary three-dimensional object like the background, it will approach the vehicle with time and will move downward and outward on the image, so the speed will be faster than when observed for the first time.

On the other hand, if it is a moving body that moves away from the background little by little, it approaches the host vehicle, but moves downward and inward (center of the image) on the image, so the moving speed becomes slow. Therefore, the determination unit 33 compares the moving speed over time, determines that an object whose speed has increased with the passage of time from the first detection is a stationary object, and the speed has decreased with the passage of time since the first detection. Is determined as a moving object.

That is, the determination unit 33 according to the present embodiment defines in advance the movement direction of the grouped pixel group and the movement direction of the reference boundary where the y-coordinate value in the vertical direction of the pixel group and the captured image is common. Even when the relationship is approximately the opposite direction, if the moving speed of the grouped pixel group decreases with time, the region corresponding to the grouped pixel group is determined to include a moving object. To do.

  Next, the moving object detection processing procedure of the moving object detection device 100 of this embodiment will be described with reference to the flowchart shown in FIG. FIG. 14 is a flowchart showing processing of the moving object detection device 100 of the present embodiment.

  This process is executed by a program that is activated when an ignition switch (not shown) is turned on.

  First, an image in front of the host vehicle captured by the camera 10 is recorded in the image memory 11.

  In step S101, the image memory 11 outputs the captured image in front of the host vehicle recorded on itself to the image processing unit 20 at a predetermined cycle. After this, the flow moves to step S102.

  In step S <b> 102, the captured image acquisition unit 21 acquires a captured image from the image memory 11 (or the camera 10) and sends it to the reduced image generation unit 22. The reduced image generation unit 22 combines a plurality of pixels with an arbitrary number to generate reduced images with different reduction ratios. Arbitrary plural patterns can be set as the number of combined pixels. The reduced image generated by the reduced image generation unit 22 is stored in the image memory 11. After this, the flow moves to step S103.

  In step S <b> 103, the feature point extraction unit 23 extracts feature points from the captured image stored in the image memory 11 and reduced images having different reduction ratios. Specifically, the feature point extraction unit 23 performs edge extraction processing on the captured image and the reduced image, and generates an edge image in which the contour of an object existing in the captured image is extracted. After this, the flow moves to step S104.

  In step S104, the movement information calculation unit 24 calculates movement information including the movement speed and movement direction of edges (feature points) included in the captured image and the reduced image. After this, the flow moves to step S105.

  Subsequently, in step S105, the composition unit 25 converts the reduced image into the same size as the captured image, converts the speed unit of the moving speed obtained in each reduced image into pixels / second, and performs normalization. After this, the flow moves to step S106.

  In step S106, the synthesis unit 25 synthesizes the movement information for each feature point in the captured image based on the movement information of the captured image and the movement information of the reduced image at each reduction ratio, and generates combined movement information. . The synthesizing unit 25 of the present embodiment generates a moving image as synthesized moving information obtained by converting each synthesized moving information into a predetermined gradation value. After this, the flow moves to step S107.

  In step S107, the grouping unit 31 sets a strip-like area for detecting a moving object on the calculated composite moving image. After this, the flow moves to step S108.

  In step S108, the grouping unit 23 checks whether there is a pixel having speed in each strip area from the bottom to the top. When there are pixels having speed, the grouping unit 23 performs grouping assuming that these pixels represent the same object. After this, the flow moves to step S109.

  In step S109, for each group in each strip area, the coordinate conversion unit 32 sets the center coordinate of the pixel located at the uppermost (upper end) as the upper end point and the center coordinate of the pixel located at the lowermost (lower end) as the lower end. Set to point. After this, the flow moves to step S110.

  In step S110, the coordinate conversion unit 32 performs coordinate conversion of the detected coordinates of the upper end point and the lower end point into the ZX plane using (Expression 1) and (Expression 2) described above. After this, the flow moves to step S111.

  In step S111, the determination target extraction unit 31 of the determination target extraction unit 31 defines that the position of the coordinate conversion point of the upper end point and the coordinate conversion point of the lower end point of the same group as the upper end point defines the x-axis range and the z-axis range. It is determined which region of the ZX plane corresponds, and when both the coordinate transformation point of the upper end point and the coordinate transformation point of the lower end point of the same group as the upper end point are located in the same divided region of the prescribed ZX plane, When the object including the upper end point and the lower end point is determined to be a plane object, and only the coordinate transformation point of the lower end point of the same group as the upper end point is located on the specified ZX plane, the object including the upper end point and the lower end point is determined. It is determined that the object is a three-dimensional object. Also, the counter value of the counter in the area where the lower end point is located is incremented by +1. After this, the flow proceeds to step S112.

  In step S112, the determination unit 33 determines whether the object including the upper end point and the lower end point is a planar object or a three-dimensional object for all detected upper and lower end points (hereinafter referred to as object attribute determination). ) Is performed. When the object attribute determination is performed for all the detected upper and lower end points, the process proceeds to step S113. On the other hand, if the object attribute determination is not performed for all the detected upper and lower end points, the process returns to step S108.

  Subsequently, in step S113, the determination unit 33 determines, from the position distribution of the coordinate conversion point of the lower end point on the ZX plane, an area where counter values exist in a plurality of z-axis areas in the same x-axis area, Extract as a region that is likely to exist. After this, the flow proceeds to step S114.

  In step S114, regression analysis in the xy coordinate system is performed on the lower end point of the xy plane corresponding to the coordinate conversion point of the lower end point existing in the region extracted by the determination unit 33, and the slope of the straight line connecting the lower end points is obtained. Calculated. After this, the flow proceeds to step S115.

  In step S115, the determination unit 33 determines that the slope of the straight line calculated in step S114 has coordinates with the left end coordinate of the x-axis region and the representative coordinate of each z-axis region in the region where the lower end point is located (FIG. 10). And the straight line connecting the right end coordinates of the x-axis region and the points (PR1 to PR5 in FIG. 10) having the coordinates of the representative coordinates of each z-axis region as the coordinates in the xy coordinate system. Then, it is determined that there is a straight line indicating the road boundary in the extracted area, and the straight line whose inclination is calculated in step S114 is detected as a straight line indicating the road boundary. After this, the flow proceeds to step S116.

  In step S116, the determination unit 33 determines whether or not straight lines indicating all the road boundaries have been detected for the region extracted in step 113. If a straight line indicating the road boundary is detected for all regions, the process proceeds to step S117. On the other hand, when the straight line indicating the road boundary is not detected for all the regions, the process returns to step S113, and the detection of the straight line indicating the road boundary is continued.

  In step S117, the determination unit 33 detects a straight line constituted by a three-dimensional object continuously present in the distance from the vicinity of the vehicle among straight lines corresponding to the road boundary detected in step S115 as a road boundary serving as a background. To do. After this, the flow moves to step S118.

In step S118, the determination unit 33 compares the speed of the background and the speed of the three-dimensional object existing in the area divided by the background and the center of the image. It is determined that there is a moving object. Then, the process proceeds to step S120, and when a speed slower than the background speed of the same y coordinate in the same direction as the background is detected, it is determined that there is a moving object candidate having a high possibility of a moving object. After this, the flow moves to step S119.

In step S119, the determination unit 33 determines that there is a moving object candidate in the region where it is determined that a moving object candidate exists, and compares the previous and current moving speeds. If the current speed is faster than the previous speed, the moving object candidate is determined as a stationary object and excluded from the moving object candidates. On the other hand, the previous and current moving speeds are compared, and if the current speed is slower than the previous speed, the moving object candidate is determined to be a moving object. If the previous and current moving speeds do not change (if the speed difference is less than a predetermined value), the moving object candidates are determined again in the next determination process. After this, the flow moves to step S120.

In step S120, it is determined whether or not a moving object has been detected in all regions divided by the background and the center of the image. If it is determined that moving bodies are detected in all areas, the process proceeds to step S121. If it is determined that moving bodies are not detected in all areas, the process returns to step 118 to continue the moving body detection process. To do.

In step S121, it is determined whether or not the ignition switch of the host vehicle is turned off. If it is determined that the ignition switch is not turned off, the process returns to step S101 and the process is repeated. On the other hand, if it is determined that the ignition switch of the host vehicle has been turned off, the process is terminated.

Since the moving body detection apparatus 100 of this embodiment is comprised and operate | moved as mentioned above, there exist the following effects.

The moving body detection apparatus 100 of the present embodiment generates a reduced image from the captured image, and calculates the moving information in a wider range by detecting the moving body based on the combined information generated from the moving information ( Detection), accurate movement information can be calculated even in a portion of the captured image where the image change is large, and the moving object can be detected with high accuracy.

That is, by calculating movement information for a reduced image having a predetermined reduction ratio that is one pixel larger than the captured image, an edge movement that moves one pixel or more per frame in the captured image is 1 per frame. Since the movement speed can be calculated by converting the movement to less than a pixel, the speed detection range can be expanded, so the movement information can be displayed in a relatively wide range without setting the frame rate high. Can be calculated.

In other words, since accurate movement information can be calculated over a wide range, the movement speed of the background to be compared can be calculated accurately, and as a result, the comparison of the movement information between the background and the object can be performed with high accuracy. It can be carried out.

In particular, when the moving body detection apparatus 100 itself moves, such as when the moving body detection apparatus 100 is mounted on a vehicle, the change in the image in the vicinity of the vehicle, that is, the change in the image in the lower region on the left and right sides of the captured image tends to increase. However, since accurate movement information can be obtained even in a region where the change in the image is large as described above, the moving object can be detected with high accuracy.

As described above, when the moving body detection apparatus 100 is mounted on a vehicle and detects a moving body in front of the vehicle, the lower area on the left and right sides of the captured image such as both sides of the road where the scenery flows fast is also moved. Since the body can be detected with high accuracy, a moving body such as a pedestrian that may appear in the lower region on the left and right sides of the captured image can be detected with high accuracy.

In this way, by generating a reduced image from the captured image and generating composite information from the reduced image and the movement information of the captured image, accuracy can be improved based on movement information more accurate than movement information obtained only from the captured image. High synthesis information can be generated. And since a mobile body can be detected based on this highly accurate synthetic | combination information, a mobile body can be detected with high precision.

That is, movement information can be calculated in a wide detection range by calculating movement information from images with different reduction ratios, that is, images with different movement information detection ranges, and by combining the calculated movement information. Therefore, it is possible to accurately compare a small moving body and a small moving background with a large moving speed, so that the moving body can be detected with high accuracy.

Since the movement information detection area is expanded and the movement of the entire imaging screen can be detected, the movement of a reference object such as a stationary object (background) included in the imaging range can be captured with high accuracy. Since the moving body can be detected based on the comparison between the movement of the object (reference object) and the movement of the object to be detected, the moving body can be detected with high accuracy.

In addition, the moving object detection apparatus 100 according to the present embodiment creates composite information based on the movement information of the feature points extracted from the captured image and the movement information of the feature points of the reduced image corresponding to the feature points. Thus, it is possible to exclude information generated in the enlargement / reduction process performed when generating the reduced image from the target of the synthesis process, and to remove the erroneous detection factor. That is, it is possible to prevent erroneous movement information from being calculated based on information generated by image processing even though the pixel does not have a feature point.

Incidentally, a pixel having no movement information in a captured image includes a pixel having no feature point in the first place and a pixel having a feature point but having no movement information. If a combination process is performed on pixels that do not have feature points in the first place as in the former case and movement information is provided, erroneous detection may occur. For this reason, in the present embodiment, synthesis processing is performed on the pixels in which the feature points are detected in the captured image to generate synthesis information based on highly accurate movement information, and the moving object is highly accurate based on the synthesis information. Can be detected.

Further, when generating the combination information, the combination information based on the movement information with higher accuracy can be generated by using the movement information of the feature points of the reduced image having the highest reduction ratio. This is because when moving information is calculated based on the accumulated amount of feature points (edges), moving information at a slower speed can be detected in a reduced image with a higher reduction ratio, and the accumulated amount of edges is larger. This is because the S / N of the movement information can be improved. In other words, among the movement information calculated at each reduction ratio, the movement information calculated from the reduced image having the highest S / N and the highest reduction ratio is preferentially selected to generate highly accurate composite information. be able to.

Further, when the moving speed of the feature point is not calculated when generating the synthesis information, the synthesis information based on the movement information with higher accuracy is generated by using the movement information of the feature point of the reduced image having the highest reduction ratio. be able to.

  As described above, it has been explained that a reduced image with a high reduction ratio tends to have a higher S / N ratio of movement information. However, when movement information is in a low speed range or a very low speed range, variation in the speed direction is large. The moving direction calculated from the reduced image having the lower reduction ratio is higher in accuracy than the reduced image having the higher reduction ratio. For this reason, if movement information calculated from a reduced image with a high reduction ratio is simply selected, erroneous detection may occur in an extremely low speed region. In order to prevent such erroneous detection, the movement direction in the captured image with a low reduction ratio is considered from the viewpoint that the reduction ratio in the movement direction is low (the captured image has the lowest reduction ratio). By selecting the movement information of a reduced image having a movement direction in substantially the same direction (direction difference is less than a predetermined value) and having a high reduction ratio, it is possible to prevent erroneous detection in a low speed range and to improve accuracy. A composite moving image based on the other world movement information can be generated.

In addition, when calculating the movement information in the present embodiment, the count value of the pixel corresponding to the extracted feature position is counted up at a predetermined period, and the pixel corresponding to the feature section is counted based on the slope of the count value. By calculating the movement information, it is possible to calculate the movement information of the object included in the captured image without specifying the target object.

In addition, by limiting the amount of movement between frames to 1 pixel or less and eliminating the association processing between frames necessary for specifying an object, it is possible to perform calculation processing required for calculating movement information at high speed. . In addition, since sequential processing is performed in units of pixels without performing repeated computation (recursive processing), high-speed computation processing is possible.

In the present embodiment, edge movement information existing in a predetermined visual field image of the observer corresponding to the captured image is classified into a predetermined class value, and a combined movement image in which the characteristics of the movement information are expressed is synthesized. It is generated as information, and a moving object can be detected with high accuracy based on this synthesized moving image.

Then, adjacent pixels having the same speed on the synthesized moving image are grouped to obtain pixels located at the upper end and the lower end thereof, and a plane object, a three-dimensional object, and a moving object are determined based on coordinates after the overhead conversion of the pixels. Therefore, based on the captured image and reduced image obtained from one camera 10, the grouped pixel group can be determined as a three-dimensional object that is a candidate for a moving object.

Specifically, adjacent pixels having the same speed on the captured image are grouped to obtain pixels located at the upper end and the lower end, and the three-dimensional object is determined based on the coordinates after conversion to the overhead coordinates of the pixels. Therefore, it is possible to easily extract a target area including a three-dimensional object that moves based on a captured image obtained from one camera 10.

Furthermore, in this embodiment, a reference boundary for comparing movement information is obtained based on the coordinate value of the upper end pixel and the coordinate value of the lower end pixel, and the movement direction of the grouped pixel group and the movement of the reference boundary are obtained. Since a moving object can be detected based on the relationship with the direction, it is possible to easily determine whether or not the object is a moving object without estimating the observer's movement and calculating the distance to the object to be determined. it can.

In addition, even when the moving direction of the pixel group and the moving direction of the reference boundary are in a substantially opposite relationship, if the moving speed of the grouped pixel group becomes slower with the passage of time, the grouped pixels Since the group is determined to be a moving body, the determination of the moving body can be verified from a different point of view, that is, a change in moving speed with time, and the moving body can be detected with high accuracy.

In addition, since the feature point is a pixel or a pixel group corresponding to the edge of the object included in the captured image, the moving object can be easily detected by detecting the edge of the object on the captured image.

<Second Embodiment>
Next, the second embodiment will be described. The second embodiment is characterized by the content of reduced image generation processing by the reduced image generation unit 22. Except for this point, the basic configuration and processing of the second embodiment are common to those of the first embodiment, and therefore, different points will be mainly described here.

  FIG. 15 shows a block configuration of this embodiment. As illustrated in FIG. 15, the reduced image generation unit 22 includes a speed acquisition unit 221 that acquires the speed of the host vehicle on which the mobile body detection device 100 ′ is mounted (which can be equated with the speed of the mobile body detection device 100 ′). A reduction rate changing unit 222 that changes the reduction rate according to the speed. Other configurations are the same as those of the first embodiment described with reference to FIG.

  Hereinafter, each configuration of the reduced image generation unit 22 will be described.

  The reduced image generation unit 22 of the present embodiment generates a reduced image at a reduction rate corresponding to the moving speed of the moving object detection device 100 ′.

  The speed acquisition unit 221 included in the reduced image generation unit 22 acquires the vehicle speed from the vehicle controller 200 of the in-vehicle device 1000. Similarly, the reduction rate changing unit 222 included in the reduced image generation unit 22 changes the reduction rate according to the speed of the host vehicle.

Next, a method for determining the reduction rate based on the speed of the host vehicle will be described. If there is a speed V (km / h) of the host vehicle, a distance Z (m) from the host vehicle to an object, and a lateral position X (m), the speed of the stationary object at that position is expressed by the following formula (Formula 3). It can be calculated. Here, F represents the frame rate (fps), and PXr represents the angular resolution (Rad) per pixel in the horizontal direction.

S = F * ((ATAN (X / (Z + V / (3.6 * F)) − (ATAN (X / Z)) / PXr) (Formula 3)
If Expression 1 and Expression 2 described in the description of the first embodiment are used, the maximum speed of the stationary object in the imaging range and the maximum detection speed of the stationary object in the movement information detection target range can be calculated. The amount of movement (number of pixels) per frame is calculated from this maximum detection speed, and the reduction rate is determined so that the amount of movement is within one pixel. When obtaining the reduction ratio, the reduction ratio changing unit 222 uses a low speed range in a speed detection range (a range in which movement information is calculated) obtained from a reduced image with a high reduction ratio and a speed detection range obtained from an image with a low reduction ratio. It is preferable to obtain the reduction ratio of the reduced image generated so that the high-speed region in FIG.

Here, if the reduction ratio is set focusing on a stationary object, it seems that the speed detection range of the captured image is not necessarily required. However, since the speed of the moving object is observed in the low speed range, the speed information of the captured image is always It is preferable to measure. From this point of view, the composition unit 25 of this embodiment always generates composite information by using the movement information of the captured image, that is, generates composite information including the movement information of the captured image. It is a flowchart figure which shows the process of moving body detection apparatus 100 'in a form.

As shown in FIG. 16, when the ignition switch is turned on, the program is executed as a started program. In FIG. 16, the same processing steps as those in the flowchart of the first embodiment of the present invention shown in FIG. Here, the description will focus on the processing that is surrounded by a two-dot chain line and that is characteristic of the second embodiment.

  In step S1021, the required maximum detected speed of a stationary object is calculated from the speed of the host vehicle. After this, the flow moves to step S1022.

  In step S1022, the amount of movement per frame is calculated from the maximum detection speed, and a reduction rate is calculated such that the amount of movement falls within one pixel. After this, the flow moves to step S1023.

  In step S1023, the reduction ratio is set so that the low speed area of the speed detection range of the image with the high reduction ratio overlaps with the high speed area of the speed detection range of the image with the low reduction ratio. After this, the flow moves to step S1024.

  In step S1024, a reduced image of each reduction ratio is generated based on the set reduction ratio and recorded in the image memory 11. After this, the flow moves to step S103, and the same processing as in the first embodiment described with reference to FIG. 14 is performed.

  As described above, the moving object detection apparatus 100 ′ of the second embodiment has the following actions and effects in addition to the actions and effects of the first embodiment.

  By changing the reduction ratio in accordance with the speed of the moving body detection device 100 ′, it is possible to appropriately set detection areas having different speeds according to the moving speed. Although the speed information of the entire captured image changes according to the speed of movement such as the vehicle speed of the mounted vehicle, the speed detection area can be moved to an appropriate range by changing the reduction ratio. It becomes possible to always detect the speed information of the entire screen. For this reason, the identification accuracy which isolate | separates a moving body from a background can be ensured, without receiving to the influence of a vehicle speed.

  In addition, since the movement information of the captured image is always included in the composite information, the speed detection range in the captured image is prevented from being excluded even when the reduction rate is set by focusing on the stationary object as the background. can do. This is because the movement information of the moving body that affects the detection subject (the moving body detection device 100 or the host vehicle on which the moving body detection apparatus 100 is mounted) is observed in the low speed range, but this speed range is obtained from the captured image. is there. As a result, it is possible to calculate the movement information necessary for detecting the moving body while always detecting the speed information of the entire screen of the captured image, so that the moving body can be detected accurately.

  The embodiment described above is described for facilitating the understanding of the present invention, and is not described for limiting the present invention. Therefore, each element disclosed in the above embodiment is intended to include all design changes and equivalents belonging to the technical scope of the present invention.

  That is, in this specification, although the vehicle-mounted apparatus 1000 including the moving body detection apparatus 100 according to the present invention will be described as one aspect thereof, the present invention is not limited to this.

  Further, in this specification, as an aspect of the moving body detection apparatus 100, the camera 10 as an example of an imaging unit, the reduced image generation unit 22 as an example of a reduced image generation unit, and an example of a feature point extraction unit The movement provided with the feature point extraction part 23, the movement information calculation part 24 as an example of a movement information calculation means, the synthetic | combination part 25 as an example of a synthetic | combination means, and the mobile body detection part 30 as an example of a mobile body detection means Although the body detection apparatus 100 has been described, the present invention is not limited to this.

  Furthermore, as described in this specification, the moving body detection apparatus 100 includes a grouping unit 31 as an example of a grouping function, a coordinate conversion unit 32 as an example of a coordinate conversion function, and an example of a determination function. And a determination unit 33.

  Although the embodiment of the present invention has been described in detail with reference to the drawings, the embodiment is merely an example of the present invention, and the present invention is not limited only to the configuration of the embodiment. Therefore, it goes without saying that the present invention includes the following design changes and the like without departing from the scope of the present invention.

  For example, the block diagram is not limited to that shown in the above-described embodiment, and may have other configurations having equivalent functions.

  The camera mounting position is not limited to the position described in the embodiment. The optical axis of the camera faces the front front direction (Z direction) of the vehicle, and the horizontal axis and the vertical axis of the imaging surface are substantially the same as the road surface, respectively. What is necessary is just to set so that it may become parallel and substantially perpendicular | vertical.

In addition, in normalizing the detected edge width, the edge width is not limited to three pixels, and an arbitrary number of pixels can be set. In this case, since the pixel at the center of the edge is used in the subsequent processing, the number of pixels with the edge width is desirably an odd number.

Further, the number of strip regions set by dividing the xy plane is not limited to that shown in the above embodiment, and can be set by dividing it into an arbitrary number.

Further, the number of regions set by dividing the ZX plane is not limited to that shown in the above embodiment, and can be set by dividing it into an arbitrary number.

Further, the vertical and horizontal ranges of the ZX plane can be set to arbitrary values.

Moreover, although the said embodiment demonstrated the example which mounts the mobile body detection apparatus 10 in the vehicle which drive | works a road, you may mount in another mobile body.

Furthermore, in the above-described embodiment, the example of the curbstone, the white line, and the contact point between the outer wall and the road surface has been described as the road boundary to be detected, but is not limited thereto, for example, the guard rail, the boundary between the parked vehicle and the road surface, You may detect the boundary with areas (fields, fields, etc.) other than a road surface.

It is a figure which shows an example of the block configuration of the mobile body detection apparatus of 1st Embodiment. (A) And (B) is a figure which shows the example of mounting of the camera 10. FIG. It is a figure which shows an example of the captured image which imaged the front of the own vehicle. (A)-(c) is a figure for demonstrating a reduction image. (A) is an example of a captured image, (b) is an example of a reduced image reduced to ½ with respect to the captured image, and (c) is reduced to ¼ with respect to the captured image. It is an example of a reduced image. (A)-(f) is a figure for demonstrating the calculation process example of a moving speed. It is a figure for demonstrating the synthetic | combination process of movement information. It is a figure which shows an example of the synthetic | combination moving image as synthetic | combination information. It is a figure for demonstrating the example of a grouping process. It is a figure for demonstrating the example of a process for determining a solid object. It is a figure for demonstrating the example of a method which extracts a track boundary. It is a figure for demonstrating the example of the method of extracting the runway boundary used as a background. It is a figure for demonstrating the example of a method which determines a moving body based on the extracted track boundary. It is a figure for demonstrating the process example which determines a moving body. It is a flowchart figure which shows an example of a mobile body detection process. It is a figure which shows an example of the block configuration of the mobile body detection apparatus of 2nd Embodiment. It is a flowchart figure which shows the example of creation of the other reduced image based on 2nd Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1000 ... In-vehicle apparatus 100,100 '... Moving body detection apparatus 10 ... Camera 11 ... Image memory 20 ... Image processing part 21 ... Captured image acquisition part 22 ... Reduced image generation part 23 ... Feature point extraction part 24 ... Movement information calculation part 25 ... Composition unit 30 ... Moving object detection unit 31 ... Grouping unit 32 ... Coordinate conversion unit 33 ... Determination unit 200 ... Vehicle controller 300 ... Alarm device 400 ... Driving support device 500 ... Output device (display / speaker)

Claims (13)

Imaging means;
Reduced image generating means for generating reduced images having different reduction ratios from the captured images obtained by the imaging means;
Feature point extraction means for extracting the feature points of the reduced image generated by the reduced image generation means and the feature points of the captured image obtained by the imaging means;
Movement information calculation means for calculating movement information including movement speed and / or movement direction of each feature point extracted by the feature point extraction means;
Synthesis means for generating synthesis information from movement information of feature points of the reduced image calculated by the movement information calculation means and movement information of feature points of the captured image;
And a moving body detection unit that detects the moving body based on the combined information generated by the combining unit. The moving object detection apparatus according to claim 1,
The synthesizing unit generates synthesis information from the movement information of each feature point extracted by the feature point extraction unit in the captured image and the movement information of the feature point of the reduced image corresponding to the feature point of the captured image. A moving body detection device. In the moving body detection device according to claim 2,
The synthesizing unit generates synthesis information based on movement information of feature points of the reduced image having the highest reduction ratio among the movement speeds of the feature points of the reduced images respectively corresponding to the feature points extracted in the captured image. A moving body detection device. In the moving body detection apparatus of Claim 3,
When the moving speed of the feature point extracted in the captured image is not calculated, the synthesizing unit has a reduced image with the highest reduction ratio among the moving speeds of the feature points of the reduced images corresponding to the feature points. The moving body detection apparatus which produces | generates synthetic | combination information based on the movement information of a feature point. In the moving body detection apparatus of Claim 3 or 4,
The synthesizing unit is characterized in that, among the movement information of the feature points of each reduced image corresponding to the feature points extracted in the captured image, the direction difference from the movement direction of the feature points of the captured image is less than a predetermined value A moving body detection apparatus that generates composite information based on movement information of feature points of a reduced image that is a point and has the highest reduction ratio. In the moving body detection apparatus as described in any one of Claims 1-5,
It further comprises speed acquisition means for acquiring the speed of the mobile object detection device,
The reduced image creation means is a moving object detection device that generates a reduced image at a reduction rate corresponding to the speed acquired by the speed acquisition means. The moving body detection device according to claim 6,
The said synthetic | combination means is a moving body detection apparatus which produces | generates the synthetic | combination information containing the movement information of a captured image. In the moving body detection apparatus in any one of the said Claims 1-7,
The movement information calculation means counts up the count value of the pixel corresponding to the position of the feature portion extracted by the feature point extraction means at a predetermined period, and corresponds to the feature portion based on the slope of the count value. A moving body detection apparatus that calculates pixel movement information. In the moving body detection apparatus as described in any one of Claims 1-8,
The synthesized information is a synthesized moving image synthesized from feature point movement information of the reduced image and feature point movement information of the captured image,
The moving body detecting means has a grouping function for grouping adjacent pixel groups in a substantially vertical direction in the synthesized moving image, wherein the moving speed calculated by the moving information calculating means is within a predetermined range;
A coordinate conversion function for converting the pixels included in the composite moving image into a conversion coordinate that is an overhead view seen from a predetermined viewpoint and divided into predetermined divided regions;
When the coordinates of the pixel located at the upper end in the transformed coordinate belong outside the area of the transformed coordinate and the coordinate of the pixel located at the lower end belongs to any one of the divided areas, the grouped pixel group is A moving body detection apparatus having a determination function for determining that a solid object is a candidate for a moving body. In the moving body detection apparatus as described in any one of Claims 1-9,
The synthesized information is a synthesized moving image synthesized from feature point movement information of the reduced image and feature point movement information of the captured image,
The moving body detecting means is a grouping function for grouping pixel groups adjacent to each other in a substantially vertical direction in the synthesized moving image and having a moving speed calculated by the moving information calculating means within a predetermined range;
A reference boundary having a predetermined relationship with the installation surface of the three-dimensional object included in the synthesized moving image based on the coordinate value of a pixel located at the lower end in the substantially vertical direction of the synthesized moving image among the grouped pixel group The movement direction of the pixel group grouped by the grouping function and the movement direction of the reference boundary where the y-coordinate value in the vertical direction of the pixel group and the synthesized moving image is common are defined in advance. And a determination function for determining that the grouped pixel group is a moving object in the case of a substantially opposite relationship. In the moving body detection apparatus of Claim 10,
In the determination function, a moving direction of the pixel group grouped by the grouping function and a moving direction of the reference boundary where the y-coordinate value in the vertical direction in the captured image and the pixel group are common are defined in advance. Even if the relationship is in a substantially opposite direction, if the moving speed of the grouped pixel group becomes slower as time passes, the grouped pixel group is determined to be a moving object. Body detection device. In the moving body detection apparatus as described in any one of Claims 1-11,
The moving object detection apparatus, wherein the feature point is a pixel or a pixel group corresponding to an edge of an object included in the captured image. Generate reduced images with different reduction ratios from captured images,
Calculating movement information including the moving speed and / or moving direction of the feature point in the reduced image and the feature point in the captured image;
Generating composite information from movement information of feature points of the reduced image and movement information of feature points of the captured image;
A moving body detection method for detecting a moving body based on the composite information.

Images