JP2016053867A - Object detection device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an object detection device that prevents incorrect detection of an object other than a vehicle as a vehicle.SOLUTION: A horizontal edge extraction part extracts a horizontal edge h1 from an image 21 photographed with a camera. A feature point extraction part extracts feature points P1 to P5 present in a first search range SA1 that is set in an upper direction of the horizontal edge h1. A three-dimensional coordinate calculation part calculates three-dimensional coordinates of a real space as for a feature point group in which a corresponding relationship in time-series photographed images has been determined by the feature point correspondence determination part. An object recognition part, when a feature point having a height of a real space position equal to or higher than a predetermined value is present, recognizes that the horizontal edge h1 is an image part of a vehicle.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an object detection device that detects an object existing in the vicinity based on a captured image of a camera.

  2. Description of the Related Art Conventionally, an object detection apparatus that detects a preceding vehicle by extracting a horizontal edge component from a captured image in front of the host vehicle by an in-vehicle camera is known (see, for example, Patent Document 1).

  In the object detection device described in Patent Document 1, a ratio between the width of the extracted horizontal edge and the road width is calculated, and a horizontal edge whose ratio is within a predetermined range is detected as a lower end candidate of the preceding vehicle. When another horizontal edge line exists within a predetermined range above the horizontal edge detected as the lower end candidate, the horizontal edge detected as the lower end candidate is recognized as the image portion of the lower end of the preceding vehicle. Yes.

JP-A-8-43083

  In the object detection apparatus described in Patent Document 1, a plurality of horizontal portions are obtained for an image portion of a planar object (a road surface sign, a shadow of an object existing on a road, etc.) that is long in the depth direction (camera imaging direction) in real space. When the edge is extracted, the horizontal edge at the lower end is erroneously recognized as the image portion at the lower end of the preceding vehicle.

  The present invention has been made in view of such a background, and an object thereof is to provide an object detection device that prevents an object other than a vehicle from being erroneously detected as a vehicle.

The present invention has been made to achieve the above object, and the object detection apparatus of the present invention includes:
A camera that captures the surroundings,
A horizontal edge extraction unit for extracting a horizontal edge from a captured image of the camera;
In the captured image, a feature point extraction unit that extracts a feature point existing in a first search range set in the upward direction of the horizontal edge;
A real space position calculation unit for calculating the height of the real space position corresponding to the feature point;
A vehicle recognition unit is provided that recognizes that the horizontal edge is an image portion of a vehicle when the corresponding feature point whose height of the corresponding real space position is a predetermined value or more exists.

  According to the present invention, a horizontal edge is extracted from the captured image of the camera by the horizontal edge extraction unit, and a feature point existing in the first search range set in the upward direction of the horizontal edge is the feature point. Extracted by the extraction unit. The vehicle recognizing unit is configured such that when there is a feature point whose height of the corresponding real space position is equal to or greater than the predetermined value among the feature points extracted within the first search range, the horizontal edge is the vehicle. It is recognized as an image part. In this case, among the feature points extracted in the first search range, the corresponding real space position has a height equal to or lower than the predetermined value (an object having no height such as a road sign or a road repair mark) Can be excluded). Therefore, it is possible to prevent a planar object other than the vehicle from being erroneously recognized as a vehicle.

  The vehicle recognizing unit recognizes that the horizontal edge is an image portion of the vehicle when there are a predetermined number or more of the feature points whose corresponding real space positions are higher than the predetermined value. Features.

  According to this configuration, an object having a plurality of characteristic portions above the real space position corresponding to the image portion of the horizontal edge can be recognized more accurately as a vehicle.

  In addition, the vehicle recognition unit may detect other horizontal edges existing in the second search range set in the upper direction of the horizontal edge recognized as the image portion of the vehicle in the captured image in the image portion of the vehicle. It is characterized by being recognized.

  According to this configuration, another horizontal edge (assumed to be an image portion such as a window frame or a roof) existing in the vicinity of the upper side in the horizontal direction recognized as an image portion of the vehicle is connected to the same vehicle. It can be recognized as an image part.

  The vehicle recognizing unit may set the second search range according to a distribution of the feature points in the first search range.

  According to this configuration, an appropriate range corresponding to the position of the object estimated from the distribution of the feature points in the first search range can be set as the second search range.

  The vehicle recognizing unit may change a setting range of the second search range according to a country or a region where the object detection device is used.

  According to this configuration, for example, when the object detection device is used in a country or region where the traffic volume of a large vehicle is large, the second search range is set wide, and the object detection device has a large traffic volume of a small vehicle. When used in a country or region, by setting the second search range to be narrow, a horizontal edge that is recognized as an image portion of the same object can be searched efficiently.

The block diagram of an object detection apparatus. The flowchart of a vehicle recognition process. Explanatory drawing of extraction of the horizontal edge and feature point in a vehicle recognition process.

  An example of an embodiment of the object detection device of the present invention will be described with reference to FIGS.

  Referring to FIG. 1, an object detection device 10 is mounted on a vehicle 1 that includes a camera 2 (color camera), a warning control unit 3, a vehicle control unit 4, and the like.

  The object detection device 10 is an electronic circuit unit configured by a CPU, a memory, various interface circuits, and the like (not shown). By executing a program for object detection held in the memory by the CPU, the captured image acquisition unit 11, It functions as a horizontal edge extraction unit 12, a feature point extraction unit 13, a feature point correspondence determination unit 14, a three-dimensional coordinate calculation unit 15, a detection target edge selection unit 16, a target object recognition unit 17, and a target object position calculation unit 18.

  The captured image acquisition unit 11 inputs a video signal around the vehicle 1 (forward in this embodiment) output from the camera 2 for each predetermined control cycle, and the color component (R value, G) of the video signal. Value, B value) is obtained, and a color time-series captured image having R value, G value, and B value is obtained as data of each pixel.

  The horizontal edge extraction unit 12 performs a process of converting each pixel of the color captured image into a gradation, and generates a gray captured image (multi-valued image) 21. Data of time-series gray captured images 21 (21a, 21b,..., 21n) generated in this way is held in the image memory 20.

  Note that when the camera 2 is a monochrome camera, a grayscale image is obtained from the gradation of each pixel, and thus the process of generating the gray captured image 21 from the color captured image described above is unnecessary.

  For the gray captured image 21, the horizontal edge extraction unit 12 has an edge point (a pixel whose gradation difference (amount of change in gradation) with a surrounding pixel is greater than or equal to a predetermined value. The gradation changes from dark to bright. A positive edge point and a negative edge point whose gradation changes from light to dark are extracted, and are composed of edge points continuous in a horizontal edge (horizontal direction (x direction shown in FIG. 3)). (Line segment) is extracted. From the gray captured image 21 in FIG. 3, seven horizontal edges h1 to h7 are extracted.

  The object detection apparatus 10 performs processing (vehicle recognition processing) according to the flowchart shown in FIG. 2 on the horizontal edge extracted by the horizontal edge extraction unit 12, and each horizontal edge is an image portion of the vehicle (preceding vehicle). Determine whether or not. Hereinafter, the processing according to the flowchart illustrated in FIG. 2 will be described with reference to the example of the captured image illustrated in FIG. 3.

  The gray captured image 21 shown in FIG. 3 is a two-dimensional image based on the horizontal coordinate x and the vertical coordinate y. The image portion 60 of the preceding vehicle, the image portion 61 of the road repair trace, and the object existing on the road shoulder. A shadow image portion 62 is included. The negative direction of the vertical coordinate y corresponds to the upward direction in the captured image of the present invention.

  STEP 1 in FIG. 2 is processing by the detection target edge selection unit 16. The detection target edge selection unit 16 sequentially selects horizontal edges (h1 to h7 in FIG. 3) to be processed in STEP2 and subsequent steps.

  The subsequent STEP 2 is processing by the feature point extraction unit 13. The feature point extraction unit 13 sets the first search range above the selected horizontal edge and extracts the feature points. For example, in FIG. 3, the first search range SA1 is set above h1 (the negative direction of the y axis) with respect to the horizontal edge h1 extracted with respect to the lower end of the image portion 60 of the vehicle. Five feature points P1 to P5 are extracted in SA1.

  Further, a first search range SA2 is set above h5 with respect to the horizontal edge h5 extracted with respect to the lower end of the image portion 61 of the road repair trace, and two feature points P6 are set within the first search range SA2. , P7 are extracted.

  In addition, the first search range SA3 is set above h6 with respect to the horizontal edge h6 extracted with respect to the lower end of the shadow image portion 62 of the object existing beside the road, and 2 within the first search range SA3. Feature points P8 and P9 are extracted.

  Here, an image portion having a shape unique to an artificial object such as a corner edge or a rectangle is extracted as the feature point.

  The next STEP 3 is processing by the object recognition unit 17. The target object recognition unit 17 determines whether or not a predetermined number (for example, four) or more feature points have been extracted by the processing of STEP2. If a predetermined number or more of feature points are not extracted, it can be determined that the selected horizontal edge is not the image portion at the lower end of the three-dimensional object, so that the process branches to STEP 20, and the object recognition unit 17 The horizontal edge is excluded from the vehicle detection targets, and the process proceeds to STEP8.

  On the other hand, when a predetermined number or more of feature points have been extracted, the process proceeds to STEP 4 where the feature point correspondence determination unit 14 and the three-dimensional coordinate calculation unit 15 determine the height of the real space position corresponding to the feature point (height from the ground surface). Is calculated.

  The feature point correspondence determining unit 14 determines the correspondence relationship between a plurality of feature points extracted from the first search range among the time-series gray captured images 21a, 21b,. Then, the three-dimensional coordinate calculation unit 15 uses a known SFM (Structure From Motion) technique to create a matrix from the corresponding point group of feature points between a plurality of time-series images, and to obtain the three-dimensional position of the feature points. Factoring into a matrix that represents the position of the camera 2 and a matrix that represents the position of the camera 2, the three-dimensional coordinates of the real space position corresponding to the feature point (Z is the traveling direction of the vehicle 1, X is the vehicle width direction, Y is the vehicle height direction) Is calculated).

  In addition, the structure which calculates the real space position corresponding to a feature point with the feature point corresponding | compatible determination part 14 and the three-dimensional coordinate calculation part 15 is equivalent to the real space position calculation part of this invention.

  The object recognition unit 17 determines whether the height (Y coordinate) of the real space position corresponding to the feature point is equal to or greater than a threshold value (corresponding to a predetermined value of the present invention). When the height is greater than or equal to the threshold, it can be determined that the image portion is a three-dimensional object.

  When the height of the feature point is equal to or greater than the threshold value, the process proceeds to STEP 5 where the object recognition unit 17 sets the selected horizontal edge as a detection target (image portion of the vehicle). In the example of FIG. 3, it is determined that the heights of the feature points P1 to P5 extracted with respect to the horizontal edge h1 are equal to or greater than the threshold value, and the horizontal edge h1 is recognized as an image portion of the vehicle.

  In subsequent STEP 6, as shown in FIG. 3, the object recognizing unit 17 sets the feature point P1 on the upper side of the horizontal edge h1 according to the distribution of the feature points P1 to P5 extracted in the first search range SA1. A region obtained by offsetting the positions of P5 to the outside is set as the second search region SB1. In this way, not the limit of the positions of the feature points P1 to P5 but the area offset outward is set as the second search area SB1 because most of the feature points P1 to P5 are image portions at the corners of the vehicle. When the second search area SB1 is set at the very limit of the feature points P1 to P5, the image portion of the horizontal line that protrudes from the second search area SB1 increases, and processing for excluding the horizontal edge of the image portion of the same vehicle, which will be described later, is performed. This is because it cannot be done effectively.

  Also, as a method of setting the second search range SB1, the distance between the feature points P1 to P5 and the distribution position are applied to the vehicle type determination conditions set in advance so that the feature points P1 to P5 are trucks, buses, It may be estimated which type of vehicle such as a passenger car is likely to be an image portion, and the range of the second search area SB1 may be set according to the standard size of the estimated type of vehicle.

  Furthermore, the setting range of the second search area SB1 may be changed according to the country or region where the object detection device 10 is used. For example, when used in the United States where there are many large vehicles, the second search area SB1 may be set wider than in Japan where there are many small vehicles. In addition, when the object detection device 10 is used on a highway with a large amount of truck travel, the second search area SB1 may be set wide in the upward direction (the height direction of the vehicle in real space). Good.

  In addition, the structure which the target object recognition part 17 recognizes that a horizontal edge is an image part of a vehicle, and the structure which sets 2nd search range SB1 are equivalent to the vehicle recognition part of this invention.

  In the next STEP 7, the object recognition unit 17 determines the presence or absence of another horizontal edge, branches to STEP 30 when there is another horizontal edge, and proceeds to STEP 8 when there is no other horizontal edge. In STEP 30, the object recognition unit 17 excludes the found horizontal edge from the detection target, and proceeds to STEP 8.

  In the example of FIG. 3, horizontal edges h2, h3, and h4 found in the second search range SB1 are excluded from detection targets. Thus, by excluding the horizontal edge assumed to be extracted from the image portion of the same vehicle from the detection target, the subsequent vehicle recognition processing can be performed efficiently.

  In STEP 8, the object recognition unit 17 determines whether there is an undetermined horizontal edge. If there is an undetermined horizontal edge, the process returns to STEP 1 to select the next horizontal edge to be recognized, and the processing from STEP 2 onward is executed. Further, when there is no undetermined horizontal edge, the process proceeds to STEP 9 and the process is terminated.

  The object position calculation unit 18 calculates the position of the preceding vehicle with respect to the vehicle 1 (own vehicle) from the position in the real space corresponding to the horizontal edge recognized as the image portion of the vehicle by the object recognition unit 17. To do. And the object detection apparatus 10 performs the warning sound output by the warning control part 3, the warning display to a display part (not shown), the braking control by the vehicle control part 4, steering control, etc. as needed.

  In the present embodiment, the object detection device 10 that recognizes the vehicle by processing a captured image of the camera mounted on the vehicle 1 is shown. However, as another embodiment, information of a camera built-in type that a pedestrian carries. An embodiment configured as a function of a processing terminal (smart phone, mobile phone, tablet, etc.) or an embodiment configured as a function of a traffic infrastructure facility that is installed beside a road and recognizes a road situation from a captured image of a camera is also possible. .

  DESCRIPTION OF SYMBOLS 1 ... Vehicle, 2 ... Camera, 3 ... Warning control part, 4 ... Vehicle control part, 10 ... Object detection apparatus, 11 ... Captured image acquisition part, 12 ... Horizontal edge extraction part, 13 ... Feature point extraction part, 14 ... Feature Point correspondence determination unit, 15 ... three-dimensional coordinate calculation unit, 16 ... detection target edge selection unit, 17 ... target object recognition unit, 18 ... target object position calculation unit.

Claims (5)

A horizontal edge extraction unit that extracts a horizontal edge from an image captured by a camera that images the periphery;
In the captured image, a feature point extraction unit that extracts a feature point existing in a first search range set in the upward direction of the horizontal edge;
A real space position calculation unit for calculating the height of the real space position corresponding to the feature point;
An object detection comprising: a vehicle recognition unit for recognizing that the horizontal edge is an image portion of a vehicle when the corresponding feature point having a height of a corresponding real space position equal to or greater than a predetermined value exists apparatus. The object detection apparatus according to claim 1,
The vehicle recognizing unit recognizes that the horizontal edge is an image portion of a vehicle when there are a predetermined number or more of feature points whose corresponding real space positions are equal to or higher than the predetermined value. Object detection device. In the object detection device according to claim 1 or 2,
In the captured image, the vehicle recognizing unit determines that another horizontal edge existing in the second search range set above the horizontal edge recognized as the image portion of the vehicle is the image portion of the vehicle. An object detection device characterized by recognition. The object detection device according to claim 3,
The vehicle recognition unit sets the second search range according to a distribution of the feature points in the first search range. In the object detection device according to claim 3 or 4,
The vehicle recognition unit changes the setting range of the second search range according to a country or a region where the object detection device is used.

Images