JP2012159470A - Vehicle image recognition device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a vehicle image recognition device having a correction function for improving accuracy of a result of measuring the distance to an object.SOLUTION: The vehicle image recognition device includes imaging means (monocular camera 11) outputting an image of surroundings of a vehicle, and measuring means (three-dimensional object extracting part 15, distance correction part 16) for measuring the distance to an object by measuring a vertical position on the image of the object from the image. The measuring means corrects the distance to the object by use of a vertical position on the image of a road sign measured by a method different from the method of measuring the distance to the object.

Description

  The present invention comprises imaging means for outputting an image around the vehicle, and measuring means for measuring the distance to the distance measurement object by actually measuring the vertical position of the distance measurement object on the image from the image. The present invention relates to a vehicle image recognition apparatus.

  As a conventional technique, in a two-dimensional image, a technique for acquiring stereoscopic information by obtaining a distance from a viewpoint to a non-reference point based on an illuminance difference based on a reference point whose distance from the viewpoint is known is known. (For example, see Patent Document 1).

JP 2010-2273 A

  However, in order to measure the distance to the distance measurement target in an actual traffic environment, it is necessary to efficiently remove factors on the imaging means side that can change the measurement value by performing some correction.

  For example, when measuring the distance from a vertical position on a captured image of a distance measurement object to the distance measurement object, an error is included in the measurement value due to vehicle pitch variation due to vehicle pitching or vehicle posture change, etc. .

  SUMMARY OF THE INVENTION An object of the present invention is to provide a vehicle image recognition apparatus having a correction function for increasing the accuracy of distance measurement results.

In order to achieve the above object, a vehicle image recognition apparatus according to the present invention includes:
Imaging means for outputting an image around the vehicle;
Measuring means for measuring the distance to the distance measurement object by actually measuring the vertical position on the image of the distance measurement object from the image;
The measuring means includes
The distance to the distance measuring object is corrected by a road marking whose distance is measured by a method different from the method of measuring the distance to the distance measuring object.

  According to the present invention, it is possible to perform correction that increases the accuracy of the distance measurement result.

It is the block diagram which showed the structure of the image recognition apparatus for vehicles 1 which is one Embodiment of this invention. It is the figure which showed the captured image 21 of the advancing direction of the own vehicle obtained with the monocular camera 11. FIG. It is the flowchart which showed the process example until calculating the distance to the pedestrian crossing which is a road marking. It is a figure for demonstrating the pedestrian crossing 25 on the captured image 21 of FIG. It is the figure which showed the distance L to the pedestrian crossing 25 which has the pixel width a on the captured image 21. FIG. It is the calculation result of the distance to the pedestrian crossing in the real scene in a general road environment. It is the flowchart which showed the process example until calculating the distance to the pedestrian who is a ranging object. It is the figure which showed the relationship between the attachment height Hc of the camera 11, and the ordinates Yv and Yo at the distance L. It is a graph showing Formula (3). It is the figure which showed an example of the distance correction result of the distance correction part 16 based on Formula (4).

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration of a vehicle image recognition apparatus 1 according to an embodiment of the present invention. The vehicular image recognition apparatus 1 is mounted on a vehicle, and includes a monocular camera 11, a road marking extraction unit 12, a size detection unit 13, a distance calculation unit 14, a three-dimensional object extraction unit 15, and a distance correction unit 16. It has. Part or all of the functions of each unit can be realized by a processor, a microcomputer, or the like that executes image recognition processing.

  The monocular camera 11 is an imaging unit that captures an image of the periphery of the host vehicle (for example, the traveling direction of the host vehicle) at a predetermined angle of view and outputs the captured image. The traveling direction of the host vehicle may be the forward direction or the reverse direction. The monocular camera 11 includes an image sensor such as a CCD or a CMOS.

  The road sign extraction unit 12 is a means for extracting a predetermined part of the road sign from the image captured by the monocular camera 11. The predetermined part extracted by the road marking extraction unit 12 is predetermined in design, and may be the whole or a part of the road marking.

  In the road marking, characters, symbols, or combinations thereof are marked on the road surface by paint or the like. The size of the road marking is based on dimensions predetermined by laws and regulations. Specific examples of road markings include regulation markings and instruction markings. Specific examples of the restriction marking include a turn prohibition mark and a maximum speed mark. Specific examples of the indication sign include a pedestrian crossing, a rhombus mark indicating that there is a pedestrian crossing ahead, a triangular mark indicating that there is a priority road ahead, and a traveling direction mark.

  FIG. 2 is a view showing a captured image 21 of the traveling direction of the host vehicle obtained by the monocular camera 11. The captured image 21 includes a pedestrian crossing 25 marked on the course ahead of the host vehicle as a road marking. The code | symbol 26 represents a vanishing point (infinity point). In the case of FIG. 2, the road marking extraction unit 12 extracts candidate lines p <b> 1 to p <b> 4 constituting the pedestrian crossing 25 from the captured image 21, for example. The captured image 21 is an image in which the number of pixels in the horizontal direction is W and the number of pixels in the vertical direction is H.

  The size detection unit 13 is a means for detecting the size (preferably the lateral width) on the image captured by the monocular camera 11 for a predetermined part of the road marking extracted by the road marking extraction unit 12. For example, the size detection unit 13 can detect the size of the predetermined portion on the captured image by acquiring the number of pixels constituting the predetermined portion of the road marking from the captured image.

  The distance calculation unit 14 determines a predetermined size in the standard for a predetermined part of the road sign extracted by the road sign extraction unit 12, a size actually detected by the size detection unit 13, and the monocular camera 11. It is a means for calculating the distance from the host vehicle to the road marking based on the angular resolution.

  FIG. 3 is a flowchart showing a processing example until the distance to the pedestrian crossing, which is a road marking, is calculated. The flowchart will be described with reference to FIGS.

  In step S <b> 10, the road sign extraction unit 12 extracts an edge in the vertical direction (vertical direction) where the luminance of the pixel suddenly changes from the captured image 21 based on the pixel luminance distribution of the captured image 21.

  In step S20, the road sign extraction unit 12 classifies the edges extracted in step S10 according to a luminance change pattern in the horizontal direction (left and right direction when the pedestrian crossing is viewed from the own vehicle). The road marking extraction unit 12 determines that the left region is brighter than the right region based on the pixel luminance distribution of the left region and the right region of each edge when the left region is darker than the right region. In this case, it is determined as “falling edge”.

  In step S <b> 30, the road sign extraction unit 12 searches the captured image 21 in the horizontal direction, and groups lines in which the edges classified in step S <b> 20 appear alternately and repeatedly appear as candidate lines constituting the pedestrian crossing 25. To do. Then, the road sign extraction unit 12 searches the captured image 21 in the vertical direction, groups the candidate lines according to the adjacent relationship of the grouped candidate lines and the connection relationship of the edges constituting the candidate lines, and crosses the candidate lines. Extract the sidewalk area.

  For example, the road sign extraction unit 12 includes a pair of rising edges and falling edges adjacent to each other, and there are a predetermined number or more pairs (for example, 3 pairs or more), and each of the predetermined number or more of pairs. When the distances in the horizontal direction (left-right direction) between the two edges constituting the pair (for example, distances b1, b2, b3 in the case of FIG. 4) are within a predetermined range, the respective pairs are designated as crosswalk 25. Are extracted as candidate lines, and the extracted candidate lines are grouped as pedestrian crossing areas.

  In step S40, the size detection unit 13 detects the number of pixels in the left-right direction (width) of the pedestrian crossing area, and the distance calculation unit 14 uses the detected number of pixels in the left-right direction to monocular the host vehicle. The distance from the camera 11 to the pedestrian crossing 25 is calculated.

FIG. 4 is a diagram for explaining the pedestrian crossing 25 on the captured image 21 of FIG. Among the pairs extracted as the pedestrian crossing area by the road marking extraction unit 12, the size detection unit 13 is, for example, from the rising edge p1a of the leftmost pair to the falling edge p3b of the third pair from the left. The pixel width a for three pairs corresponding to the distance is detected. The width length between both ends of the three pairs on the standard of the pedestrian crossing 25 (that is, a value required in the standard of the part corresponding to the portion of the pixel width a) is A, and the angular resolution in the horizontal direction of the monocular camera 11 is As shown in FIG. 5, if P is P and L is the distance from the vehicle to the pedestrian crossing 25,
L = A / tan (a × P) (1)
The following relational expression holds. Here, the angle resolution P is defined as “P = when the horizontal angle of view of the monocular camera 11 is α [degrees] and the number of pixels of the captured image 21 obtained by the monocular camera 11 is W [pix.]. It is a fixed value that can be represented by “α / W [degree / pix.]”. That is, the distance calculation unit 14 can calculate the distance L to the pedestrian crossing 25 based on the formula (1).

  Of course, as long as it is a pair of edges extracted as a pedestrian crossing region, the pixel width between any edges may be detected. If the detected pixel width is a, the part corresponding to the portion of the pixel width a The standard value corresponds to the fixed value A.

  Therefore, the vehicular image recognition apparatus 1 includes the monocular camera 11, the road marking extraction unit 12, the size detection unit 13, and the distance calculation unit 14 having the above-described configuration, so that a predetermined part of the road marking such as a pedestrian crossing is provided. Since the distance to the road marking is calculated by taking into account the size of the standard, the distance to the road marking is captured by a monocular camera even if the road marking is far away from the host vehicle. Can be calculated with high accuracy.

  FIG. 6 is a calculation result of the distance to the pedestrian crossing in a real scene under a general road environment, where (a) shows an embodiment of the present invention, and (b) Comparison with the conventional technology that calculates the distance from the lowest ordinate of the pedestrian crossing area on the image to the pedestrian crossing based on the correspondence between the ordinate representing the position of the direction and the distance to the distance measuring point An example is shown. As is clear from FIG. 6, the embodiment of the present invention is more effective in disturbances such as a change in the pitch of the host vehicle and a change in the road gradient than the conventional technique in which the distance is calculated based on the ordinate representing the vertical position. Therefore, it can be seen that the distance can be calculated accurately with little deviation from the true value of the distance to the pedestrian crossing.

  Incidentally, as shown in FIG. 1, the vehicular image recognition apparatus 1 further includes a three-dimensional object extraction unit 15 and a distance correction unit 16 as measurement means for measuring the distance to the distance measurement target (three-dimensional object). .

  The three-dimensional object extraction unit 15 is a captured image obtained by the monocular camera 11 of a three-dimensional object that exists in the vicinity of a road marking that exists in the traveling direction of the host vehicle (for example, on the road where the road marking is marked or on the side of the road). To extract by pattern recognition such as template matching. Specific examples of this three-dimensional object include pedestrians, other vehicles such as preceding vehicles and parked vehicles, and obstacles (including pedestrians) that may collide with the host vehicle.

  The three-dimensional object extraction unit 15 actually measures the ordinate representing the vertical position on the captured image of the extracted three-dimensional object. For example, as illustrated in FIG. 2, the three-dimensional object extraction unit 15 measures the ordinate Yp on the captured image 21 at the lowermost end of the pedestrian 22.

  The distance correction unit 16 corrects the distance to the three-dimensional object by using a road marking whose distance is measured by a method different from the method described later for measuring the distance to the three-dimensional object, and calculates the correction value. It is. This other method is, for example, the above-described distance calculation method executed by the distance calculation unit 14.

  FIG. 7 is a flowchart showing an example of processing until calculating the distance to a pedestrian who is a distance measurement target. The flowchart will be described with reference to FIGS.

  The area detection of the pedestrian crossing 25 in step S80 and the calculation of the distance L to the pedestrian crossing 25 in step S90 are the same as described above.

In step S <b> 100, the road sign extraction unit 12 calculates the ordinate Yo representing the vertical position on the captured image 21 corresponding to the distance L calculated by the distance calculation unit 14 based on the above formula (1). . Assuming that the mounting height of the monocular camera 11 from the road surface is Hc, the Y coordinate of the vanishing point 26 is Yv, the horizontal angular resolution of the monocular camera 11 is P, and the vertical angular resolution of the monocular camera 11 is Q. As is clear from 8,
Yo = Yv + (tan ⁻¹ (Hc / L)) / Q (3)
The following relational expression holds. Here, the angular resolution Q can be considered in the same manner as the angular resolution P described above. That is, the angle resolution Q is defined as “Q = β /, where β [deg.] Is the vertical angle of view of the monocular camera 11 and H [pix.] Is the number of pixels in the vertical direction of the captured image obtained by the monocular camera 11. It is a fixed value that can be represented by “H [degree / pix.]”. That is, the road sign extraction unit 12 can calculate the ordinate Yo based on the equation (3).

  FIG. 9 is a graph representing Expression (3) (Q = 0.027, Yv = 240, Hc = 1.28). For example, when the distance L to the pedestrian crossing 25 calculated by the distance calculation unit 14 is 40 m, the ordinate Yo is 308 pixels.

  In step S <b> 110, the road sign extraction unit 12 calculates the pitch fluctuation amount of the host vehicle based on the vertical movement of the pedestrian crossing 25 on the captured image 21. That is, the road sign extraction unit 12 first measures the ordinate representing the vertical position on the captured image 21 of the extracted pedestrian crossing 25. For example, as illustrated in FIG. 2, the road marking extraction unit 12 measures the ordinate Yc on the captured image 21 at the lowermost end of the pedestrian crossing 25. Then, the road sign extracting unit 12 calculates a difference ΔY between the ordinate Yo and the ordinate Yc. This difference ΔY corresponds to the pitch fluctuation amount of the host vehicle. If the actually measured ordinate Yc is 280 pixels, for example, when the ordinate Yo is 308 pixels, the difference ΔY is “308−280 = + 28” pixels.

In step S120, the distance correction unit 16 calculates the distance to the pedestrian 22. That is, the distance correction unit 16 uses the coordinate obtained by adding the difference ΔY to the ordinate Yp on the captured image 21 at the lowermost end of the pedestrian 22 measured by the three-dimensional object extraction unit 15 from the own vehicle to the pedestrian 22. The distance Dp is calculated. In order to obtain the calculation formula for the distance Dp, the inverse function of the formula (3) can be considered.
Dp = Hc / (tan (Yp + ΔY−Yv) × Q) (4)
The following formula is established.

  FIG. 10 is a diagram showing an example of the distance correction result of the distance correction unit 16 based on the equation (4). Walking based on a predetermined correspondence between the ordinate Yp and the distance to the subject without correcting the ordinate Yp on the captured image 21 at the lowermost end of the pedestrian 22 measured by the three-dimensional object extraction unit 15. When the distance to the pedestrian 22 is calculated, the distance to the pedestrian 22 is 226 m when the measured value of the ordinate Yp is 252 pixels by being strongly influenced by the pitch fluctuation of the vehicle. On the other hand, when it calculates based on Formula (4), the distance Dp to the pedestrian 22 will be 68 m, and the distance to the pedestrian 22 can be calculated accurately.

The pitch fluctuation amount D is
D = Q × ΔY (5)
Can be obtained approximately. In this case, when D is a positive value, it indicates that the host vehicle is pitching upward, and when D is a negative value, it indicates that the host vehicle is pitching downward.

  Therefore, the vehicular image recognition apparatus 1 includes the three-dimensional object extraction unit 15 and the distance correction unit 16 having the above-described configuration, so that the standard size of a predetermined part of a road marking such as a pedestrian crossing is taken into account. Since the distance to the three-dimensional object such as a pedestrian is calculated, the distance to the three-dimensional object can be accurately calculated from the image captured by the monocular camera even in a situation where the three-dimensional object exists far from the host vehicle. Moreover, since the distance to the three-dimensional object is corrected by the road marking whose distance is measured by a method different from the method of measuring the distance to the three-dimensional object such as a pedestrian, As a result of reducing the measurement error of the vertical position on the captured image of the three-dimensional object, the accuracy of the distance measurement result can be increased.

  The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the above-described embodiments, and various modifications and substitutions can be made to the above-described embodiments without departing from the scope of the present invention. Can be added.

  For example, in the above-described embodiment, the distance to the road marking is calculated using a monocular camera, but the distance to the road marking may be calculated by another method. For example, the distance to the road marking may be calculated using a stereo camera.

DESCRIPTION OF SYMBOLS 1 Vehicle image recognition apparatus 11 Monocular camera 12 Road marking extraction part 13 Size detection part 14 Distance calculation part 15 Three-dimensional object extraction part 16 Distance correction part 21 Captured image 22 Pedestrian 25 Crosswalk 26 Vanishing point (infinity point)
p1-p4 candidate lines

Claims (7)

Imaging means for outputting an image around the vehicle;
Measuring means for measuring the distance to the distance measurement object by actually measuring the vertical position on the image of the distance measurement object from the image;
The measuring means includes
An image recognition apparatus for a vehicle that corrects a distance to the distance measuring object by a road marking whose distance is measured by a method different from a method of measuring the distance to the distance measuring object.   The vehicular image recognition apparatus according to claim 1, wherein the measurement unit uses a vertical position of the road marking on the image for the correction.   The measuring means is a difference between a vertical position corresponding to the distance to the road marking measured by the other method on the image and a vertical position actually measured on the image of the road marking, The vehicular image recognition apparatus according to claim 2, which is used for the correction. Yp is the ordinate representing the longitudinal position actually measured on the image to be measured, Yp is the difference, Yv is the ordinate of the vanishing point of the image, and Q is the angular resolution in the vertical direction of the imaging means. If the height to which the imaging means is attached is Hc and the distance to the distance measuring object is Dp,
The measuring means includes
Dp = Hc / (tan (Yp + ΔY−Yv) × Q)
The vehicular image recognition apparatus according to claim 3, wherein a distance to the distance measurement target is calculated based on the information.   The another method is to calculate a distance to the road marking using a standard value of the size of the predetermined portion of the road marking and a detected value of the size of the predetermined portion on the image. The image recognition apparatus for vehicles as described in any one of Claim 1 to 4.   The said another method calculates the distance to the said road marking using the detected value of the horizontal width on the said image of the predetermined site | part of the said road marking. Vehicle image recognition apparatus.   The vehicle image recognition apparatus according to any one of claims 1 to 6, wherein the road marking is a pedestrian crossing or a rhombus mark.

Images