JP6270604B2 - Object recognition device - Google Patents

Description

  The present invention relates to an object recognition apparatus for recognizing a road situation based on a captured image of a camera.

  2. Description of the Related Art Conventionally, an object recognition device that recognizes a lane mark such as a white line for traveling lane classification provided on a road from an image of a road ahead of the vehicle captured by an in-vehicle camera is known (for example, Patent Document 1). reference).

  In the object recognition device described in Patent Document 1, an image portion estimated to be an image portion of the same object is extracted between time-series images captured by a camera, and a plurality of features are included in the image portion. The movement vector of the part is calculated. Then, an object having a degree of difference between the movement vectors of a plurality of characteristic portions that is equal to or higher than a predetermined level is determined to be an object having a vertical height (such as a three-dimensional roadside sign), and is detected from the lane mark detection target. Excluded.

JP 2012-108665 A

  According to the object recognition apparatus described in Patent Document 1, it is possible to prevent the three-dimensional label from being erroneously recognized as a lane mark by excluding the image portion of the three-dimensional label.

  However, the object recognition apparatus described in Patent Document 1 needs to calculate a plurality of movement vectors for each image portion when image portion candidates for a large number of lane marks are extracted from the image captured by the camera. Therefore, there is an inconvenience that the calculation amount of the image processing increases and is not efficient.

  The present invention has been made in view of such a background, and an object thereof is to provide an object recognition apparatus capable of efficiently recognizing an object existing on a road.

  The present invention has been made to achieve the above object, and relates to an object recognition apparatus, a vehicle, and an object recognition method for recognizing an object based on a peripheral image captured by a camera.

And the target object recognition apparatus of this invention is
A candidate image part extracting unit that extracts a plurality of candidate image parts having a length equal to or longer than a predetermined length in a specific direction corresponding to a vertical direction in real space from a captured image by a camera; and
In the captured image, among the plurality of candidate image portions, the plurality of candidate image portions having left and right end positions as a leftmost candidate image portion position and a rightmost candidate image portion position. An object recognizing unit for recognizing an object based on an area excluding a range extending in the specific direction with a connecting line as a base is provided.

  According to the present invention, the plurality of candidate image portions extracted by the candidate image portion extraction unit have a certain length in the vertical direction in real space and are continuously arranged in a certain direction. Therefore, there is a high possibility that it is an image portion of a continuous roadside object (such as a pole provided at the boundary of the lane) provided on the side of the road. When objects existing on the road (lane marks laid on the road, other vehicles, pedestrians, and the like) are to be recognized, it is not necessary to perform recognition processing for a range outside the road.

  Therefore, the object recognition unit extends in the specific direction with a line connecting the plurality of candidate image portions as a base in the captured image (corresponding to a roadside range delimited by continuous roadside objects in real space). The object is recognized based on the area excluding. Accordingly, it is possible to efficiently recognize an object existing on the road by excluding the image area on the roadside without performing a heavy calculation process such as calculation of a movement vector.

  The object recognition unit may use a line connecting lower end portions in the specific direction of the plurality of candidate image portions as a line connecting the plurality of candidate image portions.

  According to this configuration, the image range corresponding to the roadside can be more accurately excluded by using the line connecting the lower end portions (corresponding to the ground contact portion in the real space) of the plurality of candidate image portions.

Further, the camera is mounted on a vehicle and images the front of the vehicle,
The certain direction is a traveling direction of the vehicle.

  According to this configuration, since the traveling direction of the vehicle can be estimated to be the road extending direction, the image portion of the continuous roadside object arranged in the road extending direction can be extracted separately from the other image portions. .

Next, the vehicle of the present invention is
A camera that captures the surroundings,
A candidate image part extraction unit that extracts a plurality of candidate image parts having a length equal to or longer than a predetermined length in a specific direction corresponding to a vertical direction in real space from a captured image by the camera; and
In the captured image, among the plurality of candidate image portions, the plurality of candidate image portions having left and right end positions as a leftmost candidate image portion position and a rightmost candidate image portion position. An object recognizing unit for recognizing an object based on an area excluding a range extending in the specific direction with a connecting line as a base is provided.

  According to the present invention, similarly to the target object recognition apparatus, the target object recognition unit has a range (in real space) that extends in the specific direction with a line connecting the plurality of candidate image parts as a base in the captured image. The object is recognized based on the area excluding the area on the side of the road separated by the continuous roadside object. Thus, it is possible to efficiently recognize an object existing on the road by excluding the image area on the side of the road without performing a calculation process such as calculation of a movement vector.

Next, the object recognition method of the present invention includes:
A candidate image extraction step of extracting a plurality of candidate image portions having a length equal to or longer than a predetermined length in a specific direction corresponding to a vertical direction in real space from a captured image by a camera; and
In the captured image, among the plurality of candidate image portions, the plurality of candidate image portions having left and right end positions as a leftmost candidate image portion position and a rightmost candidate image portion position. And an object recognition step of recognizing an object based on a region excluding a range extending in the specific direction with a connecting line as a base.

  According to the present invention, the plurality of candidate image portions extracted by the candidate image step have a certain length in the vertical direction in the real space, and are images of objects continuously arranged in a certain direction. Since it is a portion, there is a high possibility that it is an image portion of a continuous roadside object (such as a pole provided at the boundary of a lane) provided on the side of the road. When objects existing on the road (lane marks laid on the road, other vehicles, pedestrians, and the like) are to be recognized, it is not necessary to perform recognition processing for a range outside the road.

  Therefore, in the captured image, in the captured image, a range extending in the specific direction with a line connecting the plurality of candidate image portions as a base (corresponding to a roadside range delimited by continuous roadside objects in real space). The object is recognized based on the area excluding. Accordingly, it is possible to efficiently recognize an object existing on the road by excluding the image area on the roadside without performing a heavy calculation process such as calculation of a movement vector.

The block diagram of a target object recognition apparatus. The flowchart of a target object recognition process. Explanatory drawing of a captured image and an edge image. Explanatory drawing of the process which excludes the image range of a roadside.

  An embodiment of an object recognition device of the present invention will be described with reference to FIGS.

  Referring to FIG. 1, an object recognition device 10 is mounted on a vehicle 1 (corresponding to a vehicle of the present invention) including a camera 2 (color camera), a speaker 5, a display 6, and a steering mechanism 7. Yes.

  The object recognition apparatus 10 is an electronic unit configured by a CPU, a memory, various interface circuits, and the like (not shown), and a captured image is obtained by executing a program for object recognition and steering control held in the memory by the CPU. It functions as an acquisition unit 11, an edge image generation unit 12, a candidate image part extraction unit 13, an object recognition unit 14, and a steering control unit 15. Further, the object recognition method of the present invention is implemented by the object recognition apparatus 10.

  Hereinafter, processing for recognizing an object (lane mark, other vehicle, etc.) existing on the road by the object recognition device 10 will be described according to the flowchart shown in FIG. The target object recognition apparatus 10 performs the process according to the flowchart shown in FIG. 2 at every predetermined control period, and recognizes the target object existing on the road on which the vehicle 1 is traveling.

  STEP 1 in FIG. 2 is processing by the captured image acquisition unit 11. The captured image acquisition unit 11 inputs a video signal around the vehicle 1 output from the camera 2 (front), and demosaices the color components (R value, G value, B value) of the video signal. A color captured image 21 having R value, G value, and B value as pixel data is acquired. Then, data of the captured image 21 in front of the vehicle 1 is held in the image memory 20.

  The subsequent STEP 2 is processing by the edge image generation unit 12. The edge image generation unit 12 performs a process of converting the color component of each pixel of the captured image 21 into luminance, and generates a grayscale image (multi-valued image). Then, the edge image generation unit 12 is a pixel in which the luminance difference (luminance change amount) from the edge point (the surrounding pixel (image portion)) is a predetermined value or more from the grayscale image. The luminance changes from dark to bright. Edge image 22 (see FIG. 1) is generated by extracting positive edge points and negative edge points whose luminance changes from light to dark.

  Note that when the camera 2 is a monochrome camera, a grayscale captured image is obtained from the luminance of each pixel, and thus the above-described processing for generating a grayscale image from a color captured image is not necessary.

  Here, Im1 shown in FIG. 3A is an example of a grayscale image generated from the captured image 21, and the rightmost continuous line lane laid on the road 30 on which the vehicle 1 (own vehicle) is traveling. The image portion 31a of the mark, the image portion 32a of the continuous line lane mark at the left end, and the image portions 33a to 35a of the broken line lane mark at the center are included. Also, Im1 includes image portions 41a to 43a of continuous roadside objects (lane separation poles and the like) installed along the side of the road 30.

  Im2 illustrated in FIG. 3B is an edge image generated by performing edge extraction processing on the grayscale image Im1. The edge image Im2 corresponds to an image portion 31b corresponding to the image portion 31a of Im1, an image portion 32b corresponding to the image portion 32a, image portions 33b to 35b corresponding to the image portions 33a to 35a, and image portions 41a to 43a. Image portions 41b to 43b are extracted.

  Subsequent STEP 3 to STEP 4 are processes performed by the candidate image portion extraction unit 13. The candidate image portion extraction unit 13 labels the edge image Im2 with a label for a group of edge points (hereinafter referred to as edge portions) and the adjacent edge portions as image portions of the same object. Perform clustering to associate as there is.

  In STEP 4, the candidate image portion extraction unit 13 has a length equal to or longer than a predetermined length in the y direction (the direction corresponding to the vertical direction in the real space, which corresponds to the specific direction of the present invention) among the clustered image portions. And image portions arranged in a certain direction (for example, the traveling direction of the vehicle 1) are extracted as candidate image portions. In the example of FIG. 3B, image portions 41b to 43b (image portions of continuous roadside objects) arranged in the direction D1 are extracted as a plurality of candidate image portions.

  In addition, the process process by the candidate image part extraction part 13 is corresponded to the candidate image extraction process in the target object recognition method of this invention.

  Subsequent STEP5 to STEP7 are processes by the object recognition unit 14. In addition, the process process by the target object recognition part 14 is corresponded to the target object recognition process in the target object recognition method of this invention.

  In STEP 5, as shown in FIG. 4, the object recognition unit 14 obtains a line 60 (approximate curve obtained by the least square method or the like) connecting the lower end portions of the candidate image portions 41b to 43b. Then, a region of interest in the captured image is set by excluding a range 65 (indicated by hatching in FIG. 4) extending in the y direction with the line 60 as a base. Thereby, it is possible to efficiently recognize the object based on the attention area excluding the roadside image range including the image portion of the continuous roadside object.

  In subsequent STEP 6, the object recognition unit 14 extracts lane mark image portion candidates (31b, 32b, 33b to 35b in FIG. 4) from the region of interest, and for each lane mark image portion candidate, The back projection transformation from the camera coordinates to the real space coordinates is performed, and the position in the real space corresponding to the candidate image portion of each lane mark is calculated. In the next STEP 7, the object recognition unit 14 recognizes the boundary of the lane based on the position in the real space corresponding to the candidate lane mark image portion.

  STEP 8 is processing by the steering control unit 15. The steering control unit 15 assists the driving operation of the driver of the vehicle 1 by operating the steering mechanism 7 so that the vehicle 1 travels near the center position of the lane or suppresses deviation from the lane. In this case, the steering control unit 15 alerts the driver by sound output from the speaker 5 and display on the display 6 as necessary.

[Deformation]
In addition to the lane mark laid on the road, the object to be recognized by the object recognition unit 14 may be another vehicle such as a preceding vehicle or an object existing on the road such as a road sign. Also in this case, an object existing on the road can be recognized efficiently by excluding the image area on the side of the road.

  In the above embodiment, as illustrated in FIG. 4, the exclusion range 65 is determined using a line connecting the lower end portions of the candidate image portions 41 b to 43 b, but other than the lower end portions (upper end portions) of the candidate image portions 41 b to 43 b. , Intermediate portions, etc.) may be used.

  DESCRIPTION OF SYMBOLS 1 ... Vehicle (own vehicle), 2 ... Camera, 7 ... Steering mechanism, 10 ... Object recognition apparatus, 11 ... Captured image acquisition part, 12 ... Edge image generation part, 13 ... Candidate image part extraction part, 14 ... Object Recognizing unit, 15 ... steering control unit, 20 ... image memory, 21 ... captured image, 22 ... edge image.

Claims (5)

A candidate image part extracting unit that extracts a plurality of candidate image parts having a length equal to or longer than a predetermined length in a specific direction corresponding to a vertical direction in real space from a captured image by a camera; and
In the captured image, among the plurality of candidate image portions, the plurality of candidate image portions having left and right end positions as a leftmost candidate image portion position and a rightmost candidate image portion position. An object recognition device comprising: an object recognition unit for recognizing an object based on an area excluding a range extending in the specific direction with a connecting line as a base. The object recognition apparatus according to claim 1,
The target object recognition unit uses a line connecting lower end portions in the specific direction of the plurality of candidate image portions as a line connecting the plurality of candidate image portions. In the object recognition device according to claim 1 or 2,
The camera is mounted on a vehicle and images the front of the vehicle,
2. The object recognition apparatus according to claim 1, wherein the predetermined direction is a traveling direction of the vehicle. A camera that captures the surroundings,
A candidate image part extraction unit that extracts a plurality of candidate image parts having a length equal to or longer than a predetermined length in a specific direction corresponding to a vertical direction in real space from a captured image by the camera; and
In the captured image, among the plurality of candidate image portions, the plurality of candidate image portions having left and right end positions as a leftmost candidate image portion position and a rightmost candidate image portion position. A vehicle comprising: an object recognition unit that recognizes an object based on an area excluding a range extending in the specific direction with a connecting line as a base. A candidate image extraction step of extracting a plurality of candidate image portions having a length equal to or longer than a predetermined length in a specific direction corresponding to a vertical direction in real space from a captured image by a camera; and
In the captured image, among the plurality of candidate image portions, the plurality of candidate image portions having left and right end positions as a leftmost candidate image portion position and a rightmost candidate image portion position. And a target object recognition step for recognizing a target object based on a region excluding a range extending in the specific direction with a connecting line as a base.