JP2018073275A - Image recognition device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To appropriately recognize information required in driving assistance or automatic driving while reducing a calculation load.SOLUTION: An image recognition device 1 of a vehicle provided with a camera 9 for photographing a vehicle exterior includes: a three-dimensional map data 2 in which three-dimensional position information of a stationary object present in the vehicle exterior is set; a detection part 4 detecting the stationary object as an object subjected to image recognition processing based on relative positions of the vehicle and the stationary object and a distance between the vehicle and the stationary object; a setting part 5, from a photographing range photographed by the camera 9, narrowing down a range, in which the object is included, based on a current position of the vehicle and the three-dimensional map data 2 and then setting the aforementioned range as a recognition range; and a processing part 6 performing the image recognition processing only within the recognition range.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image recognition apparatus for a vehicle provided with a camera that images the outside of the vehicle (hereinafter referred to as “outside the vehicle”).

  Conventionally, a vehicle equipped with a driving support system for reducing the burden of driving operation of a driver has been put into practical use. In recent years, development of an automatic traveling system for automatically traveling a vehicle instead of manual driving by a driver has been promoted. A vehicle equipped with a driving support system or an automatic driving system is equipped with a camera for acquiring information outside the vehicle. Analysis processing (image recognition processing) is performed on the image captured by the camera, thereby detecting an object existing outside the vehicle (see, for example, Patent Documents 1 and 2). In such a vehicle, it is important to grasp road conditions, obstacles, and the like in real time instead of the driver. Therefore, it is desirable that the mounted camera can capture a wide range.

JP 2013-218571 A JP 2012-210864 A

  However, in the conventional image recognition processing, since the entire range captured by the camera is analyzed, if the imaging range of the camera is expanded, the computational load on the computer increases. On the other hand, if the imaging range is narrow, the computer may not be able to recognize information required for driving assistance or automatic driving.

  The present case has been devised in view of such problems, and provides an image recognition apparatus that can appropriately recognize information required for driving assistance and automatic driving while reducing the calculation load. One of the purposes is to provide The present invention is not limited to this purpose, and is a function and effect derived from each configuration shown in the embodiment for carrying out the invention described later, and has another function and effect that cannot be obtained by conventional techniques. is there.

  (1) An image recognition apparatus disclosed herein is an image recognition apparatus for a vehicle provided with a camera that captures the outside of the vehicle, and three-dimensional map data in which three-dimensional position information of a stationary object existing outside the vehicle is set. A detection unit that detects the stationary object as an object of image recognition processing based on a relative position between the vehicle and the stationary object and a distance between the vehicle and the stationary object; a current position of the vehicle; and Based on three-dimensional map data, a setting unit that narrows down a range including the object from an imaging range captured by the camera and sets the recognition range, and performs the image recognition processing only within the recognition range And a processing unit.

(2) An estimation unit that estimates the current position using a SLAM technique is provided, and the detection unit calculates the relative position and the distance using the estimated current position and the position information of the stationary object. It is preferable to calculate.
(3) The setting unit calculates a range including the object based on the estimated current position, the position information of the object, an attachment angle of the camera, and an angle of view of the camera. Is preferred.
(4) It is preferable that the setting unit calculates a minimum value of a range including the object as a minimum range, and sets a range obtained by extending the minimum range by a predetermined ratio as the recognition range.

  Since the range (recognition range) for performing the image recognition processing is set by narrowing the range including the target object, it is possible to appropriately recognize the target object of the image recognition processing while reducing the calculation load.

1 is a block diagram illustrating an image recognition device according to an embodiment. It is the schematic diagram which illustrated the case where a target object is contained in the imaging range imaged with a front camera. 3 is a flowchart illustrating the contents of control performed by the image recognition apparatus in FIG. 1.

  An image recognition apparatus as an embodiment will be described with reference to the drawings. The embodiment described below is merely an example, and there is no intention of excluding various modifications and technical applications that are not explicitly described in the following embodiment. Each configuration of the present embodiment can be implemented with various modifications without departing from the spirit thereof. Further, they can be selected as necessary, or can be appropriately combined.

[1. Constitution]
As shown in FIG. 1, the image recognition device 1 is an electronic control device (computer) mounted on a vehicle (also referred to as “own vehicle”), and is taken outside the vehicle imaged by cameras 9A to 9D mounted on the vehicle. It recognizes images. The vehicle of the present embodiment is provided with a front camera 9A, a rear camera 9B, a left side camera 9C, and a right side camera 9D that capture images of the vehicle front, vehicle rear, vehicle left side, and vehicle right side, respectively. Hereinafter, these are collectively referred to as a camera 9.

  The camera 9 is an imaging device that captures images (still images and moving images) outside the vehicle. For example, the camera 9 is a video camera with a built-in imaging device (CCD (Charge-Coupled Device), CMOS (Complementary Metal-Oxide-Semiconductor), etc.). is there. Various objects existing outside the vehicle are detected by performing analysis processing (image recognition processing) on the image captured by the camera 9. The imaging range captured by each camera 9A to 9D is determined by the arrangement of each camera 9A to 9D, the mounting angle (orientation) with respect to the vehicle body, the angle of view of each camera 9A to 9D, and the like.

  The image recognition apparatus 1 includes a processor (central processing unit), a memory (main memory, main storage device), an auxiliary storage device, an interface device, a recording medium drive, and the like. A camera 9 is connected to the input side of the image recognition apparatus 1. Further, for example, an electronic control device (not shown) for integrally controlling the vehicle is connected to the output side of the image recognition device 1. The result recognized (detected) by the image recognition device 1 is transmitted to this electronic control device, and controls (driving support), for example, steering device, braking device, driving device, display, speaker, direction indicator, etc. (all not shown). Control and automatic operation control).

  The image recognition apparatus 1 includes 3D map data 2 (three-dimensional map data), an estimation unit 3, a detection unit 4, a setting unit 5, and a processing unit 6. Elements 3 to 6 excluding the 3D map data 2 are classified functions of the image recognition device 1 for convenience, and each element may be described as an independent program. May be described as a composite program. These programs are stored in a memory or an auxiliary storage device and executed by a processor. Alternatively, these programs are recorded on a recording medium (removable medium) such as an optical disk or a semiconductor memory, and are read into the memory via a recording medium drive and executed.

The 3D map data 2 is detailed map data in which three-dimensional position information (geographic coordinate information) of a stationary object existing outside the vehicle is set, and is stored in a memory or an auxiliary storage device. Examples of stationary objects include road signs, road markings, traffic lights, utility poles, electric bulletin boards, and buildings. The position information is information representing the position of a stationary object in geographical coordinates [a combination of (longitude, latitude, height)] and is given for each stationary object. For example, when one geographic coordinate (X, Y, Z) represents a pixel of 1 [cm ³ ], a plurality of sets of geographic coordinates are given as position information to one stationary object.

  The estimation unit 3 estimates the current position of the vehicle using SLAM (Simultaneous Localization and Mapping) technology. The SLAM technique is a conventionally well-known technique that simultaneously performs self-position estimation and environmental map creation. The estimation unit 3 creates a map by scanning the surroundings using, for example, a sensor such as a laser scanner provided in the vehicle, and simultaneously estimates where the host vehicle exists on the map. That is, the estimation unit 3 also estimates (specifies) the traveling lane in which the vehicle is traveling.

  The estimation unit 3 of the present embodiment represents the estimated position of the own vehicle (such as the position of the center of gravity of the own vehicle or the position of the center of the front end of the vehicle) in geographical coordinates. In addition, by using the SLAM technology, the current position of the vehicle is estimated with higher accuracy than in the case of using GPS. The estimation unit 3 transmits information on the estimated current position of the vehicle (geographic coordinates) to the detection unit 4 and the setting unit 5 together with information on the traveling lane (hereinafter referred to as “lane information”).

  The detection unit 4 detects a stationary object as an object for image recognition processing based on the relative position between the vehicle and the stationary object and the distance E between the vehicle and the stationary object. In other words, the detection unit 4 detects, as a “target object”, an object that is highly necessary to perform image recognition processing from among all stationary objects that exist outside the vehicle. Note that the detection unit 4 may simultaneously detect a plurality of different objects.

The detection unit 4 of the present embodiment uses the current position information and lane information of the vehicle estimated by the estimation unit 3 and the position information of the stationary object included in the 3D map data 2 to determine the position of the stationary object ( Relative position) and the distance E between the vehicle and the stationary object. For example, a stationary object (existing on the traveling lane of the host vehicle) and having a distance E to the host vehicle not less than a first predetermined value E _{1 and not} more than a second predetermined value E ₂ (where E ₁ <E ₂ ) ( A stationary object within a predetermined range) is detected as an object. While the vehicle travels, the distance E between the vehicle and the stationary object changes every moment, so that the detection unit 4 can detect the same stationary object (for example, a traffic light) as a target object many times. However, the distance E between the stationary object if it becomes a less than a predetermined value E ₁ excludes the stationary object from the object.

  The detection unit 4 of the present embodiment detects an object in consideration of the type of stationary object. Since the 3D map data 2 is stored in the image recognition device 1, the calculation load is reduced by using only a stationary object having information other than position information as an object for image recognition processing. Examples of stationary objects that have information other than position information include objects whose display contents change, such as traffic lights and electronic bulletin boards, and objects that may be degraded such as road markings (for example, white lines that have disappeared) Is mentioned. Note that the priority order of the objects detected by the detection unit 4 may be set in advance. The detection unit 4 transmits the detected object to the setting unit 5.

  The setting unit 5 includes a range in which an object detected by the detection unit 4 is included in an imaging range captured by the camera 9 based on the current position of the vehicle estimated by the estimation unit 3 and the 3D map data 2. Is set as a recognition range. The recognition range set here is a range to which image recognition processing is performed in the processing unit 6. That is, the image recognition apparatus 1 according to the present embodiment performs image recognition processing only on a part of the imaging range (recognition range), not on the entire imaging range captured by the camera 9. As a result, the calculation load is greatly reduced. When a plurality of objects are detected at the same time, a plurality of recognition ranges are set in the imaging range captured by the camera 9.

  The setting unit 5 of the present embodiment is configured so that the object is selected from the imaging range based on the current vehicle position estimated by the estimation unit 3, the position information of the object, the attachment angle of the camera 9, and the angle of view of the camera 9. Calculate the included range. For example, the setting unit 5 calculates the imaging range for each of the cameras 9A to 9D using the current position of the vehicle estimated by the estimation unit 3, the attachment angle and the angle of view of each camera 9A to 9D, and the calculated imaging range The position where the object appears is calculated using the position information of the object. Note that the position information of the object is acquired from the 3D map data 2, and the attachment angle and the angle of view of each camera 9 </ b> A to 9 </ b> D are stored in advance in the image recognition device 1. The range in which the object is included is calculated as, for example, a rectangular shape surrounding the entire object.

  Furthermore, the setting unit 5 of the present embodiment calculates the minimum value of the range including the target object (a range corresponding to the outline of the target object) as the minimum range, and recognizes a range obtained by extending the minimum range by a predetermined ratio. Set as a range. As a result, a recognition range is set in consideration of an estimation error by the estimation unit 3 and an error in the mounting angle of the camera 9 (deviation from a value stored in advance). The predetermined ratio may be set in advance, or may be a variable value that changes according to the distance between the vehicle and the object. For example, the predetermined ratio may be reduced as the distance between the vehicle and the object increases, and may be increased as the distance between the vehicle and the object decreases.

  Here, an example of the imaging range imaged by the front camera 9A is shown in FIG. Note that the scenery in front of the vehicle shown in FIG. 2 is a scenery seen from the driver, and does not indicate the result of the image recognition device 1 recognizing the image captured by the front camera 9A. The imaging range includes, for example, stationary objects such as the traveling lane 10, the facing lane 11, the traffic signal on the traveling lane side 12, the traffic signal 13 on the facing lane side, the pedestrian crossing 14, the roadside tree 15, and the building 16. In the 3D map data 2, position information of these stationary objects is set. Note that the imaging range may include pedestrians and other vehicles, but moving objects are excluded from the objects of the image recognition apparatus 1 according to the present embodiment because position information is not set in the 3D map data 2. The

  When the current position of the host vehicle and the travel lane 10 are estimated by the estimation unit 3, the traffic signal 12 that exists on the travel lane 10 and whose distance from the host vehicle is within a predetermined range is detected by the detection unit 4, for example. Detected as an object. In this case, the setting unit 5 calculates the minimum value of the range including the object (the traffic signal 12) as the minimum range A as indicated by the one-dot chain line in the figure. Furthermore, a range (broken line in the figure) obtained by extending the minimum range A by a predetermined ratio is set as the recognition range B.

  The processing unit 6 performs image recognition processing only on the recognition range set by the setting unit 5. The specific method of the image recognition processing is not particularly limited, and a generally known method (for example, filtering processing and binarization processing on an image within the recognition range, edge detection processing, color analysis processing, A method of performing pattern matching processing) is employed. The processing unit 6 may perform image recognition processing on each recognition range simultaneously when a plurality of recognition ranges are set simultaneously by the setting unit 5 (for example, in front of the vehicle and on the left side of the vehicle). Processing units 6A to 6D corresponding to 9A to 9D may be provided in the image recognition apparatus 1 to share the processing.

[2. flowchart]
FIG. 3 is a flowchart showing an example of the control content executed by the image recognition apparatus 1 described above. This flowchart is executed at a predetermined calculation cycle when the vehicle is powered on, for example.
As shown in FIG. 3, first, the current position of the vehicle and the travel lane are estimated by the estimation unit 3 (step A1). Next, the relative position and distance are calculated for each stationary object by the detection unit 4 (step A2), and the target is detected from the stationary object (step A3).

  When an object is detected, a minimum range is calculated by the setting unit 5 (step A4), and a recognition range obtained by extending the minimum range by a predetermined ratio is set (step A5). Then, the image recognition process is performed only on the recognition range by the processing unit 6 (step A6), the result is output (step A7), and this flow is returned.

[3. effect]
(1) In the image recognition apparatus 1 described above, the image recognition process is not performed on the entire imaging range captured by the camera 9, but the range where the image recognition process is performed (recognition range) is set to the range including the target object. squeeze. For this reason, compared with the case where it processes with respect to the whole imaging range, the calculation load of the process part 6 (image recognition apparatus 1) can be reduced significantly, and the target of an image recognition process can be recognized appropriately. Can do.

(2) In addition, the current position of the vehicle is estimated using SLAM technology, and the relative position and distance between the vehicle and the stationary object are calculated from the current position and the position information (geographic coordinates) of the stationary object. Since it detects, a target object can be detected accurately.
(3) Further, a range including the object is calculated based on the estimated current position of the vehicle, the position information of the object included in the 3D map data 2, and the attachment angle and the angle of view of the camera 9. Therefore, the recognition range can be set with high accuracy.

  (4) Further, by calculating the minimum value of the range including the object as the minimum range and setting a range obtained by expanding the minimum range by a predetermined ratio as the recognition range, the estimation error, the camera 9 mounting error, the vehicle Even if the orientation of the camera 9 is affected by the behavior of the object, since the object can be included in the recognition range, the image recognition process can be reliably performed on the object.

[4. Others]
Although the embodiment and modification examples of the present invention have been described above, the present invention is not limited to the above-described embodiment and the like, and various modifications can be made without departing from the spirit of the present invention. For example, the setting unit 5 may calculate the range including the object with a predetermined margin and set the range as the recognition range. In other words, instead of calculating the minimum range and expanding it by a predetermined ratio, it may be configured to calculate a larger range including the object in advance. In addition, it is good also as a structure which improves estimation precision and sets a minimum range to a recognition range. Further, instead of using the SLAM technology, the estimation unit 3 may estimate the current position using GPS. Note that the number and arrangement of the cameras 9 are not particularly limited.

1 Image recognition device 2 3D map data (3D map data)
DESCRIPTION OF SYMBOLS 3 Estimation part 4 Detection part 5 Setting part 6 Processing part 9 Camera

Claims (4)

An image recognition apparatus for a vehicle provided with a camera for imaging outside the vehicle,
3D map data in which 3D position information of stationary objects existing outside the vehicle is set,
A detection unit for detecting the stationary object as an object of image recognition processing based on a relative position between the vehicle and the stationary object and a distance between the vehicle and the stationary object;
Based on the current position of the vehicle and the three-dimensional map data, a setting unit that narrows down a range including the target object from an imaging range captured by the camera, and sets it as a recognition range;
An image recognition apparatus comprising: a processing unit that performs the image recognition process only within the recognition range. An estimation unit for estimating the current position using SLAM technology;
The image recognition apparatus according to claim 1, wherein the detection unit calculates the relative position and the distance using the estimated current position and the position information of the stationary object. The setting unit calculates a range including the object based on the estimated current position, the position information of the object, an attachment angle of the camera, and an angle of view of the camera. The image recognition apparatus according to claim 2. The setting unit calculates a minimum value of a range including the target object as a minimum range, and sets a range obtained by extending the minimum range by a predetermined ratio as the recognition range. 4. The image recognition device according to any one of items 3.

Images