JP2017138660A - Object detection method, object detection device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To detect object on a travelling route of a vehicle with high accuracy.SOLUTION: An object detection method detecting objects located on a travelling route of a vehicle having an on-vehicle camera mounted from an image shot by the vehicle includes: an on-vehicle image acquisition step of acquiring an on-vehicle image shot by the on-vehicle camera of the vehicle; a behavior information acquisition step of acquiring behavior information on the vehicle subsequent to imaging of the on-vehicle image; a trajectory calculation step of obtaining a movement trajectory of the vehicle on the basis of the behavior information; an area calculation step of obtaining an area corresponding to the movement trajectory in the on-vehicle image on the basis of the movement trajectory; an object detection step of detecting an object targeted for detection from the on-vehicle image; and a calculation step of, as to the detected object, calculating an amount of overlapping between the detected object and the area corresponding to the movement trajectory in the on-vehicle image. It is preferable to further include a judgement step of judging that the object with the amount of overlapping equal to or more than a prescribed ratio is the object located on the travelling route of the vehicle.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a technique for detecting an object from an image, and more particularly to a technique for detecting an object on a travel route of the vehicle from an image taken by the vehicle.

A technique for detecting an object, particularly an obstacle, from an image taken by an in-vehicle camera has been proposed. Patent Document 1 discloses a technique for detecting an object in the traveling direction of a vehicle using a camera or a radar. Patent Document 1 particularly estimates the presence / absence of a lane change from the planned travel route and the driver's line of sight, and changes the shooting direction of the camera and the detection direction of the radar when it is predicted that the lane change will be performed.

In Patent Document 2, when detecting an object using both a camera and a radar, the detection is performed using both the camera and the radar within the travel path range of the vehicle, and only the radar is used outside the travel path range. And performing detection.

Patent Document 3 discloses that white line detection is performed to detect the road shape and the position of the host vehicle on the road, and using this and the captured image, the degree of risk along the travel locus is calculated.

JP 2005-231450 JP JP 2012-089114 A Japanese Unexamined Patent Publication No. 2006-154967

  The vehicle speed and the steering amount change from moment to moment depending on the driving operation of the driver, and it is generally difficult to predict the movement locus of the vehicle. Therefore, the methods of Patent Documents 1 and 2 using the predicted movement trajectory have low accuracy for detecting an object on the travel route.

  Even if white line detection is used, the accuracy is not necessarily improved. First, the accuracy of white line detection itself is not so high. For example, accurate detection cannot be performed at a place where the white line is about to disappear, and an erroneous detection occurs because an object having the same pattern as the white line exists in addition to the white line. Furthermore, in some countries, the conventional white line is cut and the white line is drawn again, which also causes false detection. Furthermore, in some countries, the lane is expressed by dots instead of white lines, which also causes a decrease in detection accuracy. Further, even if the white line can be detected correctly, the vehicle does not always travel along the lane, so that the movement trajectory of the vehicle cannot be accurately predicted.

  As described above, it is difficult for the conventional technology to accurately detect an object on the travel route of the vehicle. Accordingly, an object of the present invention is to accurately detect an object on a travel route of a vehicle.

In order to achieve the above object, in the first aspect of the present invention, an object detected in a vehicle-mounted image is obtained by obtaining a vehicle movement trajectory (traveling route) using vehicle behavior information after the vehicle-mounted image is captured. Is detected on the movement locus, thereby detecting an object located on the movement locus of the vehicle. In this specification, the detection of an object located on the movement locus of the vehicle means that the object is placed on the movement locus of the vehicle in addition to determining whether or not the object is located on the movement locus of the vehicle. It also includes determining the probability (probability) of being located.

More specifically, the first aspect of the present invention includes:
An object detection method for detecting an object located on a travel route of a vehicle from an image taken by a vehicle equipped with an in-vehicle camera,
An in-vehicle image acquisition step of acquiring an in-vehicle image captured by the in-vehicle camera of the vehicle;
Behavior information acquisition step for acquiring behavior information of the vehicle after taking the in-vehicle image;
A trajectory calculating step for obtaining a movement trajectory of the vehicle based on the behavior information;
An area calculation step for obtaining an area corresponding to the movement locus in the in-vehicle image based on the movement locus;
An object detection step of detecting an object to be detected from the in-vehicle image;
A calculation step of calculating an overlap amount of the object detected in the object detection step with an area corresponding to the movement locus in the vehicle-mounted image;
It is characterized by including.

  In this way, since the movement locus of the vehicle is obtained based on the actual behavior information of the vehicle, the movement locus can be obtained correctly. Therefore, an object on the moving locus of the vehicle can be detected with high accuracy.

  The in-vehicle image may be a still image or a moving image. The vehicle behavior information may be acquired after the in-vehicle image is captured, or may be continuously acquired before the capturing. Typically, while the vehicle is traveling, the in-vehicle image (moving image) is captured and behavior information is acquired in synchronization, and the information processing apparatus acquires the synchronized in-vehicle image and behavior information and performs the above processing. Good.

The behavior information includes one or a plurality of pieces of information that can determine the movement trajectory of the vehicle. Examples of behavior information include, for example, vehicle position, speed, acceleration, yaw rate, gradient, and direction. In the trajectory calculation step, the movement trajectory of the vehicle in the world coordinate system (for example, a geographic coordinate system represented by latitude, longitude, and altitude) or in the vehicle coordinate system (coordinate system based on the vehicle) based on the behavior information. It is good to ask for. In addition, it is also preferable to obtain | require a movement locus | trajectory using information other than behavior information, for example, map information etc. Further, the movement trajectory is preferably a trajectory in a three-dimensional space, but may be obtained as a trajectory in a two-dimensional plane.

  In this aspect, the movement trajectory can be defined as a trajectory accompanying the movement of the vehicle, including a line corresponding to the vehicle width, including a point corresponding to the position of the vehicle. The “point corresponding to the position of the vehicle” is preferably the center of the vehicle in the width direction and the point of the ground surface position. Alternatively, the position of the vehicle may be represented by a plane or a solid, and the movement trajectory of the vehicle may be defined as the trajectory of the plane or the solid.

  In the region calculation step, the region corresponding to the movement locus in the vehicle-mounted image is a region where the movement locus exists (captured) in the vehicle-mounted image when it is assumed that the movement locus actually exists. The calculation of the area is obtained by projecting (coordinate transforming) the movement locus of the vehicle onto the imaging surface of the in-vehicle camera using the installation position and the imaging direction of the in-vehicle camera.

Any existing method can be used as the object detection method in the object detection step. For example, an object to be detected can be detected using a discriminator previously learned by machine learning processing. When the in-vehicle image is a moving image, the object may be detected using not only one image but also a plurality of images including the front and back of the image. Moreover, you may detect an object using a distance image other than a vehicle-mounted image.

  In the calculation step, the overlapping amount of the detected object and the movement locus in the in-vehicle image is calculated. An object having a larger amount of overlap has a higher probability of being an object located on the movement locus of the vehicle. Therefore, this overlap amount can be said to be a score (certainty factor) representing the probability that the detected object is located on the movement locus of the vehicle.

  According to the present invention, the overlap amount (or score obtained from the overlap amount) for each object detected in the object detection step can be used as an output result.

  In the present invention, it is also preferable to include a determination step of determining whether or not the detected object is located on the travel route of the vehicle based on the overlap amount. In the determination step, it is determined whether or not the detected object is an object located on the travel route of the vehicle, depending on how much the detected object overlaps the corresponding region of the movement locus in the in-vehicle image. The Specifically, for example, in the object detection step, an area where the object is present in the in-vehicle image is obtained, and in the determination step, the ratio (area ratio) of the area corresponding to the movement locus in the area where the object exists is a predetermined ratio. If it is above, it can be judged that a detection object is an object on the running route of vehicles. Alternatively, in the object detection step, a rectangular region where the object is present in the vehicle-mounted image is obtained, and in the determination step, if the proportion of the portion in the region corresponding to the movement trajectory in the lower side of the rectangle is greater than or equal to a predetermined proportion It can be determined that the detected object is an object on the travel route of the vehicle.

  In the object detection method according to this aspect, an image in which an area corresponding to the movement locus obtained in the area calculation step and a display for specifying the object detected in the object detection step are superimposed on the in-vehicle image. It is also preferable that the method further includes an image generation step of outputting data. At this time, it is also preferable to generate the display in a mode corresponding to the amount of overlap. For example, it is also preferable to generate the display so that an object (an object whose overlap amount is a predetermined amount or more) located on the travel route of the vehicle and other objects can be identified.

  In addition, this invention can also be grasped | ascertained as an object detection method which performs at least one part of the said process. The present invention can be understood as an object detection device including means for executing at least a part of the above processing. The present invention can also be understood as a computer program for causing a computer to execute this method, or a computer-readable storage medium in which this computer program is stored non-temporarily. Each of the above means and processes can be combined with each other as much as possible to constitute the present invention.

  According to the present invention, an object on a travel route of a vehicle can be detected with high accuracy.

FIG. 1 is a functional block diagram of an object detection device and a vehicle according to an embodiment. FIG. 2 is a flowchart showing an overall outline of the object detection process according to the embodiment. 3 (A) -3 (C) are diagrams illustrating a process for calculating the trajectory of the vehicle. FIG. 4 is a diagram illustrating an example of an in-vehicle image and a detected object. FIGS. 5 (A) and 5 (B) are diagrams for explaining installation information of the in-vehicle camera. 6 (A) and 6 (B) are diagrams for explaining the projection processing of the vehicle trajectory onto the in-vehicle image. FIGS. 7 (A) and 7 (B) are diagrams illustrating a method for determining whether or not the detected object is located on the travel route of the vehicle. FIG. 8 is a diagram for explaining the result of the object detection process.

  In the present embodiment, the vehicle-mounted image and behavior information acquired by the vehicle are transmitted to the object detection device, and the object detection device performs processing to detect an object located on the travel route of the vehicle.

<Configuration>
FIG. 1 is a diagram illustrating a configuration of a vehicle 200 that captures an in-vehicle image and an object detection device 100 that detects an object on a travel route of the vehicle 200 from the in-vehicle image. The vehicle 200 in this embodiment includes an ECU (Electronic Control Unit) (not shown), an in-vehicle camera 202, various sensors 204, and a storage device 206. The ECU includes a microprocessor, and controls processing in the vehicle 200 when the microprocessor executes a program. The in-vehicle camera 202 is an imaging device that captures the outside of the vehicle, particularly the front of the vehicle. The sensor 204 is for acquiring a control state of a vehicle such as a GNSS (Global Satellite Positioning System) device such as a GPS device, a speed sensor, an acceleration sensor, a geomagnetic (orientation) sensor, a steering angle sensor, and an accelerator opening sensor. Various sensors are included.

  During traveling, the vehicle 200 captures an in-vehicle image 208 by the in-vehicle camera 202 and acquires behavior information (sensor information) 210 by the sensor 204 and accumulates it in the storage device 206. At this time, the acquisition time of the in-vehicle image 208 and the behavior information 210 is stored in association with each other using the same clock, or the in-vehicle image 208 and the behavior information 210 acquired at the same time are stored in association with each other. As a result, the in-vehicle image 208 and the behavior information 210 acquired at the same time can be grasped.

The storage device 206 also stores vehicle setting information 212. The vehicle setting information 212 includes, for example, the size (width, length, height) of the vehicle, the installation position of the in-vehicle camera 202, and the shooting direction.

The vehicle 200 transmits the in-vehicle image 208 and the behavior information 210 stored in the storage device 206 to the object detection device 100 together with the vehicle setting information 212 at an appropriate timing.

The object detection apparatus 100 in the present embodiment is realized by a computer (information processing apparatus). The hardware configuration of the object detection device 100 includes a processing unit (microprocessor) such as a CPU (Central Processing Unit), a main storage device such as a RAM (Random Access Memory), an HDD (Hard Disk Drive), and an SSD (Solid State Drive). A storage device 102 including an auxiliary storage device such as an input device such as a keyboard, a mouse, and a touch panel, an output device such as a display and a speaker, and a wireless communication device. When the arithmetic device loads and executes the program stored in the auxiliary storage device to the main storage device, the object detection device 100 provides each function shown in FIG.

The storage device 102 stores an in-vehicle image 104, behavior information 106, and vehicle setting information 108 transmitted from the vehicle 200. The object detection device 100 functions as a trajectory calculation unit 110, a travel region projection unit 112, an object detection unit 114, an on-route object determination unit 116, and an output image generation unit 118 by the CPU executing a program. The processing performed by each of these functional units will be described below with a flowchart.

<Method>
With reference to FIG. 2, an object detection process executed by the object detection apparatus 100 according to the present embodiment will be described. FIG. 2 is a flowchart showing the flow of the object detection method.

In step S10, the object detection device 100 acquires the in-vehicle image, the behavior information, and the vehicle setting information from the vehicle 200 and stores them in the storage device 102. In the present embodiment, these pieces of information are acquired from the vehicle 200 by wireless communication, but may be acquired via wired communication or a storage medium, or may be acquired via another device.

In step S12, as shown in FIG. 3A, the trajectory calculation unit 110 calculates the movement trajectory of the vehicle 200 in the three-dimensional space based on the behavior information 106. In the present embodiment, the trajectory calculation unit 110 uses the Kalman filter as a trajectory estimation algorithm based on position information (latitude / longitude / altitude), speed, yaw rate, and gradient information included in the behavior information 106, and moves the vehicle. Calculate the trajectory. The movement trajectory can be represented by a geographic coordinate system (world coordinate system). By using the Kalman filter, the movement trajectory can be smoothed. Note that the trajectory calculation unit 110 also preferably performs map matching using map information for calculating the movement trajectory. In the present embodiment, the movement trajectory is calculated over the entire acquisition period of the behavior information 106. In addition,
The movement trajectory may be obtained by simply integrating the velocity and the yaw rate, or may be obtained by simply connecting the position information at each time point.

More specifically, the movement trajectory can be obtained as a plane 32 that is a trajectory of a line segment 31 corresponding to the current position of the vehicle 200 as shown in FIGS. 3 (B) and 3 (C). The line segment 31 preferably has the same length as the width of the vehicle 200 and has a height on the road surface. Specifically, since a locus (line) of a point is obtained by the locus estimation algorithm, a locus of a line segment having the same length as the width of the vehicle 200 around the point may be obtained based on this result. In FIG. 3B, the line segment 31 is drawn so as to be positioned in the forefront of the vehicle 200, but this is an example, and the line segment 31 does not necessarily correspond to the forefront of the vehicle 200. The position of the line segment 31 is preferably the height of the road surface. This can be achieved by map matching using map information. The width of the line segment 31 is not necessarily the same value as the width of the vehicle 200, and may be a value obtained by adding a predetermined value to the width of the vehicle 200 or a predetermined coefficient.

  The subsequent processing is repeatedly executed for the processing target frame constituting the in-vehicle image that is a moving image (loop L1). The processing target frame may be determined in any way. For example, all the frames may be processed, or frames every predetermined time may be processed.

In step S14, the object detection unit 114 detects the detection target object from the processing target frame. The object to be detected may be arbitrary and includes, for example, an object such as a vehicle or a pedestrian (human). The object detection unit 114 can detect an object using a discriminator learned in advance using an image to be detected. The learning algorithm may be arbitrary, but for example, a method such as Deformable Part Model or Deep Learning can be used. It is not necessary to detect the object based on only one image of the processing target frame, and the object may be detected using images of frames before and after that. For example, an object in the current frame may be detected using a so-called object tracking method. Also, moving objects in the image may be extracted by a background extraction technique. Furthermore, the distance image obtained from radar or LIDAR may be used for object detection. Furthermore, white line information may be used when white lines are detected with high reliability.

FIG. 4 shows an in-vehicle image 40 of a processing target frame and three objects (vehicles here) 41, 42, 43 detected therein. The object detection unit 114 also identifies the position (region) of the detected object in the in-vehicle image. In the present embodiment, a rectangle including the detection object is specified as the position of the detection object in the in-vehicle image.

In step S16, the travel route projection unit 112 projects the vehicle movement trajectory calculated in step S12 onto the in-vehicle image using the behavior information and the vehicle information. As described above, the movement trajectory of the vehicle is obtained in the world coordinate system. The vehicle behavior information 106 includes vehicle position information, and the vehicle setting information 108 includes the installation position and shooting direction of the in-vehicle camera 202.

Here, the vehicle setting information 108 related to the in-vehicle camera 202 will be described with reference to FIGS. 5 (A) and 5 (B). FIG. 5A is a schematic view of the vehicle as viewed from above, and FIG. 5B is a schematic view of the vehicle as viewed from the side. The installation position of the in-vehicle camera 202 is expressed as an offset 52 from the vehicle center line 51 and a ground height 53. The installation position may include the front-rear direction of the vehicle, but may not include it. The shooting direction of the in-vehicle camera 202 is represented by an azimuth angle 54 and a depression angle 55. The vehicle setting information 108 includes the width 56 of the vehicle 200.

As shown in FIG. 6 (A), the travel route projection unit 112 performs coordinate transformation on the movement locus 61 of the vehicle using the position of the vehicle 200, the installation position of the in-vehicle camera 202, and the shooting direction, thereby The region 63 of the movement locus can be obtained. The region 63 where the movement locus 61 is projected onto the in-vehicle image 62 can be said to be a region where the movement locus 61 is imaged when it is assumed that the movement locus 61 exists in real space.

FIG. 6B shows a movement locus 65 projected on the in-vehicle image 64. Here, the vehicle 200 is traveling in the right lane of the two-lane road, and the movement locus 65 projected on the in-vehicle image 64 reflects that fact. The in-vehicle image 64 also shows detection objects 66 and 67. Further, the image shown in FIG. 6B is for the purpose of explanation, and it is not necessary to display this image or generate data of this image.

The processing in the next steps S17 to S22 is executed for each detected object in the image of the processing target frame (loop L2). For example, in the example shown in FIG. 6 (B), processing is performed on the vehicle 66 and the vehicle 67, respectively.

  In step S17, the on-route object determination unit 116 obtains the overlapping amount of the detected object and the movement locus projected on the in-vehicle image. The larger this overlap amount, the higher the possibility that the detected object is located on the travel route of the vehicle. That is, this overlap amount is useful as an index for determining whether or not the detected object is located on the travel route. Two methods for calculating the overlap amount will be described with reference to FIGS. 7 (A) and 7 (B).

FIG. 7A is a diagram for explaining a first method for calculating the overlap amount. Fig. 7 (A) shows the in-vehicle image.
A rectangular area 71 of the detection object and a projected movement locus 72 are shown. Here, the overlapping amount of the detected object and the movement locus can be defined as the ratio (ratio) of the length 74 of the portion of the lower side that overlaps the movement locus 72 to the length 73 of the entire lower side 71a of the rectangular area 71. .

FIG. 7B is a diagram for explaining a second method for calculating the overlap amount. The overlap amount of the detected object and the movement locus can be defined as the ratio (ratio) of the area of the area 75 (shaded portion) overlapping the movement locus 72 in the rectangular area 71 to the entire area of the rectangular area 71.

Both the first method and the second method can be used, but the detected object area is detected as a rectangular area, and the lower side of the rectangular area is set to a position corresponding to the road surface of the detected object. The first method is preferable. In this case, the fact that the lower side of the rectangular area overlaps with the movement locus means that the detected object is located on the movement locus.

Alternatively, a value obtained based on the two overlap amounts according to the first and second methods may be used as the overlap amount used for the determination. For example, a simple average or weighted average of two values, or a maximum value or minimum value may be used as the overlap amount used for the determination.

  In step S18, the on-route object determination unit 116 determines whether or not the overlap ratio obtained as described above is equal to or greater than a predetermined ratio. The threshold value (predetermined ratio) in this determination may be appropriately determined according to the application. Note that different threshold values may be used when the first method is used and when the second method is used.

  When the overlap amount is equal to or greater than the predetermined ratio (S18-YES), the on-route object determination unit 116 determines that the detected object is located on the travel route of the vehicle 200 (S20). On the other hand, when the overlap amount is less than the predetermined ratio (S18-NO), the on-route object determination unit 116 determines that the detected object is not located on the travel route of the vehicle 200 (S22). In the example of FIG. 6B, it is determined that the vehicle 66 is located on the travel route of the vehicle 200, whereas the vehicle 67 is determined not to be located on the travel route of the vehicle 200.

The object detection device 100 includes information on the position of the vehicle 200, the travel route, information on the detected object (for example, the type of object, the size of the object, the position in the vehicle-mounted image, the position relative to the vehicle 200, etc.) Information on whether or not the vehicle 200 is located on the travel route is stored in the storage device. Alternatively, the object detection device 100 may output such information to an external device.

In step S24, the output image generation unit 118 generates image data obtained by synthesizing the display of the moving locus of the vehicle and the detected object with respect to the in-vehicle image to be processed, and displays the data on the display unit or stores it in the storage device. To do. Here, the display of the detected object varies depending on whether or not the detected object is located on the travel route so that it can be determined whether or not the object is located on the travel route of the vehicle. It is preferable. FIG. 8 is a diagram illustrating an example of an output image generated by the output image generation unit 118. A display 81 representing the movement trajectory of the vehicle 200 and displays 82 and 83 representing two detection objects are superimposed on the in-vehicle image 80. Here, the display 82 of an object on the travel route of the vehicle 200 is a solid line, and the display 83 of an object not on the travel route is a dotted line. Of course, in addition to changing the type of line, it may be distinguishable by other methods such as changing the color of the line. Further, the value of the overlap amount obtained in step S17 may be displayed. Alternatively, the display of objects that are not on the travel route may be omitted.

<Advantageous effects of this embodiment>
According to this embodiment, since the vehicle movement trajectory is obtained based on the actual behavior information of the vehicle, in particular, the behavior information after photographing the processing target image, the vehicle movement trajectory can be obtained with high accuracy. Therefore, an object located on the travel route of the vehicle can be accurately detected from the in-vehicle image.

  Further, in the present embodiment, even when the accuracy of the position information (GNSS position) is poor, it is possible to detect the movement locus and thus the object on the travel route with high accuracy. When there is an error in the position information, the position of the movement locus in the geographic coordinate system has an error, but the vehicle position information used when projecting the movement locus also includes the same error. Therefore, the relative movement trajectory with respect to the vehicle is obtained with high accuracy. That is, the position of the movement trajectory relative to the vehicle can be obtained with high accuracy. Thus, even if the accuracy of the position information is poor, the final detection accuracy does not decrease.

  Since the processing of this embodiment cannot be performed immediately after image capturing, it is not necessarily optimal when real-time processing such as collision prediction is required. However, when learning control of automatic driving using actually measured data, not only the types and positions of surrounding objects but also information on whether these objects are located on the traveling route of the host vehicle is useful. is there. According to the present embodiment, it is possible to automatically and accurately detect whether or not a surrounding object is located on the travel route of the host vehicle in the measured data. Therefore, it can be said that it is particularly useful for generating learning data for automatic driving.

<Modification>
The above description is one embodiment of the present invention, and the present invention is not limited to the content. The present invention can be variously modified within the scope of its technical idea.

The object detection apparatus according to the present invention is realized by a programmable integrated circuit such as an FPGA (Field Programmable Gate Array) or a dedicated integrated circuit such as an ASIC (Application Specific Integrated Circuit) instead of being realized by a microprocessor and a program. May be. Moreover, you may implement | achieve by the combination of these. In addition, the object detection device is not necessarily realized by a single computer, and may be realized by a plurality of computers connected via a network. Further, the object detection device may be mounted on a vehicle that captures a vehicle-mounted image to be processed.

  It will be apparent that the object to be detected by the object detection apparatus according to the present invention is not particularly limited. In the above description, a vehicle or a pedestrian is taken as an example, but the detection target object is not limited to these and may be an arbitrary object.

The processing shown in the flowchart of FIG. 2 is merely an example, and the order and contents of the processing can be changed as appropriate. For example, the execution order of the object detection (S14) and the movement trajectory projection (S16) may be reversed or may be performed in parallel.

  In step S12, the entire movement trajectory of the vehicle 200 is obtained in advance from the behavior information. For each in-vehicle image of the processing target frame, the movement trajectory is obtained using behavior data for a predetermined period after the shooting time of the image. Also good. In this case, the movement trajectory of the vehicle may be obtained in the local coordinate system (vehicle coordinate system) without being obtained in the world coordinate system (geographic coordinate system). In addition, the period for obtaining the movement locus (the predetermined period described above) can be determined as appropriate. For example, the period for obtaining the movement locus may be a predetermined period, or may be a period in which the movement locus is a predetermined distance (or more). Moreover, the said period may be determined according to the shape of the road in driving | running | working, and you may make it obtain | require the movement locus | trajectory of a shorter period in a curve road compared with a straight road.

  In addition, when obtaining the movement trajectory of the vehicle, the trajectory (plane) of the line segment corresponding to the vehicle at each time point is obtained as the movement trajectory of the vehicle. ) First, and then the trajectory can be expanded according to the width of the vehicle to obtain a similar planar trajectory.

  In addition, the display image data generation process in step S24 may be omitted.

In the above embodiment, whether or not the detected object is located on the travel route of the vehicle 200 is determined according to the overlap amount of the detected object and the projected movement locus in the in-vehicle image. However, without making this determination, the overlap amount or a value (score) calculated based on the overlap amount may be output as a certainty factor that the detected object is located on the travel route of the vehicle 200. For example, it can only be determined probabilistically whether the preceding vehicle detected during automatic driving is located on the travel route of the host vehicle. Therefore, in order to perform learning that varies the control in accordance with the probability (confidence level), a configuration that outputs the certainty level is suitable.

100 Object detection device
110 Trajectory calculation unit 112 Travel path projection unit 114 Object detection unit
116 On-path object determination unit 118 Output image generation unit
200 vehicles

Claims (10)

An object detection method for detecting an object located on a travel route of a vehicle from an image taken by a vehicle equipped with an in-vehicle camera,
An in-vehicle image acquisition step of acquiring an in-vehicle image captured by the in-vehicle camera of the vehicle;
Behavior information acquisition step for acquiring behavior information of the vehicle after taking the in-vehicle image;
A trajectory calculating step for obtaining a movement trajectory of the vehicle based on the behavior information;
An area calculation step for obtaining an area corresponding to the movement locus in the in-vehicle image based on the movement locus;
An object detection step of detecting an object to be detected from the in-vehicle image;
A calculation step of calculating an overlap amount of the object detected in the object detection step with an area corresponding to the movement locus in the vehicle-mounted image;
An object detection method including: A determination step of determining, among the objects detected in the object detection step, an object having the overlapping amount equal to or greater than a predetermined ratio as an object located on a travel route of the vehicle;
The object detection method according to claim 1. In the locus calculation step, a locus of a line segment corresponding to the width of the vehicle is obtained as a movement locus of the vehicle.
The object detection method according to claim 1 or 2. In the object detection step, a region where the object exists in the vehicle-mounted image is obtained,
In the determination step, a ratio of a portion included in an area corresponding to the movement locus in an area where the object exists is calculated as the overlap amount.
The object detection method of any one of Claim 1 to 3. In the object detection step, a rectangular area where the object exists is obtained,
In the calculation step, a ratio of a portion included in an area corresponding to the movement locus in a lower side of the rectangular area where the object exists is calculated as the overlap amount.
The object detection method of any one of Claim 1 to 3. An image generation step of outputting image data in which the region corresponding to the movement locus obtained in the region calculation step and the display for specifying the object detected in the object detection step are superimposed on the in-vehicle image; Including,
The object detection method according to claim 1. In the image generation step, the display is generated in a mode corresponding to the overlap amount.
The object detection method according to claim 6. In-vehicle image acquisition means for acquiring an in-vehicle image captured by the in-vehicle camera of the vehicle;
Behavior information acquisition means for acquiring behavior information of the vehicle after photographing the vehicle-mounted image;
Based on the behavior information, a trajectory calculating means for obtaining a moving trajectory of the vehicle;
An area calculating means for obtaining an area corresponding to the movement locus in the in-vehicle image based on the movement locus;
Object detection means for detecting an object to be detected from the in-vehicle image;
A determination unit that calculates an overlap amount with a region corresponding to the movement locus in the in-vehicle image for the detected object;
An object detection device. The determination means determines that an object whose overlapping amount is a predetermined ratio or more among detected objects is an object located on a travel route of the vehicle.
The object detection apparatus according to claim 8.   The program which makes a computer perform each step of the method of any one of Claim 1 to 7.

Images