JP6317986B2 - Object recognition device - Google Patents

Description

  The present invention relates to an object recognition device that recognizes an object existing around a moving body.

  Japanese Patent Laid-Open No. 2004-133867 discloses that even if the size information of the fusion target cannot be acquired properly due to the instantaneously unstable detection result by the imaging means, the fusion target is updated without being affected as much as possible. It is an issue ([0006], summary).

In order to solve the problem, in Patent Document 1, the radar target MP can be detected when the generated fusion target FP is updated, but the imaging means generates an instantaneous image lost (undetected) size. Even if the information cannot be acquired properly, it is estimated that there is an error in the detection result of the imaging means and the previous detected object exists. Then, the image lost fusion targets LFP _{1 to} LFP ₄ are generated using the interpolated image targets LIP _{1 to} LIP ₄ interpolated by the estimation based on the size information of the fusion target FP acquired in the past. Update the fusion target FP (summary).

  Also, in Patent Document 1, in addition to the case where the image lost occurs, the detection result of the millimeter wave radar 2 is incorrect and the object no longer exists, for example, the light hit condition (sunset, shadow, etc.) ), The case where the image detection object is instantaneously lost (not detected) on the stereo camera 3 side is assumed ([0032]).

JP 2007-226680 A

In Patent Document 1, even when an image lost occurs, size information can be used by generating the image lost fusion targets LFP _{1 to} LFP ₄ . However, there is room for improvement in the technique of Patent Document 1.

  For example, in Patent Document 1, the size information of the fusion target FP is specified, but the type of the fusion target FP is not specified, and the “type” is taken over when the image lost occurs. Has not been considered.

The present invention has been made in consideration of the above-described problems, and an object of the present invention is to provide an object recognition apparatus capable of suitably recognizing attributes of an object existing around a moving body.

An object recognition apparatus according to the present invention recognizes first recognition means for recognizing the position of the first object based on reflection of electromagnetic waves from the first object, and recognition of the position and attribute of the second object present in the captured image. The second recognizing means, and the same position when the position of the first object recognized by the first recognizing means and the position of the second object recognized by the second recognizing means match or are within a predetermined distance. And a matching means for determining that the target object is mounted on a moving body,
When the second object is not recognized after the combination of the first object and the second object determined to be the same target object by the matching unit is recognized, the attribute of the second object is set. It further comprises takeover determining means for determining whether to take over,
The takeover determining means takes over the attribute of the second object on at least one of the conditions that the non-recognition time after the second object is no longer recognized is below a predetermined takeover permission threshold. .

According to the present invention, as one non-recognition time is at least provided that below a predetermined takeover permission threshold from no longer being recognized second object to be used for recognition of the attributes of the object, the attribute of the second object take over. For this reason, it is possible to appropriately determine whether or not to inherit the attribute of the second object, and to increase the accuracy of determining the attribute of the target object.

The takeover determining means recognizes the distance from the moving body to the target object after the combination of the first object and the second object determined to be the same target object by the matching means is recognized. It may be determined whether or not the attribute of the second object is taken over as an additional condition that the distance is shorter than a predetermined distance threshold. For this reason, it is possible to continue to recognize the attributes of the target object for a relatively long time compared to the processing when the target object cannot be detected instantaneously due to the influence of the lighting condition (sunset, shadow, etc.) It becomes.

Further, if the target object is not so close to the moving body that it exceeds the distance threshold with the moving body, it is possible to determine whether or not to inherit the attribute of the second object. Therefore, it is possible to reduce the calculation load in the object recognition apparatus by limiting the scenes where the determination about the takeover is performed.

Before Ki引 splicing determination means, wherein the distance to the target object whether shorter than the distance threshold, it is determined using a recognized position of the first object in the first recognition means from the mobile Also good. Thereby, it is possible to determine whether the distance from the moving body to the target object is too short by using the reflection of the electromagnetic wave instead of the captured image. For this reason, when the position recognition accuracy of the first recognition unit is higher than that of the second recognition unit, the distance from the moving body to the target object can be recognized with high accuracy.

The takeover determining means determines the takeover permission threshold based on a duration time that the matching means determines that the first object and the second object are the same target object before the second object is not recognized. May be changed. The duration time affects the recognition accuracy of the attribute of the target object. For this reason, it becomes possible to improve the determination accuracy of the attribute of the target object by changing the takeover permission threshold according to the duration.

The takeover determining means may change the takeover permission threshold based on a moving speed of the moving body or a relative speed of the moving body and the target object. The moving speed of the moving body or the relative speed between the moving body and the target object affects the recognition accuracy of the attribute of the target object. For this reason, it is possible to improve the determination accuracy of the attribute of the target object by changing the takeover permission threshold according to the moving speed or the relative speed.

The takeover determining means may change the takeover permission threshold based on the number of the first objects recognized by the first recognizing means. The number of first objects recognized by the first recognition means affects the recognition accuracy of the attribute of the target object. For this reason, it is possible to increase the determination accuracy of the attribute of the target object by changing the takeover permission threshold according to the number of first objects.

The takeover determining means may prohibit taking over the attribute of the target object based on a change in the reflection level of the electromagnetic wave from the first object. The change in the reflection level of the electromagnetic wave from the first object affects the recognition accuracy of the attribute of the target object. For this reason, it is possible to improve the determination accuracy of the attribute of the target object by prohibiting the takeover according to the change in the reflection level.

According to the present invention, it is possible to appropriately recognize the attributes of an object existing around a moving body.

It is a block diagram which shows the structure of the vehicle carrying the object recognition apparatus which concerns on one Embodiment of this invention. It is a 1st flowchart of the attribute determination control in the said embodiment. It is a 2nd flowchart of the attribute determination control in the said embodiment. It is a figure which shows the relationship between the attribute information continuation time of the last (past) and temporary attribute takeover permission threshold value. It is a figure which shows the relationship between the relative speed of the said vehicle and a target object, and a temporary attribute inheritance permission threshold value. It is a figure which shows the relationship between the number of the 1st objects which exist in an image non-recognition area | region, and a temporary attribute inheritance permission threshold value. It is a figure which shows the positional relationship of the pedestrian as a 2nd object or a target object, and the said vehicle. It is a figure which shows the 1st example of the captured image of a camera. It is a figure which shows the 2nd example of the captured image of the said camera.

A. Embodiment A1. Configuration [A1-1. overall structure]
FIG. 1 is a block diagram showing a configuration of a vehicle 10 (hereinafter also referred to as “own vehicle 10”) equipped with an object recognition device 12 according to an embodiment of the present invention. In addition to the object recognition device 12, the vehicle 10 includes a vehicle behavior stabilization system 14 (hereinafter referred to as "VSA system 14"), an electric power steering system 16 (hereinafter referred to as "EPS system 16"), and a pop-up hood system 18 (hereinafter referred to as "". A PUH system 18 ”) and a vehicle speed sensor 20.

  The object recognition device 12 detects various objects 100 (for example, other vehicles, pedestrians, walls) that appear around the host vehicle 10. Then, the object recognition device 12 selects or specifies the object 100 (hereinafter also referred to as “detected object 100”) that is related to the control of the host vehicle 10 as the target object 100tar (hereinafter also referred to as “object 100tar”). To do. The object recognition device 12 calculates the distance L from the host vehicle 10 to the target object 100tar, and determines the attribute Prtar of the target object 100tar. The attribute Prtar here includes, for example, the type Ca (for example, vehicle, pedestrian (human) or wall), size, and the like of the target object 100tar.

  An electronic control unit 30 (hereinafter referred to as “VSA ECU 30”) of the VSA system 14 performs vehicle behavior stabilization control, and the vehicle 10 is turned when turning on a curved road through control of a brake system (not shown). The behavior of the vehicle 10 when the other vehicle approaches the vehicle is stabilized.

  An electronic control device 32 (hereinafter referred to as “EPS ECU 32”) of the EPS system 16 executes steering assist control, and is a constituent element of an electric power steering device {an electric motor, a torque sensor, and a steering angle sensor (all shown in FIG. Etc.) assists the steering by the driver.

  The electronic control device 34 (hereinafter referred to as “PUH ECU 34”) of the PUH system 18 is a pop-up hood (not shown) of the vehicle 10 when the vehicle 10 encounters a collision with the pedestrian 102 (FIGS. 7 to 9). ) And the impact when the head of the pedestrian 102 hits the vehicle body is reduced.

  The vehicle speed sensor 20 detects the vehicle speed V [km / h] of the vehicle 10 and outputs it to the object recognition device 12 or the like.

[A1-2. Object recognition device 12]
As shown in FIG. 1, the object recognition device 12 includes a radar 40, a camera 42, and an object recognition electronic control device 44 (hereinafter referred to as “object recognition ECU 44” or “ECU 44”).

(A1-2-1. Radar 40)
The radar 40 outputs a transmission wave Wt, which is an electromagnetic wave (here, a millimeter wave), to the outside of the vehicle 10, and reflects back to the detection object 100 (for example, another vehicle, pedestrian 102) out of the transmission wave Wt. The reflected wave Wr is received. Then, a detection signal corresponding to the reflected wave Wr (hereinafter referred to as “reflected wave signal Swr” or “signal Swr”) is output to the ECU 44. Hereinafter, the detected object 100 detected by the radar 40 is also referred to as “first object 100r” or “radar target 100r”.

  The radar 40 is disposed on the front side of the vehicle 10 (for example, a front bumper and / or a front grill). In addition to the front side or instead of the front side, the vehicle 10 may be arranged on the rear side (for example, the rear bumper and / or the rear grille) or the side (for example, the side of the front bumper).

  As will be described later, a sensor such as a laser radar or an ultrasonic sensor can be used instead of the radar 40 that outputs millimeter waves.

(A1-2-2. Camera 42)
The camera 42 (imaging means) acquires an image 200 (hereinafter also referred to as “peripheral image 200” or “captured image 200”) (FIGS. 8 and 9) around the vehicle 10 (including the target object 100tar). . Then, a signal corresponding to the image 200 (hereinafter referred to as “image signal Si” or “signal Si”) is output to the ECU 44. Hereinafter, the detected object 100 detected by the camera 42 is also referred to as a “second object 100c” or a “camera target 100c”.

  In the present embodiment, one camera 42 is used, but a stereo camera may be configured by arranging two cameras 42 symmetrically. The camera 42 acquires the image 200 at 15 frames or more (for example, 30 frames) per second. The camera 42 is a monochrome camera that mainly uses light having a wavelength in the visible light region, but may be a color camera or an infrared camera. The camera 42 is disposed, for example, at the center in the vehicle width direction (for example, around the rearview mirror) in the front portion of the vehicle interior of the vehicle 10. Alternatively, the camera 42 may be disposed at the center in the vehicle width direction in the front bumper portion of the vehicle 10.

(A1-2-3. Object recognition ECU 44)
The object recognition ECU 44 controls the entire object recognition device 12, and includes an input / output unit 50, a calculation unit 52, and a storage unit 54 as shown in FIG.

  The reflected wave signal Swr from the radar 40 and the image signal Si from the camera 42 are supplied to the object recognition ECU 44 via the input / output unit 50. Communication between the object recognition ECU 44 and the VSA ECU 30, EPS ECU 32, and PUH ECU 34 is performed via the communication line 70 and the input / output unit 50. The input / output unit 50 includes an A / D conversion circuit (not shown) that converts an input analog signal into a digital signal.

  The calculation unit 52 performs calculations based on the signals Swr and Si from the radar 40 and the camera 42, and generates signals for the VSA ECU 30, EPS ECU 32, and PUH ECU 34 based on the calculation results.

  As shown in FIG. 1, the calculation unit 52 includes a radar information processing unit 60 (first recognition unit), a camera information processing unit 62 (second recognition unit), and a target object information processing unit 64 (matching unit). Each processing unit 60, 62, 64 is realized by executing a program stored in the storage unit 54. The program may be supplied from the outside via a wireless communication device (mobile phone, smartphone, etc.) not shown. A part of the program can be configured by hardware (circuit parts).

  The radar information processing unit 60 detects information (hereinafter “radar information Ir” and “first object information Ir”) of the detected object 100 (first object 100r) based on the reflected wave Wr (reflected wave signal Swr) detected by the radar 40. Or “information Ir”). The camera information processing unit 62 uses information (hereinafter “camera information Ic”, “second object information”) of the detected object 100 (second object 100c) using the peripheral image 200 (see FIGS. 8 and 9) acquired by the camera 42. Ic "or" information Ic ").

  The target object information processing unit 64 is information combining the radar information Ir calculated by the radar information processing unit 60 and the camera information Ic calculated by the camera information processing unit 62 (hereinafter “target object information It” or “information It”). Is calculated). In other words, the processing unit 64 performs so-called fusion processing. The information It is information on the target object 100tar specified based on the detected object 100 (first object 100r) detected by the radar 40 and the detected object 100 (second object 100c) detected by the camera 42. The processing unit 64 includes an attribute determination unit 66 (attribute takeover determination unit) that executes part of the attribute determination control for determining the attribute Prtar of the target object 100tar.

  The storage unit 54 includes an imaging signal converted into a digital signal, a RAM (Random Access Memory) that stores temporary data used for various arithmetic processes, and a ROM (Read Only Memory) that stores an execution program, a table, a map, or the like. ) Etc.

A2. Control [A2-1. Overview of control]
In the present embodiment, the target object information It (position Potar, speed Vtar, acceleration atar, attribute Prtar, etc. of the target object 100tar) detected or recognized by the object recognition device 12 (object recognition ECU 44) is used as the VSA ECU 30, EPS ECU 32, and PUH ECU 34. Output to. Each ECU 30, 32, 34 performs control using the target object information It. For example, the VSA ECU 30 performs automatic brake control using the information It. EPS ECU32 performs automatic steering control using information It, for example. PUH ECU34 performs PUH control using information It.

[A2-2. Attribute judgment control]
(A2-2-1. Overall flow of attribute determination control)
2 and 3 are first and second flowcharts of attribute determination control in the present embodiment. The attribute determination control is executed by the calculation unit 52 (processing units 60, 62, 64) of the object recognition ECU 44. The calculation unit 52 repeats the processes shown in FIGS. 2 and 3 at a predetermined calculation cycle (for example, any one of several μsec to several hundred msec).

  In step S1, the ECU 44 (radar information processing unit 60) calculates radar information Ir of the detected object 100 (first object 100r) based on the reflected wave signal Swr from the radar 40. The radar information Ir includes the position Por, the speed Vr, the acceleration ar, and the like of the first object 100r. When there are a plurality of first objects 100r within the detection range of the radar 40, the ECU 44 calculates the position Por, the speed Vr, the acceleration ar and the like for each first object 100r.

  In step S2, the ECU 44 (camera information processing unit 62) calculates camera information Ic of the detected object 100 (second object 100c) based on the image signal Si (captured image 200) of the camera 42. The camera information Ic includes the position Poc of the second object 100c, the lateral speed Vlc, the acceleration ac, the attribute Prc, and the like. Further, the attribute Prc includes the type Ca (pedestrian, vehicle, etc.), size, etc., of the second object 100c. When there are a plurality of second objects 100c in the captured image 200, the ECU 44 calculates the position Poc, the lateral velocity Vlc, the acceleration ac, the attribute Prc, and the like for each second object 100c. If the second object 100c gets too close to the vehicle 10, the ECU 44 cannot calculate the attribute Prc (details will be described later).

  In step S3, the ECU 44 (target object information processing unit 64) performs a matching determination for matching the radar information Ir (radar target 100r) and the camera information Ic (camera target 100c). The matching determination is the same when the position Por of the first object 100r recognized by the radar information processing unit 60 and the position Poc of the second object 100c recognized by the camera information processing unit 62 match or are within a predetermined distance. It is determined that the target object is 100 tar.

  When there is radar information Ir (radar target 100r) for which there is no camera information Ic (camera target 100c) as a matching partner, the radar information Ir is the target object information It (target It is associated with the object 100 tar). In this case, the estimated position of the radar target 100r (or the target object 100tar) at the current time is calculated based on the moving direction and speed of the radar target 100r (or the target object 100tar) in the previous calculation cycle. Then, the radar target 100r that is recognized in the current calculation cycle and that is closest to the estimated position is linked to the target object 100tar calculated in the previous calculation cycle.

  Steps S4 to S18 are performed by the attribute determination unit 66 of the ECU 44. That is, in step S4, the ECU 44 determines whether or not the target object 100tar in the front-rear direction (traveling direction) is within the image non-recognition region 154 (FIG. 7). The image non-recognition area 154 means an area where the pedestrian 102 cannot fit within the shooting range or angle of view of the captured image 200. In the present embodiment, the image non-recognition region 154 is set with the size of the pedestrian 102 as a fixed value (design value). Therefore, the determination as to whether or not the target object 100tar is within the image non-recognition region 154 is performed using the position Por of the first object 100r calculated based on the reflected wave signal Swr from the radar 40. In other words, the determination in step S4 is a determination as to whether or not the first object 100r is within the image non-recognition region 154, and is performed based on the radar information Ir (position Por).

  When the target object 100tar is not in the image non-recognition area 154 (S4: NO), in step S5, the ECU 44 updates the attribute information duration Tpc (hereinafter also referred to as “duration Tpc” or “time Tpc”). . Specifically, 1 is added to the previous value of time Tpc to obtain the current value of time Tpc.

  The time Tpc means a time during which the camera information processing unit 62 continues to recognize the attribute Prc of the second object 100c (in other words, the number of calculation cycles). In other words, at time Tpc, the target object information processing unit 64 (matching means) determines that the first object 100r and the second object 100c are the same target object 100tar before the second object 100c is no longer recognized. It's time. For this reason, when the camera information processing unit 62 cannot recognize the attribute Prc of the second object 100c, the time Tpc is reset (S15 in FIG. 3 described later).

  In subsequent step S6, the ECU 44 resets the attribute non-recognition time Tpn (hereinafter also referred to as “non-recognition time Tpn” or “time Tpn”). The time Tpn means a time during which the attribute Prc of the second object 100c cannot be recognized in the camera information processing unit 62 (in other words, the number of calculation cycles). The time Tpn is used particularly in steps S9 and S14 in FIG.

  In step S7, the ECU 44 uses the attribute Prc calculated in step S2 (that is, the attribute Prc based on the captured image 200) as the attribute Prtar of the target object 100tar.

  Returning to step S4, when the target object 100tar is in the image non-recognition region 154 (S4: YES), the process proceeds to step S8. In step S <b> 8, the ECU 44 determines whether or not the attribute Prtar of the target object 100 tar can be recognized based on the captured image 200. This determination can be made based on whether the attribute Prtar of the target object 100tar can be recognized or whether the attribute Prc of the second object 100c can be recognized.

  If the attribute Prtar is recognized (S8: YES), steps S5 to S7 are executed in the same manner as described above. If the attribute Prtar is not recognized (S8: NO), the ECU 44 updates the attribute non-recognition time Tpn in step S9 of FIG. Specifically, 1 is added to the previous value of time Tpn to obtain the current value of time Tpn.

  In Steps S10 to S12, a plurality of attribute takeover permission threshold Tpa candidates (temporary attribute takeover permission thresholds Tpa_p1 to Tpa_p3) based on different setting indexes are calculated. In step S13, an attribute takeover permission threshold Tpa is selected from the temporary attribute takeover permission thresholds Tpa_p1 to Tpa_p3.

  Specifically, in step S10, the ECU 44 calculates a temporary attribute takeover permission threshold Tpa_p1 based on the latest (past) attribute information continuation time Tpc.

  FIG. 4 is a diagram showing a relationship between the latest (past) attribute information duration Tpc and the temporary attribute takeover permission threshold Tpa_p1. As shown in FIG. 4, when the attribute information duration Tpc is less than Tpc1, the threshold value Tpa_p1 is set to zero. When the time Tpc is equal to or greater than Tpc1 and less than Tpc2, the threshold value Tpa_p1 is set to gradually increase. When the time Tpc is equal to or longer than Tpc2, the threshold value Tpa_p1 is made constant at the maximum value Tpa1_max.

  Returning to FIG. 3, in the subsequent step S11, the ECU 44 calculates a temporary attribute takeover permission threshold Tpa_p2 based on the relative speed Vrel between the host vehicle 10 and the target object 100tar.

  FIG. 5 is a diagram illustrating a relationship between the relative speed Vrel between the vehicle 10 and the target object 100tar and the temporary attribute takeover permission threshold value Tpa_p2. As shown in FIG. 5, when the relative speed Vrel is less than Vrel1, the threshold value Tpa_p2 is made constant at the maximum value Tpa2_max. When the relative speed Vrel is equal to or higher than Vrel1 and lower than Vrel2, the threshold value Tpa_p2 is set to be gradually decreased. When the relative speed Vrel is equal to or higher than Vrel2, the threshold value Tpa_p2 is made constant at the minimum value Tpa2_min.

  Returning to FIG. 3, in the subsequent step S12, the ECU 44 calculates a temporary attribute takeover permission threshold Tpa_p3 based on the number Nr of the first objects 100r existing in the image non-recognition region 154.

  FIG. 6 is a diagram showing the relationship between the number Nr of the first objects 100r existing in the image non-recognition region 154 and the temporary attribute takeover permission threshold Tpa_p3. As shown in FIG. 6, when the number Nr is less than Nr1, the threshold value Tpa_p3 is made constant at the maximum value Tpa3_max. When the number Nr is greater than or equal to Nr1 and less than Nr2, the threshold value Tpa_p3 is set to gradually decrease. When the number Nr is greater than or equal to Nr2, the threshold value Tpa_p3 is made constant at the minimum value Tpa3_min.

  Returning to FIG. 3, in the following step S13, the ECU 44 sets the minimum value among the threshold values Tpa_p1 to Tpa_p3 as the attribute takeover permission threshold value Tpa. As will be described later, a value other than the minimum value may be set as the permission threshold value Tpa.

  In step S14, the ECU 44 determines whether or not the attribute non-recognition time Tpn updated in step S9 is equal to or less than the permission threshold Tpa set in step S13. If the time Tpn is not less than or equal to the threshold Tpa (S14: NO), the ECU 44 resets the attribute information duration Tpc in step S15. As a result, when step S5 in FIG. 2 is reached in the next calculation cycle, the duration time Tpc starts from zero.

  In subsequent step S16, the ECU 44 determines that the attribute Prtar of the target object 100tar cannot be determined. In other words, the ECU 44 determines that the attribute Prtar is not taken over.

  Returning to step S14, when the attribute non-recognition time Tpn is equal to or less than the permission threshold Tpa (S14: YES), in step S17, the ECU 44 determines whether or not the change in the reflection level Lr of the reflected wave Wr from the first object 100r is large. Determine whether. Specifically, the ECU 44 has an absolute value | ΔLr | of the difference ΔLr between the current value of the reflection level Lr and the average value Lr_ave of the reflection level Lr equal to or less than a threshold value THΔLr (hereinafter also referred to as “reflection level threshold value THΔLr”). It is determined whether or not. The threshold value THΔLr is set, for example, as a value indicating a difference ΔLr that cannot be obtained with the same object, and an experimental value or a simulation value can be used.

  When the change in the reflection level Lr is large (S17: YES), the process proceeds to step S16, and the ECU 44 determines that the attribute Prtar of the target object 100tar cannot be determined without taking over the attribute Prtar. When the change in the reflection level Lr is not large (S17: NO), the process proceeds to step S18, and the ECU 44 takes over the attribute Prtar of the target object 100tar in the previous calculation cycle.

(A2-2-2. Image Unrecognized Area 154)
FIG. 7 is a diagram illustrating a positional relationship between the pedestrian 102 and the vehicle 10 as the second object 100c or the target object 100tar. FIG. 8 is a diagram illustrating a first example of a captured image 200 of the camera 42. FIG. 9 is a diagram illustrating a second example of the captured image 200 of the camera 42.

  In FIG. 7, an area 150 is an imageable area of the camera 42, an area 152 is an image recognition area, and an area 154 is an image non-recognition area. A solid line frame 160 is a frame corresponding to the first object 100r detected based on the reflected wave signal Swr. A dashed frame 162 is a frame corresponding to the second object 100c detected based on the image signal Si.

  Note that the frame 160 is shown even when the pedestrian 102 is in any of the positions Po1 and Po2. On the other hand, the frame 162 is shown only when the pedestrian 102 is at the position Po1, and is not shown when the pedestrian 102 is at the position Po2. This indicates that when the pedestrian 102 is at the position Po2, the position Poc by the captured image 200 cannot be recognized.

  As shown in step S4 of FIGS. 7 and 2, in this embodiment, an image non-recognition region 154 is used. In the region 154, when the pedestrian 102 as the second object 100c approaches the vehicle 10, the camera information processing unit 62 (second recognition unit) of the ECU 44 recognizes the second object 100c in the captured image 200 as the pedestrian 102. An area on the near side of the distance that cannot be performed (hereinafter referred to as “distance Linv” or “image non-recognition distance Linv”) is shown. In other words, the distance Linv means a distance (distance threshold) at which the pedestrian 102 cannot fit within the shooting range or angle of view of the captured image 200.

  However, depending on the height and posture of the pedestrian 102, the attribute Prc of the pedestrian 102 (second object 100c) may be recognized based on the image signal Si even when the pedestrian 102 is in the image non-recognition region 154. is there. In other words, the distance Linv is set based on the size / posture of the pedestrian 102 as a reference value or a design value. The distance Linv is set to a value longer than the shortest shooting distance of the camera 42 lens.

  In the case of FIG. 8, the pedestrian 102 is at a position Po1 in FIG. 7, for example. In this case, the object recognition device 12 (ECU 44) can detect the pedestrian 102 based on either the detection value (reflected wave signal Swr) of the radar 40 or the captured image 200 (image signal Si) of the camera 42.

  In other words, the ECU 44 detects the pedestrian 102 as the first object 100r based on the reflected wave signal Swr from the radar 40, and detects the pedestrian 102 as the second object 100c based on the image signal Si from the camera 42. . Therefore, the target object information processing unit 64 (matching means) can match the first object 100r and the second object 100c, and can determine the position Potar, the attribute Prtar, and the like of the pedestrian 102 (target object 100tar). is there.

  On the other hand, in the case of FIG. 9, the pedestrian 102 is, for example, at a position Po2 in FIG. In this case, the object recognition device 12 (ECU 44) can detect the pedestrian 102 based on the reflected wave signal Swr from the radar 40, but can detect the pedestrian 102 based on the captured image 200 of the camera 42. Can not. For this reason, basically, the attribute Ptar of the pedestrian 102 (target object 100tar) cannot be determined.

  As shown in the flowchart of FIG. 2, even if the target object 100tar exists in the image non-recognition area 154 (S4: YES), the attribute Prtar of the target object 100tar can be recognized (S8: YES). ), The attribute Prtar is used (S7).

  As described above, in this embodiment, when it is determined in steps S6 to S18 in FIG. 3 whether or not the attribute Prtar of the target object 100tar is to be taken over, the target object 100tar exists in the image non-recognition region 154. (S4: YES) is one of the conditions. As a result, with respect to the image non-recognition region 154 (the region that the camera 42 is not good at), it is possible to continue control of the target object 100tar such as the pedestrian 102 even if the calculation load is increased.

A3. As described above, according to the present embodiment, the attribute non-recognition time Tpn after the second object 100c used for recognition of the type Ca of the target object 100tar is not recognized is equal to or less than the attribute takeover permission threshold Tpa. (S14: YES in FIG. 3) is at least one of the conditions, and the type Ca of the second object 100c is taken over as the attribute Prtar of the target object 100tar (S18). For this reason, it is possible to appropriately determine whether or not the type Ca of the second object 100c should be taken over, and to improve the determination accuracy of the type Ca of the target object 100tar.

  According to the present embodiment, after the combination of the first object 100r and the second object 100c is recognized by the target object information processing unit 64 (matching means) (S3 in FIG. 2), the distance from the vehicle 10 to the target object 100tar. With another condition that L is shorter than the distance Linv (distance threshold) (S4: YES), it is determined whether or not to inherit the type Ca of the second object 100c as the attribute Prtar of the target object 100tar (FIG. 3). S18). For this reason, compared with the processing in the case where the target object 100tar cannot be detected instantaneously due to the influence of the lighting condition (sunset, shadow, etc.), the type Ca of the target object 100tar is continuously recognized for a relatively long time. It becomes possible.

  Further, if the target object 100tar is not so close to the vehicle 10 that the distance Linv with the vehicle 10 is exceeded, it is possible to determine whether or not to take over the type Ca of the second object 100c. Therefore, it is possible to reduce the calculation load in the object recognition device 12 by limiting the scenes where the determination about the takeover is performed.

  In the present embodiment, the attribute determination unit 66 determines that the target object 100tar (pedestrian 102) is in the image non-recognition area 154 (S4: YES in FIG. 2), in other words, the target object from the vehicle 10 (moving body). It is determined using the position Por of the first object 100r recognized by the radar information processing unit 60 (first recognition means) that the distance L up to 100 tar is shorter than the image non-recognition distance Linv (distance threshold) (FIG. 2). S4).

  Thereby, it is possible to determine whether the distance L from the vehicle 10 to the target object 100tar is too short using the reflected wave Wr (electromagnetic wave reflection) instead of the captured image 200. For this reason, when the radar information processing unit 60 has higher position recognition accuracy than the camera information processing unit 62 (second recognition unit), the distance L from the vehicle 10 to the target object 100 tar can be recognized with high accuracy. It becomes possible.

  In the present embodiment, the attribute determination unit 66 (attribute takeover determination means) changes the attribute takeover permission threshold Tpa based on the attribute information duration Tpc (S10 and S13 in FIG. 3). The duration Tpc affects the recognition accuracy of the type Ca of the target object 100tar. For this reason, it is possible to increase the determination accuracy of the type Ca of the target object 100tar by changing the attribute takeover permission threshold value Tpa according to the duration time Tpc.

  In the present embodiment, the attribute determination unit 66 changes the attribute takeover permission threshold Tpa based on the relative speed Vrel between the vehicle 10 and the target object 100tar (S11 and S13 in FIG. 3). The relative speed Vrel affects the recognition accuracy of the type Ca of the target object 100tar. For this reason, it is possible to increase the determination accuracy of the type Ca of the target object 100tar by changing the attribute takeover permission threshold Tpa according to the relative speed Vrel.

  In the present embodiment, the attribute determination unit 66 changes the attribute takeover permission threshold Tpa based on the number Nr of the first objects 100r recognized by the radar information processing unit 60 (first recognition unit) (S12 in FIG. 3). S13). The number Nr of the first objects 100r recognized by the radar information processing unit 60 affects the recognition accuracy of the type Ca of the target object 100tar. For this reason, it is possible to improve the determination accuracy of the type Ca of the target object 100tar by changing the attribute takeover permission threshold Tpa according to the number Nr of the first objects 100r.

  In the present embodiment, the attribute determination unit 66 (attribute takeover determination means) prohibits the takeover of the type Ca of the target object 100tar based on the change in the reflection level Lr of the reflected wave Wr (electromagnetic wave) from the first object 100r ( S17 in FIG. 3: YES → S16). The change in the reflection level Lr from the first object 100r affects the recognition accuracy of the type Ca of the target object 100tar. For this reason, it is possible to improve the determination accuracy of the type Ca of the target object 100tar by prohibiting takeover according to the change in the reflection level Lr.

  In the present embodiment, the object recognition device 12 includes a camera 42 that acquires the captured image 200. Further, the image non-recognition distance Linv (distance threshold) is set to a value longer than the shortest shooting distance of the lens of the camera 42.

  Thereby, even if acquisition of the captured image 200 is performed in a focused state, if there is a high possibility that the recognition accuracy of the camera information processing unit 62 will decrease thereafter, the type Ca of the second object 100c is taken over. Determination (S8 in FIG. 2, S9 to S18 in FIG. 3) can be started. For this reason, it is possible to improve the determination accuracy of the type Ca of the target object 100tar while considering the calculation load.

  In the present embodiment, the distance Linv is a distance that prevents the camera information processing unit 62 from recognizing the second object 100c in the captured image 200 as the pedestrian 102 when the pedestrian 102 as the second object 100c approaches the vehicle 10. (FIGS. 7 and 9).

  Accordingly, when there is a possibility that the camera information processing unit 62 may not be able to recognize the type Ca of the second object 100c (pedestrian 102) due to the pedestrian 102 being too close to the vehicle 10, the second object 100c It becomes possible to start determination of takeover of the type Ca (pedestrian 102) (S8 in FIG. 2, S9 to S18 in FIG. 3). For this reason, it is possible to improve the determination accuracy of the type Ca of the target object 100tar while considering the calculation load.

B. Modifications It should be noted that the present invention is not limited to the above-described embodiment, and it is needless to say that various configurations can be adopted based on the description of the present specification. For example, the following configuration can be adopted.

B1. Application target In the above-described embodiment, the object recognition device 12 is applied to the vehicle 10, but the present invention is not limited thereto, and may be applied to another target. For example, the object recognition device 12 can be used for a moving body such as a ship or an aircraft. Alternatively, the object recognition device 12 may be applied to a robot, a security monitoring device, or a home appliance. The object recognition device 12 may be arranged outside the vehicle 10 instead of being mounted on the vehicle 10 (moving body) itself. In this case, it is possible to communicate between the vehicle 10 and the object recognition device 12 to notify the vehicle 10 of the recognition result (target object information It) of the object recognition device 12.

B2. Configuration of Object Recognizing Device 12 In the above embodiment, the output (target object information It) of the object recognizing device 12 is used by the VSA ECU 30, EPS ECU 32, and PUH ECU 34, but may be used for other purposes. For example, it can be used for parking assistance of the vehicle 10 or prevention of erroneous start.

  In the above embodiment, the radar 40 using the transmission wave Wt and the reflected wave Wr that are millimeter waves is used. However, for example, from the viewpoint of obtaining the information Ir of the first object 100r using the reflected wave Wr of the transmission wave Wt as an electromagnetic wave, a sensor such as a laser radar or an ultrasonic sensor can be used.

B3. Control of the object recognition ECU 44 [B3-1. Inheritance of attribute Prtar of target object 100tar]
In the above embodiment, as a condition for taking over the attribute Prtar of the target object 100tar (S18 in FIG. 3), the target object 100tar is in the image non-recognition area 154 (S4 in FIG. 2: YES), and the target object 100tar The attribute Prtar is not recognizable (S8: NO), the attribute non-recognition time Tpn is less than or equal to the attribute takeover permission threshold Tpa (S14: YES in FIG. 3), and the change in the reflection level Lr is not large (S17: NO) ).

  However, from the viewpoint of taking over the attribute Prtar of the target object 100tar, only one, two, or three of these determinations (S4, S8, S14, S17) are taken over from the attribute Prtar of the target object 100tar. It is good also as conditions to perform. Alternatively, determinations other than the above determinations can be added.

  In the above embodiment, after the combination of the first object 100r and the second object 100c determined by the target object information processing unit 64 (matching means) to be the same target object 100tar is recognized, the second object 100c is recognized. The determination that it was not performed was performed based on the latest attribute information duration Tpc. That is, when determining whether to take over the attribute Prtar of the target object 100tar, the type of the second object 100c is that the latest attribute information duration Tpc is not zero (in this embodiment, it is not less than Tpc1). It was determined that Ca can be taken over (S10 and S13 in FIG. 3).

  However, the determination can be performed by other methods. For example, when the type Ca of the second object 100c is recognized in the previous calculation cycle, but the type Ca of the second object 100c is not recognized in the current calculation cycle, the first object 100r and the second object 100c. After the combination is recognized, it may be determined that the second object 100c is no longer recognized. Alternatively, when the target object information processing unit 64 sets a flag indicating that the matching is successful for the target object 100tar, and the matching is not successful in the subsequent calculation cycle, the combination of the first object 100r and the second object 100c It is also possible to determine that the second object 100c is no longer recognized after being recognized.

[B3-2. Image non-recognition area 154]
In the embodiment described above, it is determined whether or not the target object 100tar exists in the image non-recognition region 154 that is the region on the near side of the vehicle 10 (S4 in FIG. 2). However, for example, from the viewpoint of determining whether or not to take over the attribute Prtar of the target object 100tar, the present invention is not limited to this.

  For example, in addition to or instead of the area 154 on the near side of the vehicle 10, it may be determined whether or not the target object 100 tar exists in the area on the far side of the vehicle 10. The back side region is set on the back side of the imageable region 150. For example, since the distance L from the vehicle 10 is too long, an area that is not a target of various controls of the vehicle 10 (for example, control by the ECUs 30, 32, and 34) can be set as the back area.

[B3-3. Attribute takeover permission threshold Tpa]
In the above embodiment, the minimum value among the three temporary attribute takeover permission threshold values Tpa_p1 to Tpa_p3 is set as the attribute takeover permission threshold value Tpa (S13 in FIG. 3). However, for example, from the viewpoint of setting the attribute takeover permission threshold Tpa using the three temporary attribute takeover permission thresholds Tpa_p1 to Tpa_p3, the present invention is not limited to this. For example, an average value, an intermediate value, or a maximum value of the three temporary attribute takeover permission threshold values Tpa_p1 to Tpa_p3 may be set as the attribute takeover permission threshold value Tpa.

  Also, only one or two of the three temporary attribute takeover permission thresholds Tpa_p1 to Tpa_p3 can be used for setting the attribute takeover permission threshold Tpa. Alternatively, the attribute takeover permission threshold value Tpa may be a fixed value instead of a variable value.

  In the above embodiment, the attribute takeover permission threshold value Tpa is updated for each calculation cycle (see FIG. 3). However, for example, from the viewpoint of using the permission threshold Tpa, this is not limiting. For example, once the integration of the attribute non-recognition time Tpn is started, the attribute takeover permission threshold Tpa can be used as a fixed value without being updated.

  In the above embodiment, the temporary attribute takeover permission threshold Tpa_p2 is calculated based on the relative speed Vrel between the host vehicle 10 and the target object 100tar (S11 in FIG. 3). However, for example, from the viewpoint of the detection accuracy of the radar 40, the temporary attribute takeover permission threshold Tpa_p2 can be calculated based on the vehicle speed V of the host vehicle 10. In this case, the horizontal axis in FIG. 5 is used by replacing the relative speed Vrel with the vehicle speed V.

  In the above embodiment, the temporary attribute takeover permission threshold value Tpa_p3 is calculated based on the number Nr of the first objects 100r existing in the image non-recognition region 154 (S12 in FIG. 3). However, for example, from the viewpoint of changing the temporary attribute takeover permission threshold Tpa_p3 based on the number of objects 100 present around the vehicle 10, this is not a limitation. For example, the temporary attribute takeover permission threshold Tpa_p3 may be calculated based on the number Nr of the first objects 100r in the imageable area 150.

[B3-4. Reflection level Lr]
In the above embodiment, in order to determine the abnormality of the reflection level Lr of the reflected wave Wr, it is determined whether or not the change in the reflection level Lr is large (S17 in FIG. 3). However, for example, from the viewpoint of determining an abnormality in the reflection level Lr, the present invention is not limited to this. For example, it is possible to set an upper limit threshold value for determining that the reflection level Lr is too high, and to determine whether or not the current value of the reflection level Lr exceeds the upper limit threshold value.

  The reflection level Lr is not for the reflected wave Wr from the first object 100r corresponding to the target object 100tar, but the reflection level Lr of the reflected wave Wr from the entire scanning area of the radar 40 can also be used. .

DESCRIPTION OF SYMBOLS 10 ... Vehicle (moving body) 12 ... Object recognition apparatus 42 ... Camera 60 ... Radar information processing part (1st recognition means)
62 ... Camera information processing section (second recognition means)
64 ... Target object information processing unit (matching means)
66 ... Attribute determining unit (attribute takeover determining means) 100 ... detected object 100c ... second object 100r ... first object 100tar ... target object 102 ... pedestrian 200 ... captured image Ca ... type L ... target object from vehicle (moving body) Distance Linv ... Image non-recognition distance (distance threshold)
Lr ... Reflection level of reflected wave (electromagnetic wave) Nr ... Number of first objects Poc ... Position of second object Por ... Position of first object Prtar ... Attribute of target object Tpa ... Attribute takeover permission threshold Tpc ... Attribute information duration ( Duration) Tpn ... attribute non-recognition time V ... vehicle speed Vrel ... relative speed Wr ... reflected wave Wt ... transmitted wave (electromagnetic wave)

Claims (6)

First recognition means for recognizing the position of the first object based on reflection of electromagnetic waves from the first object; second recognition means for recognizing the position and attribute of the second object present in the captured image; Matching that determines the same target object when the position of the first object recognized by the first recognition means and the position of the second object recognized by the second recognition means match or are within a predetermined distance And an object recognition device mounted on a moving body,
When the second object is not recognized after the combination of the first object and the second object determined to be the same target object by the matching unit is recognized, the attribute of the second object is set. It further comprises takeover determining means for determining whether to take over,
The takeover determining means takes over the attribute of the second object, at least as one of the conditions that the non-recognition time after the second object is no longer recognized is below a predetermined takeover permission threshold,
The takeover determining means determines the takeover permission threshold based on a duration time that the matching means determines that the first object and the second object are the same target object before the second object is not recognized. An object recognition device characterized by changing the value. First recognition means for recognizing the position of the first object based on reflection of electromagnetic waves from the first object; second recognition means for recognizing the position and attribute of the second object present in the captured image; Matching that determines the same target object when the position of the first object recognized by the first recognition means and the position of the second object recognized by the second recognition means match or are within a predetermined distance And an object recognition device mounted on a moving body,
When the second object is not recognized after the combination of the first object and the second object determined to be the same target object by the matching unit is recognized, the attribute of the second object is set. It further comprises takeover determining means for determining whether to take over,
The takeover determining means takes over the attribute of the second object, at least as one of the conditions that the non-recognition time after the second object is no longer recognized is below a predetermined takeover permission threshold,
The takeover determining means changes the takeover permission threshold based on a moving speed of the moving body or a relative speed of the moving body and the target object. First recognition means for recognizing the position of the first object based on reflection of electromagnetic waves from the first object; second recognition means for recognizing the position and attribute of the second object present in the captured image; Matching that determines the same target object when the position of the first object recognized by the first recognition means and the position of the second object recognized by the second recognition means match or are within a predetermined distance And an object recognition device mounted on a moving body,
When the second object is not recognized after the combination of the first object and the second object determined to be the same target object by the matching unit is recognized, the attribute of the second object is set. It further comprises takeover determining means for determining whether to take over,
The takeover determining means takes over the attribute of the second object, at least as one of the conditions that the non-recognition time after the second object is no longer recognized is below a predetermined takeover permission threshold,
The handover determination unit changes the handover permission threshold based on the number of the first objects recognized by the first recognition unit. The object recognition apparatus according to any one of claims 1 to 3,
The takeover determining means recognizes the distance from the moving body to the target object after the combination of the first object and the second object determined to be the same target object by the matching means is recognized. An object recognition apparatus characterized by determining whether or not to inherit the attribute of the second object with an additional condition that the distance is shorter than a predetermined distance threshold. The object recognition device according to claim 4.
The handover determination unit determines whether the distance from the moving body to the target object is shorter than the distance threshold using the position of the first object recognized by the first recognition unit. An object recognition device. In the object recognition device according to any one of claims 1 to 5,
The object recognizing device, wherein the takeover determining means prohibits taking over the attribute of the target object based on a change in a reflection level of the electromagnetic wave from the first object.

Images