JP7027279B2 - Vehicle control devices, vehicle control methods, and programs - Google Patents

Description

The present invention relates to an object recognition device, a vehicle control device, an object recognition method, and a program.

Conventionally, a technique for recognizing an object by integrating outputs from a plurality of devices has been disclosed (see, for example, Patent Document 1). Such a technique is called sensor fusion. In general, when an object is recognized by sensor fusion, it can be said that the recognition reliability is higher than when the object is recognized based on the output of a single device. Therefore, in the technique described in Patent Document 1, when the object is recognized by the device alone, the degree of vehicle control is lowered as compared with the case where the object is recognized by the sensor fusion.

Japanese Unexamined Patent Publication No. 2005-239114

When recognizing an object by sensor fusion, the reliability is high, but on the other hand, it is expected that the completion of processing will be delayed as compared with the case of recognizing an object based on the output of a single device. For this reason, for example, even if an attempt is made to perform preliminary control on an object at a long distance from the vehicle, it may not be possible to perform this because the processing by the sensor fusion is not determined. As described above, in the conventional sensor fusion technology, the recognition of an object may be delayed.

The present invention has been made in consideration of such circumstances, and provides an object recognition device, a vehicle control device, an object recognition method, and a program capable of obtaining a prompt sensor fusion result more quickly. Is one of the purposes.

The object recognition device, the vehicle control device, the object recognition method, and the program according to the present invention have adopted the following configurations.
(1): The object recognition device according to one aspect of the present invention is an object recognition device mounted on a vehicle, and at least process A is performed based on the output of the first device for recognizing an object. The first processing unit that outputs the object information and outputs the object information by performing the processing B that requires more work than the processing A, and the second device for recognizing the object, the first device. Based on the output of the second device attached to the vehicle so as to have the same direction as the detection range, at least the process C is performed to output the object information, and the process requires more steps than the process C. A second processing unit that performs D and outputs object information, and a first integrated recognition unit that recognizes an object based on the object information obtained as a result of the processing A and the object information obtained as a result of the processing C. An object recognition device including an object information obtained as a result of the process B and a second integrated recognition unit for recognizing an object based on the object information obtained as a result of the process D.

(2): In the embodiment of (1) above, the process A is a process executed for the output data with a smaller number of output cycles than the first device as compared with the process B. And / or, the process C is a process executed for the output data with a smaller number of output cycles than the first device as compared with the process D.

(3): In the aspect of (2) above, the first device is a camera, and the process A includes a process of generating object information based on an image having a smaller number of frames than the preprocess B. ..

(4): In the aspect of (1) above, the process A is a process executed with a smaller number of process procedures than the process B, and / or the process C is compared with the process D. Therefore, it is a process that is executed with a small number of processing procedures.

(5): In the aspect of (4) above, the second device is a lidar, and the process C specifies a distance between a reference position and an object, and the specified distance is included in the object information and output. Is included.

(6): In the embodiment of the above (5), the process D includes a process of specifying the distance and the size of the object, including the specified distance and the size in the object information, and outputting the process.

(7): Another aspect of the present invention is the object recognition device according to any one of the above (1) to (6) and the operation control unit that controls one or both of the vehicle speed and the steering angle. And, the operation control unit controls to adjust the positional relationship with the object by using the recognition result of the first integrated recognition unit until the recognition result by the second integrated recognition unit is obtained. After the recognition result by the second integrated recognition unit is obtained, the ratio of the recognition result by the second integrated recognition unit is set higher than the recognition result by the first integrated recognition unit, and the positional relationship with the object is set. It is a vehicle control device that controls to adjust.

(8): In the aspect of (7) above, the operation control unit is compared after the recognition result by the second integrated recognition unit is obtained until the recognition result by the second integrated recognition unit is obtained. Therefore, the degree of control is reduced.

(9): In the embodiment (7) or (8), the operation control unit can obtain the recognition result by the second integrated recognition unit until the recognition result by the second integrated recognition unit is obtained. The degree of relative control over the positional relationship with the object is reduced as compared with the latter.

(10): Another aspect of the present invention is an object recognition method executed by an object recognition device mounted on a vehicle, at least based on the output of a first device for recognizing an object, processing A. Is performed to output the object information, and the process B, which requires more work than the process A, is performed to output the object information, and is a second device for recognizing the object, which is the first device. Based on the output of the second device attached to the vehicle so as to have the same direction as the detection range, at least the process C is performed to output the object information, and the process requires more steps than the process C. Performing D to output the object information, recognizing the object based on the object information obtained by the process A, the object information obtained by the process C, and the object information obtained by the process B. It is an object recognition method including recognizing an object based on the object information obtained by the process D.

(11): Another aspect of the present invention is a program executed by a processor of an object recognition device mounted on a vehicle, based on the output of the first device for recognizing an object to the processor. At least, it is a second device for recognizing an object by performing process A to output object information and performing process B which requires more steps than process A to output object information. Based on the output of the second device attached to the vehicle so that the detection range is in the same direction as the one device, at least the process C is performed to output the object information, and the number of steps is larger than that of the process C. A lot of processing D is performed to output the object information, the object is recognized based on the object information obtained by the processing A and the object information obtained by the processing C, the object information obtained by the processing B, and the processing. It is a program that recognizes an object based on the object information obtained by D.

According to (1) to (11), a quick sensor fusion result can be obtained more quickly.

It is a block diagram of the vehicle system 1 of the 1st Embodiment using an object recognition apparatus 300. It is a block diagram of the object recognition apparatus 300. It is a figure which exemplifies the traveling scene of the own vehicle M generated by the process executed by the vehicle system 1. It is a flowchart which shows an example of the flow of processing executed by the 1st processing unit 310. It is a flowchart which shows an example of the flow of the processing executed by the 2nd processing unit 320. It is a flowchart which shows an example of the flow of the process executed by the 1st integrated recognition unit 330. It is a flowchart which shows an example of the flow of the process executed by the 2nd integrated recognition unit 340. It is a figure which shows an example of the relationship between the distance and the detection accuracy. It is a figure which shows an example of the relationship between the distance and the detection accuracy. It is a figure which shows an example of the relationship between the distance and the detection accuracy by an embodiment. It is a block diagram of the vehicle system 2 of the 2nd Embodiment using the object recognition apparatus 300.

Hereinafter, embodiments of the object recognition device, vehicle control device, object recognition method, and program of the present invention will be described with reference to the drawings.

<First Embodiment>
[Overall configuration of vehicle]
FIG. 1 is a configuration diagram of a vehicle system 1 of a first embodiment using an object recognition device 300. The vehicle on which the vehicle system 1 is mounted is, for example, a vehicle such as a two-wheeled vehicle, a three-wheeled vehicle, or a four-wheeled vehicle, and the drive source thereof is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. When equipped with a motor, the motor operates using the power generated by the generator connected to the internal combustion engine or the discharge power of the secondary battery or the fuel cell.

The vehicle system 1 includes, for example, a camera 10, a radar device 12, a finder 14, an object recognition device 300, a communication device 20, an HMI (Human Machine Interface) 30, a vehicle sensor 40, a navigation device 50, and the like. It includes an MPU (Map Positioning Unit) 60, a driving controller 80, an automatic driving control device 100, a traveling driving force output device 200, a braking device 210, and a steering device 220. These devices and devices are connected to each other by a multiplex communication line such as a CAN (Controller Area Network) communication line, a serial communication line, a wireless communication network, or the like. The configuration shown in FIG. 1 is merely an example, and a part of the configuration may be omitted or another configuration may be added. The camera 10 and the finder 14 are examples of the first device and the second device, respectively.

The camera 10 is, for example, a digital camera using a solid-state image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). One or a plurality of cameras 10 are attached to any position of the vehicle (hereinafter referred to as own vehicle M) on which the vehicle system 1 is mounted. When photographing the front, the camera 10 is attached to the upper part of the front windshield, the back surface of the rear-view mirror, and the like. The camera 10 periodically and repeatedly images the periphery of the own vehicle M, for example. The camera 10 may be a stereo camera.

The radar device 12 radiates radio waves such as millimeter waves around the own vehicle M, and also detects radio waves (reflected waves) reflected by the object to detect at least the position (distance and direction) of the object. One or a plurality of radar devices 12 may be attached to any position of the own vehicle M. The radar device 12 may detect the position and velocity of the object by the FM-CW (Frequency Modulated Continuous Wave) method.

The finder 14 is a LIDAR (Light Detection and Ranging). The finder 14 irradiates the periphery of the own vehicle M with light and measures the scattered light. The finder 14 detects the distance to the target based on the time from light emission to light reception. The emitted light is, for example, a pulsed laser beam. One or a plurality of finder 14s may be attached to any position of the own vehicle M. The finder 14 measures scattered light by irradiating it with light while shifting the angle in the horizontal direction, for example. Then, the finder 14 outputs the information of the reflection point that emitted the scattered light to the object recognition device 300 in association with the irradiation angle at that time.

The object recognition device 300 performs sensor fusion processing on the detection results of a part or all of the camera 10, the radar device 12, and the finder 14, and recognizes the position, type, speed, and the like of the object. The object recognition device 300 outputs the recognition result to the automatic operation control device 100. Further, the object recognition device 300 may output the detection results of the camera 10, the radar device 12, and the finder 14 to the automatic driving control device 100 as they are, if necessary. The details of the object recognition device 300 will be described later.

The communication device 20 communicates with another vehicle existing in the vicinity of the own vehicle M by using, for example, a cellular network, a Wi-Fi network, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), or wirelessly. Communicates with various server devices via the base station.

The HMI 30 presents various information to the occupants of the own vehicle M and accepts input operations by the occupants. The HMI 30 includes various display devices, speakers, buzzers, touch panels, switches, keys and the like.

The vehicle sensor 40 includes a vehicle speed sensor that detects the speed of the own vehicle M, an acceleration sensor that detects the acceleration, a yaw rate sensor that detects the angular velocity around the vertical axis, an orientation sensor that detects the direction of the own vehicle M, and the like.

The navigation device 50 includes, for example, a GNSS (Global Navigation Satellite System) receiver 51, a navigation HMI 52, and a routing unit 53, and the first map information 54 is stored in a storage device such as an HDD (Hard Disk Drive) or a flash memory. Holds. The GNSS receiver 51 identifies the position of the own vehicle M based on the signal received from the GNSS satellite. The position of the own vehicle M may be specified or complemented by an INS (Inertial Navigation System) using the output of the vehicle sensor 40. The navigation HMI 52 includes a display device, a speaker, a touch panel, keys, and the like. The navigation HMI 52 may be partially or wholly shared with the above-mentioned HMI 30. The route determination unit 53, for example, has a route from the position of the own vehicle M (or an arbitrary position input) specified by the GNSS receiver 51 to the destination input by the occupant using the navigation HMI 52 (hereinafter,). The route on the map) is determined with reference to the first map information 54. The first map information 54 is, for example, information in which a road shape is expressed by a link indicating a road and a node connected by the link. The map route determined by the route determination unit 53 is output to the MPU 60. Further, the navigation device 50 may perform route guidance using the navigation HMI 52 based on the route on the map determined by the route determination unit 53. The navigation device 50 may be realized by the function of a terminal device such as a smartphone or a tablet terminal owned by an occupant, for example. Further, the navigation device 50 may transmit the current position and the destination to the navigation server via the communication device 20 and acquire the route on the map returned from the navigation server.

The MPU 60 functions as, for example, a recommended lane determination unit 61, and holds the second map information 62 in a storage device such as an HDD or a flash memory. The recommended lane determination unit 61 divides the route provided by the navigation device 50 into a plurality of blocks (for example, divides the route into 100 [m] units with respect to the vehicle traveling direction), and refers to the second map information 62 for each block. Determine the recommended lane. The recommended lane determination unit 61 determines the recommended lane so that the own vehicle M can travel on a reasonable route to proceed to the branch destination when a branch point, a merging point, or the like exists in the route.

The second map information 62 is map information with higher accuracy than the first map information 54. The second map information 62 includes, for example, information on the center of the lane, information on the boundary of the lane, and the like. Further, the second map information 62 may include road information, traffic regulation information, address information (address / zip code), facility information, telephone number information, and the like. The second map information 62 may be updated at any time by accessing another device using the communication device 20.

The driving controller 80 includes, for example, an accelerator pedal, a brake pedal, a shift lever, a steering wheel, a deformed steering wheel, a joystick, and other controls. A sensor for detecting the amount of operation or the presence or absence of operation is attached to the operation controller 80, and the detection result is the automatic operation control device 100, or the traveling driving force output device 200, the brake device 210, and the steering device. It is output to a part or all of 220.

The automatic operation control device 100 includes, for example, a first control unit 120 and a second control unit 160. The first control unit 120 and the second control unit 160 are each realized by executing a program (software) by a hardware processor such as a CPU (Central Processing Unit). In addition, some or all of these components are hardware (circuits) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), and GPU (Graphics Processing Unit). It may be realized by the part; including circuitry), or it may be realized by the cooperation of software and hardware. The combination of the object recognition device 300 and the automatic driving control device 100 is an example of a vehicle control device. The object recognition device 300 may be a function of the automatic driving control device 100.

FIG. 2 is a functional configuration diagram of the first control unit 120 and the second control unit 160. The first control unit 120 includes, for example, a recognition unit 130, a travel locus prediction unit 140, and an action plan generation unit 150.

The recognition unit 130 recognizes the surrounding situation of the own vehicle M based on the information input from the camera 10, the radar device 12, and the finder 14 via the object recognition device 300. The peripheral situation recognized by the recognition unit includes various objects. For example, the position of the object is first recognized as a position on absolute coordinates with the representative point of the own vehicle M (sensor position, center of gravity, center of drive axis, etc.) as the origin, and if necessary, on the road coordinates along the road. It is converted to a position and used for control. In the present embodiment, the recognition unit 130 shall exclusively acquire information about the object (object information) from the object recognition device 300.

The surrounding conditions recognized by the recognition unit 130 include the road structure, other vehicles, the relative position and posture of the own vehicle M and the traveling lane, the state of bicycles and pedestrians, temporary stop lines, obstacles, red lights, and tolls. It may include road events such as places and other information. The recognition result by the recognition unit 130 is output to the travel locus prediction unit 140 and the action plan generation unit 150.

In principle, the action plan generation unit 150 travels in the recommended lane determined by the recommended lane determination unit 61, and on top of that, does not come into contact with the object recognized by the object recognition device 300 or the recognition unit 130. Generate a target track on which the vehicle M will travel in the future. The target trajectory includes, for example, a plurality of trajectory points and a velocity element. For example, the target track is expressed as an arrangement of points (track points) to be reached by the own vehicle M in order. The track point is a point to be reached by the own vehicle M for each predetermined mileage (for example, about several [m]) along the road, and separately, for a predetermined sampling time (for example, about 0 comma number [sec]). ) Target velocity and target acceleration are generated as part of the target trajectory. Further, the track point may be a position to be reached by the own vehicle M at the sampling time for each predetermined sampling time. In this case, the information of the target velocity and the target acceleration is expressed by the interval of the orbital points.

The second control unit 160 sets the traveling driving force output device 200, the brake device 210, and the steering device 220 so that the own vehicle M passes the target trajectory generated by the action plan generation unit 150 at the scheduled time. Control. The second control unit 160 acquires, for example, information on the target trajectory (orbit point) generated by the action plan generation unit 150, stores it in a memory (not shown), and stores the speed associated with the target trajectory stored in the memory. The traveling driving force output device 200 or the braking device 210 is controlled based on the elements. Further, the second control unit 160 controls the steering device 220 according to the degree of bending of the target trajectory stored in the memory. In this way, the action plan generation unit 150 determines the target trajectory to which the speed element is attached, so that both acceleration / deceleration and steering of the own vehicle M are controlled.

The traveling driving force output device 200 outputs a traveling driving force (torque) for the vehicle to travel to the drive wheels. The traveling driving force output device 200 includes, for example, a combination of an internal combustion engine, a motor, a transmission, and the like, and an ECU that controls them. The ECU controls the above configuration according to the information input from the second control unit 160 or the information input from the operation controller 80.

The brake device 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to the information input from the second control unit 160 or the information input from the operation controller 80 so that the brake torque corresponding to the braking operation is output to each wheel. The brake device 210 may include a mechanism for transmitting the hydraulic pressure generated by the operation of the brake pedal included in the operation operator 80 to the cylinder via the master cylinder as a backup. The brake device 210 is not limited to the configuration described above, and is an electronically controlled hydraulic brake device that controls the actuator according to the information input from the second control unit 160 to transmit the hydraulic pressure of the master cylinder to the cylinder. May be good.

The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor, for example, exerts a force on the rack and pinion mechanism to change the direction of the steering wheel. The steering ECU drives the electric motor according to the information input from the second control unit 160 or the information input from the operation controller 80, and changes the direction of the steering wheel.

[Object recognition device and related controls]
FIG. 2 is a block diagram of the object recognition device 300. The object recognition device 300 includes, for example, a first processing unit 310, a second processing unit 320, a first integrated recognition unit 330, and a second integrated recognition unit 340. These components are realized by, for example, a hardware processor such as a CPU executing a program (software). Further, some or all of these components may be realized by hardware such as LSI, ASIC, FPGA, GPU (circuit unit; including circuitry), or realized by collaboration between software and hardware. May be done. Further, these functional units do not have to be realized by a single processor, and may be realized by performing distributed processing by a plurality of processors.

The first processing unit 310 analyzes, for example, an image captured by the camera 10 to generate and output object information. That is, the first processing unit 310 functions as a so-called image analysis unit. The first processing unit 310 may generate object information by extracting contours in an image and performing pattern matching, or may generate object information using a classifier learned by deep learning or the like. good. The process performed by the first process unit 310 includes at least process A and process B which requires more man-hours than process A.

Process A is, for example, a process of deriving (acquiring) the distance, height, and size of an object projected on the captured image based on an image for k frames, and outputting the derived contents as object information. Is. For example, k is 1.

Process B includes, for example, a process of deriving (acquiring) the distance, height, and size of an object projected on the captured image based on n frames of images, and a process of converting the object into an object. Includes processing to output the objectified contents as object information. For example, n is a natural number of 2 or more. As long as the relationship of k <n holds, k and n may be arbitrary natural numbers. Objectification estimates the type of object (four-wheeled vehicle, two-wheeled vehicle, bicycle, pedestrian, falling object, etc.) based on the distance and speed of the object derived by processing based on multiple frames, and the estimation result is the object. It is a process to correspond to the distance, height, size, etc. of Regarding the speed of the object, it is necessary to subtract the speed of the own vehicle M from the relative speed, but the object recognition device 300 acquires the speed of the own vehicle M from, for example, the automatic driving control device 100, and calculates the acquired information. Used for. Process B may include approximation processing by a Kalman filter or the like.

The second processing unit 320 processes the detection result output by the finder 14, for example, and generates and outputs the object information based on the processed information. The process performed by the second processing unit 320 includes at least the process C and the process D which requires more man-hours than the process C.

The process C is, for example, a process of acquiring a distance (and a direction) from an object based on a detection result output by the finder 14 and outputting it as object information.

The process D is, for example, a process of acquiring the distance (and direction) and the size of the object based on the detection result output by the finder 14, a process of converting the object into an object, and an output of the objectified contents as object information. Including processing to do. As described above, since the finder 14 is controlled to operate while changing the irradiation direction of light, for example, when reflection points at the same distance are lined up between the irradiation angles θ1 and θ2, the finder 14 is controlled. By multiplying the distance by the absolute value of the angle difference | θ2-θ1 |, the lateral size of the object can be approximately obtained. Process D includes such a process. The process D may include an approximation process using a Kalman filter or the like. The objectification is the same as that of process B.

The object information (hereinafter, object information A) that is the result of the process A and the object information (hereinafter, the object information C) that is the result of the process C are output to the first integrated recognition unit 330. "Output" is a convenient expression and may refer to an operation involving physical "transmission" from the output source to the output destination, or an operation of writing to an area of memory accessible to the output destination. It may point to.

The first integrated recognition unit 330 recognizes an object based on the object information A and the object information C. The recognition result (first integrated recognition result) by the first integrated recognition unit 330 is output to the automatic driving control device 100 as breaking news information. When the difference between the distance to the object included in the object information A and the distance to the object included in the object information C is within a predetermined range, the first integrated recognition unit 330 recognizes that the object exists at those distances. do. That is, the first processing unit 310 determines that some object exists at a predetermined distance X1 in the image for one frame from the camera 10, and the second processing unit 320 outputs the finder 14 to the predetermined distance X1. When suggesting that an object exists at a close position, the first integrated recognition unit 330 generates a first integrated recognition result indicating that the object exists at a position of a predetermined distance X1 and generates an automatic operation control device 100. Output to.

The object information (hereinafter, object information B) that is the result of the process B and the object information (hereinafter, the object information D) that is the result of the process D are output to the second integrated recognition unit 340.

The second integrated recognition unit 340 recognizes an object based on the object information B and the object information D. The recognition result by the second integrated recognition unit 340 (second integrated recognition result) is output to the automatic operation control device 100 as information having higher reliability than the recognition result of the first integrated recognition unit 330. The second integrated recognition unit 340 uses the matching portion of the object B and the object D when the objectified information contained in both the object B and the object D matches within the permissible range and the types of the objects match. It recognizes that an object of the estimated type exists at the indicated position, and outputs the recognition result to the automatic operation control unit 100. For example, the first processing unit 310 determines that the "vehicle" exists at the position of the predetermined distance X2 in the images for a plurality of frames from the camera 10, and the second processing unit 320 outputs the output of the finder 14 to the predetermined distance X2. If it suggests that a "vehicle" is located close to (for example, the size of the object contained in the object information matches the lateral size of the vehicle, and the object is at a speed that is difficult to walk on foot). (When moving), the second integrated recognition unit 340 generates a second integrated recognition result indicating that a "vehicle" exists at a position of a predetermined distance X2, and outputs the second integrated recognition result to the automatic driving control device 100.

In the automatic driving control device 100, for example, the own vehicle M prevents the action plan generation unit 150 from coming into contact with the object indicated by each recognition result based on the first integrated recognition result and the second integrated recognition result. Generates a target track to be driven in the future and controls the vehicle (adjusts the positional relationship with the object).

At this time, the action plan generation unit 150 controls to adjust the positional relationship with the object by using the first integrated recognition result from the time when the first integrated recognition result is obtained to the time when the second integrated recognition result is obtained. After the second integrated recognition result is obtained, the ratio of the second integrated recognition result is set higher than that of the first integrated recognition result to control the positional relationship with the object. "Increasing the ratio" includes setting the ratio of the second integrated recognition result to 100% and the ratio of the first integrated recognition result to 0%.

The action plan generation unit 150 reduces the degree of control between the time when the first integrated recognition result is obtained and the time when the second integrated recognition result is obtained, as compared with the time after the second integrated recognition result is obtained. "Reducing the degree of control" means reducing the degree of control given to the traveling driving force output device 200, the brake device 210, or the steering device 220. When the degree of control is reduced, the action plan generation unit 150 may generate a target trajectory so that the control amount such as the braking torque, the acceleration torque, and the change amount of the steering steering angle becomes small, or as a result of the control. The target track may be generated so that the vehicle behavior such as the appearing acceleration, deceleration, and turning angle becomes small.

Reducing the degree of control described above may be an absolute guideline that does not depend on the positional relationship with the object, or may be a relative guideline that depends on the positional relationship with the object. In the former case, for example, the upper limit of the brake torque between the time when the first integrated recognition result is obtained and the time when the second integrated recognition result is obtained is set from the upper limit of the brake torque after the second integrated recognition result is obtained. By reducing the torque, it is possible to reduce the degree of control. In the latter case, for example, when the brake torque is determined in a tendency to be inversely proportional to the TTC (Time To Collision) between the object and the own vehicle M, the first integrated recognition result is obtained and then the second integrated recognition result is obtained. The gain for calculating the brake torque up to is may be smaller than after the second integrated recognition result is obtained. Further, the action plan generation unit 150 is in the period from the acquisition of the first integrated recognition result to the acquisition of the second integrated recognition result, for example, "when the speed of the own vehicle M exceeds the upper limit speed, the upper limit speed is reached. Limited control (preliminary deceleration) such as "deceleration to" may be performed.

FIG. 3 is a diagram illustrating a traveling scene of the own vehicle M caused by the processing executed by the vehicle system 1. In the figure, the arrow D indicates the traveling direction of the own vehicle M. An obstacle OB exists in front of the vehicle M in the traveling direction. Since the actual distance varies depending on the performance of the device and the like, the numerical value is only an example, but for example, only the first integrated recognition result starts to be obtained at the point 200 [m] up to the obstacle OB. In response to this, the action plan generation unit 150 performs control such as the preliminary braking described above. Further, for example, the second integrated recognition result begins to be obtained at a point 150 [m] up to the obstacle OB. In response to this, the action plan generation unit 150 performs braking control according to the TTC between the own vehicle M and the obstacle OB, and stops the own vehicle M as necessary (for example, if avoidance by steering is impossible). ..

[Processing flow]
FIG. 4 is a flowchart showing an example of the flow of processing executed by the first processing unit 310. In the processing of this flowchart, for example, one routine is executed every time an image for one frame is input from the camera 10.

First, the first processing unit 310 performs a process of extracting a target area to be processed in the image input from the camera 10 (step S100). The target area is an area excluding the left and right edges and the upper part of the image, and is an area in which the front of the own vehicle M is imaged.

Next, the first processing unit 310 performs an object extraction process in the target region (step S102). The details of the processing in this step are as described above. Then, the first processing unit 310 determines whether or not the object is detected (extracted) in the image (step S104). The object here may be narrowed down to one that satisfies the condition such as "occupying more than a few pixels in the image". If no object is detected, the processing of one routine in this flowchart ends.

When an object is detected, the first processing unit 310 generates the above-mentioned object information A and outputs it to the first integrated recognition unit 330 (step S106). In addition, the first processing unit 310 performs identification processing of the object determined to be detected in step S104 and the object detected in the past in comparison with the latest frame in the past (step S108). At this time, the first processing unit 310 performs the identification process in consideration of the absolute velocity and size of the object.

Next, the first processing unit 310 refers to the processing result of step S108, and whether or not the same (presumed) object is continuously detected m times while the processing of this flowchart is repeatedly executed. Is determined (step S110). When it is determined that the same object has been detected m times in a row, the first processing unit 310 generates the object information B and outputs it to the second integrated recognition unit 340 (step S112).

FIG. 5 is a flowchart showing an example of the flow of processing executed by the second processing unit 320. In the processing of this flowchart, for example, one routine is executed every time the reflection point information is input from the finder 14.

First, the second processing unit 320 performs area-limited processing on the information input from the finder 14 (step S200). The area limitation process is a process for removing reflection points measured in areas other than the area on the road surface.

Next, the second processing unit 320 determines whether or not there is a reflection point on the road (step S202). When it is determined that there is a reflection point on the road, the second processing unit 320 generates the object information C and outputs it to the first integrated recognition unit 330 (step S204).

Further, the second processing unit 320 determines whether or not the reflection points at substantially the same distance are detected from end to end (step S206). When the reflection points at substantially the same distance are detected from end to end, the second processing unit 320 generates the object information D and outputs it to the second integrated recognition unit 340 (step S208).

FIG. 6 is a flowchart showing an example of the flow of processing executed by the first integrated recognition unit 330. First, the first integrated recognition unit 330 determines whether or not the object information A and the object information C are input in synchronization (or within a predetermined time) (step S300). When the object information A and the object information C are input in synchronization, the first integrated recognition unit 330 determines whether or not they match (step S302). "Matching" means, for example, that a predetermined number or more of both items are within the allowable range. When both are matched, the first integrated recognition unit 330 generates the first integrated recognition information. (Step S304), the output is output to the automatic operation control device 100.

FIG. 7 is a flowchart showing an example of the flow of processing executed by the second integrated recognition unit 340. First, the second integrated recognition unit 340 determines whether or not the object information B and the object information D are input in synchronization (or within a predetermined time) (step S400). When the object information B and the object information D are input in synchronization, the second integrated recognition unit 340 determines whether or not they match (step S402). When both are matched, the second integrated recognition unit 340 generates the second integrated recognition information (step S404) and outputs the second integrated recognition information to the automatic driving control device 100.

[summary]
According to the object recognition device 300 of the first embodiment described above, it is possible to obtain a quick sensor fusion result more quickly. 8 to 10 are diagrams showing an example of the relationship between the distance and the detection accuracy. These figures assume a detection rate according to the distance between the own vehicle M and the obstacle on the premise that an obstacle of 10 cm cubic and an obstacle of 15 cm cubic are placed on the road.

FIG. 8 shows a result assuming a detection rate, which is the probability that the first processing unit 310 has detected an obstacle of each size based on the output of the camera 10. Here, "an obstacle is detected" means "an object information B is generated". Further, FIG. 9 shows a result assuming a detection rate which is a probability that the second processing unit 320 detects an obstacle of each size based on the output of the finder 14. Here, "an obstacle is detected" means "an object information D is generated". As shown in FIG. 8, especially for an obstacle of 10 cm cube, it is assumed that the detection rate does not reach 80% or more unless the distance is close to about 160 m. Further, as shown in FIG. 9, it is assumed that the object information D is not generated in the first place unless the distance is close to about 180 m, especially for an obstacle of 10 cm cube.

Then, the second integrated recognition information generated by obtaining the AND of the object information B and the object information D rises further later than the rise of the detection rate shown in FIG. 8 or 9. As a result, it is expected that sufficient information cannot be given to the automatic driving control device 100, especially when the distance to the obstacle is 150 m or more.

On the other hand, FIG. 10 shows the expected performance according to the embodiment. FIG. 10 shows the probability (generation rate) that the first integrated recognition information is generated by the first integrated recognition unit 330 based on the object information A and the object information C that are started to be generated earlier than those in FIG. 8 or 9. Shows the transition of. As shown in the figure, the generation rate of the first integrated recognition information according to the present embodiment is expected to be sufficiently close to 100% when the distance is close to about 190 m. Therefore, in the approaching scene with an obstacle, it is possible to acquire a breaking fusion result at an earlier timing and provide it to the automatic driving control device 100.

Although the camera 10 and the finder 14 are exemplified as examples of the first device and the second device, one or both of them may be the radar device 12 or the other object recognition device.

<Second Embodiment>
Hereinafter, the second embodiment will be described. FIG. 11 is a configuration diagram of the vehicle system 2 of the second embodiment using the object recognition device 300. The components common to the first embodiment are designated by the same reference numerals as those of the first embodiment, and detailed description thereof will be omitted. In the second embodiment, the object recognition device 300 outputs the recognition result (first integrated recognition result and second integrated recognition result) to the driving support device 400 instead of the automatic driving control device 100.

The driving support device 400 is a device that performs various driving support controls such as acceleration / deceleration, steering intervention control, and reaction force output to the driving controller 80 when the driver is manually driving. In the following description, it is assumed that the driving support device 400 performs anti-obstacle braking control. The driving support device 400 includes, for example, a collision possibility determination unit 402, a brake operation amount derivation unit 404, and a brake amount determination unit 406. These functional units are realized by, for example, a hardware processor such as a CPU executing a program (software). Further, some or all of these components may be realized by hardware such as LSI, ASIC, FPGA, GPU (circuit unit; including circuitry), or realized by collaboration between software and hardware. May be done. Further, these functional units do not have to be realized by a single processor, and may be realized by performing distributed processing by a plurality of processors.

The collision possibility determination unit 402 refers to the recognition result of the object recognition device 300 and determines the collision possibility between the own vehicle and the object. For example, the collision possibility determination unit 402 calculates the TTC between the own vehicle and the object, and determines that there is a possibility of collision between the own vehicle and the object when the TTC is smaller than the collision determination threshold value Tcol.

The brake operation amount derivation unit 404 refers to the detection result of the brake step sensor included in the driving controller 80, and derives the brake operation amount (or the brake amount output as a result) performed by the occupant of the own vehicle. ..

The brake amount determination unit 406 first determines the brake amount (object brake amount) based on the presence of an object based on the determination result by the collision possibility determination unit 402 and the processing process thereof, and outputs the brake amount to the brake device 210. do. The amount of anti-object braking is determined, for example, so that the smaller the TTC between the own vehicle and the object, the larger the amount. When the brake amount based on the brake operation amount derived by the brake operation amount extraction unit 404 exceeds the objective brake amount, the brake amount determination unit 406 may output the latter brake amount to the brake device 220. Further, the brake amount determination unit 406 reduces the degree of control between the time when the first integrated recognition result is obtained and the time when the second integrated recognition result is obtained, as compared with the time after the second integrated recognition result is obtained. .. Further, the brake amount determination unit 406 is in the period from the acquisition of the first integrated recognition result to the acquisition of the second integrated recognition result, for example, "when the speed of the own vehicle M exceeds the upper limit speed, the upper limit speed is reached. Limited control (preliminary deceleration) such as "deceleration to" may be performed.

According to the second embodiment described above, the same effect as that of the first embodiment can be obtained.

The above embodiment can be expressed as follows.
An object recognition device mounted on a vehicle
A storage device that stores the program and
With a hardware processor capable of executing the program,
By executing the program, the hardware processor
Based on the output of the first device for recognizing an object, at least process A is performed to output the object information, and process B, which requires more man-hours than the process A, is performed to output the object information.
It is a second device for recognizing an object, and at least process C is performed based on the output of the second device attached to the vehicle so that the detection range is in the same direction as the first device. In addition to outputting information, processing D, which requires more man-hours than processing C, is performed to output object information.
The object is recognized based on the object information obtained as a result of the process A and the object information obtained as a result of the process C.
The object is recognized based on the object information obtained as a result of the process B and the object information obtained as a result of the process D.
An object recognition device configured as such.

Although the embodiments for carrying out the present invention have been described above using the embodiments, the present invention is not limited to these embodiments, and various modifications and substitutions are made without departing from the gist of the present invention. Can be added.

10 Camera 12 Radar device 14 Finder 80 Driving controller 100 Automatic driving control device 300 Object recognition device 310 1st processing unit 320 2nd processing unit 330 1st integrated recognition unit 340 2nd integrated recognition unit 400 Driving support device

Claims (10)

A vehicle control device mounted on a vehicle.
Based on the output of the first device for recognizing an object, at least the process A is performed to output the object information, and the process B, which requires more man-hours than the process A, is performed to output the object information. 1 processing unit and
It is a second device for recognizing an object, and at least process C is performed based on the output of the second device attached to the vehicle so that the detection range is in the same direction as the first device. A second processing unit that outputs information and outputs object information by performing processing D, which requires more man-hours than processing C, and
A first integrated recognition unit that recognizes an object based on the object information obtained as a result of the process A and the object information obtained as a result of the process C.
A second integrated recognition unit that recognizes an object based on the object information obtained as a result of the process B and the object information obtained as a result of the process D.
It is equipped with a driving control unit that controls one or both of the speed and steering angle of the vehicle.
The operation control unit controls to adjust the positional relationship with the object by using the recognition result of the first integrated recognition unit until the recognition result by the second integrated recognition unit is obtained, and the second After the recognition result by the integrated recognition unit is obtained, control is performed to adjust the positional relationship with the object by making the ratio of the recognition result of the second integrated recognition unit higher than the recognition result by the first integrated recognition unit. conduct,
Vehicle control device . The process A is a process executed for the output data with a smaller number of output cycles than the first device as compared with the process B.
And / or
The process C is a process executed for the output data with a smaller number of output cycles than the second device as compared with the process D.
The vehicle control device according to claim 1. The first device is a camera.
The process A includes a process of generating object information based on an image having a smaller number of frames than the preprocess B.
The vehicle control device according to claim 2. The process A is a process executed with a smaller process procedure than the process B.
And / or
The process C is a process executed with a smaller process procedure than the process D.
The vehicle control device according to claim 1. The second device is LIDAR.
The process C includes a process of specifying a distance between the LIDAR and an object, including the specified distance in the object information, and outputting the specified distance.
The vehicle control device according to claim 4. The process D includes a process of specifying the distance and the size of the object, and including the specified distance and the size in the object information and outputting the process.
The vehicle control device according to claim 5. The operation control unit reduces the degree of control until the recognition result by the second integrated recognition unit is obtained, as compared with after the recognition result by the second integrated recognition unit is obtained.
The vehicle control device according to any one of claims 1 to 6 . The operation control unit is relative to the positional relationship with the object until the recognition result by the second integrated recognition unit is obtained, as compared with after the recognition result by the second integrated recognition unit is obtained. Reduce the degree of control,
The vehicle control device according to any one of claims 1 to 7 . A vehicle control method executed by a vehicle control device mounted on a vehicle.
Based on the output of the first device for recognizing an object, at least process A is performed to output the object information, and process B, which requires more man-hours than the process A, is performed to output the object information. When,
It is a second device for recognizing an object, and at least process C is performed based on the output of the second device attached to the vehicle so that the detection range is in the same direction as the first device. In addition to outputting information, processing D, which requires more man-hours than processing C, is performed to output object information.
Recognizing an object based on the object information obtained by the process A and the object information obtained by the process C, and outputting the first integrated recognition result .
Recognizing an object based on the object information obtained by the process B and the object information obtained by the process D, and outputting the second integrated recognition result .
To control one or both of the vehicle's speed and steering angle,
The control is to control the positional relationship with the object by using the first integrated recognition result until the second integrated recognition result is obtained, and the second integrated recognition result is obtained. After that, the control including adjusting the positional relationship with the object by making the ratio of the second integrated recognition result higher than that of the first integrated recognition result is included.
Vehicle control method . A program executed by the processor of the vehicle control device mounted on the vehicle.
To the processor
Based on the output of the first device for recognizing an object, at least process A is performed to output the object information, and process B, which requires more man-hours than the process A, is performed to output the object information.
It is a second device for recognizing an object, and at least process C is performed based on the output of the second device attached to the vehicle so that the detection range is in the same direction as the first device. The information is output, and the object information is output by performing the process D, which requires more man-hours than the process C.
An object is recognized based on the object information obtained by the process A and the object information obtained by the process C, and the first integrated recognition result is output .
An object is recognized based on the object information obtained by the process B and the object information obtained by the process D, and the second integrated recognition result is output .
Control one or both of the vehicle speed and steering angle,
The control causes the processor to control the positional relationship with the object by using the first integrated recognition result until the second integrated recognition result is obtained, and the second integrated recognition result is obtained. After the recognition result is obtained, the ratio of the second integrated recognition result is made higher than that of the first integrated recognition result, and control for adjusting the positional relationship with the object is performed.
program.

Images