JP2021068423A - Future behavior estimation device, vehicle control device, future behavior estimation method, and program - Google Patents

Abstract

To provide a future behavior estimation device, vehicle control device, future behavior estimation method, and program capable of performing prediction corresponding to an actual traffic scene.SOLUTION: A future behavior estimation device includes: an object position recognition section for recognizing positions of a plurality of traffic participants; a temporary goal determination section for determining temporary goals at which each one of the plurality of traffic participants is to arrive based on a recognition result of the object position recognition section; and a simulation section for simulating movement processes of each one of the plurality of traffic participants toward temporary goals determined by the temporary goal determination section with the use of movement models in order to estimate a future behavior of each one of the plurality of traffic participants.SELECTED DRAWING: Figure 2

Description

The present invention relates to a future behavior estimation device, a vehicle control device, a future behavior estimation method, and a program.

Conventionally, research has been carried out on estimating future behavior (movement) of pedestrians and bicycles existing on the road (for example, Non-Patent Documents 1 to 5).

D. Helbing and P. Molnar, “Social Force Model for Pedestrian Dynamics”, 20 May 1998. G. Arechavaleta and J. Laumond, “The nonholonomic nature of human locomotion: a modeling study”, February 2001. K. Mombaur and A. Truong, “From human to humanoid locomotion-An inverse optimal control approach From human to humanoid locomotion --an inverse optimal”, 31 December, 2009. M. Luber, J. A. Stork, G. D. Tipaldi, and K. O. Arras, “People tracking with human motion predictions from social forces,” Proc. --IEEE Int. Conf. Robot. Autom., Pp. 464-469, 2010. F. Farina, D. Fontanelli, A. Garulli, A. Giannitrapani, and D. Prattichizzo, “When Helbing Meets Laumond: The Headed Social Force Model.” 12-14 December, 2016.

The technique described in Non-Patent Document 1 makes an estimation on the premise that a pedestrian's goal (destination) is given as known information, and cannot make a prediction according to an actual traffic scene. There was a case.

The present invention has been made in consideration of such circumstances, and provides a future behavior estimation device, a vehicle control device, a future behavior estimation method, and a program capable of making a prediction according to an actual traffic scene. One of the purposes is to do.

The future behavior estimation device, the vehicle control device, the future behavior estimation method, and the program according to the present invention have adopted the following configurations.
(1): The future behavior estimation device according to one aspect of the present invention includes an object position recognition unit that recognizes the positions of a plurality of traffic participants, and the plurality of traffic participation based on the recognition results of the object position recognition unit. A tentative goal determination unit that determines a tentative goal for each person to reach the future, and the plurality of traffic participants using a movement model in order to estimate the future behavior of each of the plurality of traffic participants. Each of the persons is provided with a simulation unit that simulates the movement process toward the temporary goal determined by the temporary goal determination unit.

(2): In the aspect of the above (1), the provisional goal determination unit includes, for each of the plurality of traffic participants, a history of the positions of the plurality of traffic participants recognized by the object position recognition unit, and a history of the positions of the plurality of traffic participants. A plurality of provisional goal candidates are set based on the information indicating the road structure, the result of simulating the traffic participant toward each of the plurality of provisional goal candidates, and the movement based on the posture of the traffic participant. The tentative goal candidate with the smallest divergence is determined as the tentative goal by seeking the divergence from the direction.

(3): In the aspect of (1) or (2) above, the movement model reflects the virtual force acting on each of the plurality of traffic participants, and the traffic participants in each future step. It simulates the movement process of the vehicle, and further includes a force estimation unit that estimates the virtual force based on the surrounding environment of the plurality of traffic participants.

(4): In the aspect of (3) above, in the force estimation unit, the influencing factor that exerts the virtual force on each of the plurality of traffic participants is centered on the front direction of each traffic participant. It is limited to the influencing factors existing in the fan-shaped range, and the influencing factors outside the range are assumed to exert no force, and the virtual force is estimated.

(5): In the aspect of (3) or (4) above, the virtual force estimated by the force estimation unit includes a force for the traffic participant himself to accelerate or decelerate in an attempt to move at a desired speed. It further includes a desired speed estimation unit that estimates the desired speed based on the history of the past positions of the traffic participants.

(6): In any one of the above (3) to (5), the virtual force estimated by the force estimation unit includes a force that repels each other among the traffic participants, and the force. The estimation unit estimates the traffic participants forming a group among the plurality of traffic participants, and among the traffic participants presumed to belong to the same group, the repulsive forces against each other are exerted in the same group. It is smaller than among traffic participants who are not presumed to belong to.

(7): The vehicle control device according to another aspect of the present invention includes the future behavior estimation device according to any one of (1) to (6) above, and the plurality of vehicle control devices estimated by the future behavior estimation device. It is provided with a driving control unit that controls the running of the vehicle based on the future behavior of each of the traffic participants.

(8): The future behavior estimation method according to another aspect of the present invention is a future behavior estimation method executed by using a computer, and recognizes the positions of a plurality of traffic participants and the result of the recognition. Based on, a movement model is used to determine a tentative goal for each of the plurality of traffic participants to reach in the future and to estimate the future behavior of each of the plurality of traffic participants. , Each of the plurality of traffic participants simulates the movement process toward the determined tentative goal.

(9): In the program according to another aspect of the present invention, the computer recognizes the positions of a plurality of traffic participants, and based on the result of the recognition, each of the plurality of traffic participants will be in the future. In order to determine the tentative goal to be reached and to estimate the future behavior of each of the plurality of traffic participants, each of the plurality of traffic participants is said to have been determined tentatively using a movement model. It simulates the process of moving toward the goal and executes it.

(10): In the storage medium according to another aspect of the present invention, the computer recognizes the positions of a plurality of traffic participants, and based on the result of the recognition, each of the plurality of traffic participants will be in the future. Each of the plurality of traffic participants determines the tentative goal using a movement model in order to determine the tentative goal to reach and to estimate the future behavior of each of the plurality of traffic participants. It is a non-temporary storage medium that stores a program that simulates the movement process toward a temporary goal determined by the department and executes it.

According to the above aspects (1) to (10), it is possible to make a prediction according to an actual traffic scene.

It is a block diagram of the vehicle system 1 using the future behavior estimation device and the vehicle control device which concerns on embodiment. It is a functional block diagram of the 1st control unit 120 and the 2nd control unit 180. It is a figure which shows typically the situation around the own vehicle M recognized by the traffic participant monitoring unit 140. It is a figure for demonstrating the content of processing by a tentative goal selection part 142B. It is a figure for demonstrating the processing by a desired speed estimation unit 144. It is a figure for demonstrating the process which obtains the force of a local frame. It is a flowchart which shows an example of the flow of the process executed by the automatic operation control device 100. It is a figure which shows an example of the hardware composition of the automatic operation control device 100 of an embodiment.

Hereinafter, the future behavior estimation device, the vehicle control device, the future behavior estimation method, and the embodiment of the program of the present invention will be described with reference to the drawings.

[overall structure]
FIG. 1 is a configuration diagram of a vehicle system 1 using a future behavior estimation device and a vehicle control device according to an embodiment. The vehicle on which the vehicle system 1 is mounted is, for example, a vehicle such as two wheels, three wheels, or four wheels, and the drive source thereof is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates by using the electric power generated by the generator connected to the internal combustion engine or the electric power generated by the secondary battery or the fuel cell.

The vehicle system 1 includes, for example, a camera 10, a radar device 12, a LIDAR (Light Detection and Ranging) 14, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, and a vehicle sensor 40. , A navigation device 50, an MPU (Map Positioning Unit) 60, a driving operator 80, an automatic driving control device 100, a traveling driving force output device 200, a braking device 210, and a steering device 220. These devices and devices are connected to each other by a multiplex communication line such as a CAN (Controller Area Network) communication line, a serial communication line, a wireless communication network, or the like. The configuration shown in FIG. 1 is merely an example, and a part of the configuration may be omitted or another configuration may be added.

The camera 10 is, for example, a digital camera using a solid-state image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 is attached to an arbitrary position of the vehicle on which the vehicle system 1 is mounted (hereinafter, the own vehicle M). When photographing the front, the camera 10 is attached to the upper part of the front windshield, the back surface of the rearview mirror, and the like. The camera 10 periodically and repeatedly images the periphery of the own vehicle M, for example. The camera 10 may be a stereo camera.

The radar device 12 radiates radio waves such as millimeter waves around the own vehicle M, and detects radio waves (reflected waves) reflected by the object to detect at least the position (distance and orientation) of the object. The radar device 12 is attached to an arbitrary position of the own vehicle M. The radar device 12 may detect the position and velocity of the object by the FM-CW (Frequency Modulated Continuous Wave) method.

The LIDAR 14 irradiates the periphery of the own vehicle M with light (or an electromagnetic wave having a wavelength close to that of light) and measures the scattered light. The LIDAR 14 detects the distance to the target based on the time from light emission to light reception. The emitted light is, for example, a pulsed laser beam. The LIDAR 14 is attached to an arbitrary position on the own vehicle M.

The object recognition device 16 performs sensor fusion processing on the detection results of a part or all of the camera 10, the radar device 12, and the LIDAR 14, and recognizes the position, type, speed, and the like of the object. The object recognition device 16 outputs the recognition result to the automatic operation control device 100. The object recognition device 16 may output the detection results of the camera 10, the radar device 12, and the LIDAR 14 to the automatic driving control device 100 as they are. The object recognition device 16 may be omitted from the vehicle system 1.

The communication device 20 communicates with another vehicle existing in the vicinity of the own vehicle M by using, for example, a cellular network, a Wi-Fi network, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), or wirelessly. Communicates with various server devices via the base station.

The HMI 30 presents various information to the occupants of the own vehicle M and accepts input operations by the occupants. The HMI 30 includes various display devices, speakers, buzzers, touch panels, switches, keys and the like.

The vehicle sensor 40 includes a vehicle speed sensor that detects the speed of the own vehicle M, an acceleration sensor that detects the acceleration, a yaw rate sensor that detects the angular velocity around the vertical axis, an orientation sensor that detects the direction of the own vehicle M, and the like.

The navigation device 50 includes, for example, a GNSS (Global Navigation Satellite System) receiver 51, a navigation HMI 52, and a routing unit 53. The navigation device 50 holds the first map information 54 in a storage device such as an HDD (Hard Disk Drive) or a flash memory. The GNSS receiver 51 identifies the position of the own vehicle M based on the signal received from the GNSS satellite. The position of the own vehicle M may be specified or complemented by an INS (Inertial Navigation System) using the output of the vehicle sensor 40. The navigation HMI 52 includes a display device, a speaker, a touch panel, keys, and the like. The navigation HMI 52 may be partially or wholly shared with the above-mentioned HMI 30. The route determination unit 53, for example, has a route from the position of the own vehicle M (or an arbitrary position input) specified by the GNSS receiver 51 to the destination input by the occupant using the navigation HMI 52 (hereinafter, hereafter). The route on the map) is determined with reference to the first map information 54. The first map information 54 is, for example, information in which a road shape is expressed by a link indicating a road and a node connected by the link. The first map information 54 may include road curvature, POI (Point Of Interest) information, and the like. The route on the map is output to MPU60. The navigation device 50 may provide route guidance using the navigation HMI 52 based on the route on the map. The navigation device 50 may be realized by, for example, the function of a terminal device such as a smartphone or a tablet terminal owned by an occupant. The navigation device 50 may transmit the current position and the destination to the navigation server via the communication device 20 and acquire a route equivalent to the route on the map from the navigation server.

The MPU 60 includes, for example, a recommended lane determination unit 61, and holds the second map information 62 in a storage device such as an HDD or a flash memory. The recommended lane determination unit 61 divides the route on the map provided by the navigation device 50 into a plurality of blocks (for example, divides the route every 100 [m] with respect to the vehicle traveling direction), and refers to the second map information 62. Determine the recommended lane for each block. The recommended lane determination unit 61 determines which lane to drive from the left. When a branch point exists on the route on the map, the recommended lane determination unit 61 determines the recommended lane so that the own vehicle M can travel on a reasonable route to proceed to the branch destination.

The second map information 62 is more accurate map information than the first map information 54. The second map information 62 includes, for example, information on the center of the lane, information on the boundary of the lane, and the like. Further, the second map information 62 may include road information, traffic regulation information, address information (address / zip code), facility information, telephone number information, and the like. The second map information 62 may be updated at any time by the communication device 20 communicating with another device.

The driving controller 80 includes, for example, an accelerator pedal, a brake pedal, a shift lever, a steering wheel, a deformed steering wheel, a joystick, and other controls. A sensor for detecting the amount of operation or the presence or absence of operation is attached to the operation operator 80, and the detection result is the automatic operation control device 100, or the traveling driving force output device 200, the brake device 210, and the steering device. It is output to a part or all of 220.

The automatic operation control device 100 includes, for example, a first control unit 120 and a second control unit 180. The first control unit 120 and the second control unit 180 are each realized by executing a program (software) by a hardware processor such as a CPU (Central Processing Unit). In addition, some or all of these components are hardware (circuits) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), and GPU (Graphics Processing Unit). It may be realized by the part; including circuitry), or it may be realized by the cooperation of software and hardware. The program may be stored in advance in a storage device (a storage device including a non-transient storage medium) such as an HDD or a flash memory of the automatic operation control device 100, or is removable such as a DVD or a CD-ROM. It is stored in a storage medium, and the storage medium (non-transient storage medium) may be installed in the HDD or flash memory of the automatic operation control device 100 by being attached to the device in the drive device.

FIG. 2 is a functional configuration diagram of the first control unit 120 and the second control unit 180. The first control unit 120 includes, for example, a recognition unit 130 and an action plan generation unit 160. The first control unit 120, for example, realizes a function by AI (Artificial Intelligence) and a function by a model given in advance in parallel. For example, the function of "recognizing an intersection" is executed in parallel with the recognition of an intersection by deep learning or the like and the recognition based on predetermined conditions (there are signals that can be pattern matched, road markings, etc.), and both are executed. It may be realized by scoring and comprehensively evaluating. This ensures the reliability of autonomous driving. A combination of the action plan generation unit 160 and the second control unit 180 is an example of the “operation control unit”.

The recognition unit 130 includes, for example, a vehicle position recognition unit 132, an object position recognition unit 134, and a traffic participant monitoring unit 140. A combination of the object position recognition unit 134 and the traffic participant monitoring unit 140 is an example of the "future behavior estimation device".

The own vehicle position recognition unit 132 has a lane (traveling lane) in which the own vehicle M is traveling and a traveling lane based on the information input from the camera 10, the radar device 12, and the LIDAR 14 via the object recognition device 16. Recognizes the position of the own vehicle M with respect to. For example, the own vehicle position recognition unit 132 around the own vehicle M recognized from the pattern of the road marking line (for example, the arrangement of the solid line and the broken line) obtained from the second map information 62 and the image captured by the camera 10. The driving lane is recognized by comparing with the pattern of the road marking line. The own vehicle position recognition unit 132 recognizes the traveling lane by recognizing not only the road lane marking but also the lane marking (road boundary) including the road lane marking, the shoulder, the curb, the median strip, the guardrail, and the like. You may. In this recognition, the position of the own vehicle M acquired from the navigation device 50 and the processing result by the INS may be added.

The own vehicle position recognition unit 132 recognizes the position and posture of the own vehicle M with respect to the traveling lane. In the own vehicle position recognition unit 132, for example, the deviation of the reference point of the own vehicle M (for example, the center of gravity and the center of the rear wheel axis) from the central portion of the traveling lane, and the traveling direction of the own vehicle M are such that the central portion of the traveling lane is determined. The angle formed with respect to the connected lanes may be recognized as the relative position and orientation of the own vehicle M with respect to the traveling lane. Instead, the own vehicle position recognition unit 132 sets the position of the reference point of the own vehicle M with respect to any side end portion (road lane marking line or road boundary) of the traveling lane, and the like relative to the traveling lane. It may be recognized as a position.

The object position recognition unit 134 performs processing such as a Kalman filter based on the information input from the camera 10, the radar device 12, and the LIDAR 14 via the object recognition device 16, thereby causing the object in the vicinity of the own vehicle M to be processed. Recognize position, type, velocity, acceleration, etc. The position of the object is recognized as, for example, a position on absolute coordinates with the representative point (center of gravity, center of drive axis, etc.) of the own vehicle M as the origin, and is used for control. The position of the object may be represented by a representative point such as the center of gravity or a corner of the object, or may be represented by a represented area. The "state" of an object may include acceleration or jerk of the object, or "behavioral state" (eg, whether or not the vehicle is changing lanes or is about to change lanes). The types of objects are classified into pedestrians, bicycles, vehicles, obstacles, and the like. Among the objects, those including pedestrians or pedestrians and bicycles are referred to as "traffic participants". Therefore, the object position recognition unit 134 recognizes the position of an object including a traffic participant.

The traffic participant monitoring unit 140 monitors the behavior of the traffic participant recognized by the object recognition unit. The configuration and operation of the traffic participant monitoring unit 140 will be described later.

In principle, the action plan generation unit 160 travels in the recommended lane determined by the recommended lane determination unit 61, and the own vehicle M automatically (driver) so as to be able to respond to the surrounding conditions of the own vehicle M. Generate a target track to run in the future (regardless of the operation of). "To be able to respond to the surrounding conditions of the own vehicle M" includes generating a target trajectory so as not to approach the future position of the traffic participant as much as possible. The target trajectory includes, for example, a velocity element. For example, the target track is expressed as a sequence of points (track points) to be reached by the own vehicle M. The track point is a point to be reached by the own vehicle M for each predetermined mileage (for example, about several [m]) along the road, and separately, for a predetermined sampling time (for example, about 0 comma [sec]). ) Target velocity and target acceleration are generated as part of the target trajectory. Further, the track point may be a position to be reached by the own vehicle M at the sampling time for each predetermined sampling time. In this case, the information of the target velocity and the target acceleration is expressed by the interval of the orbital points.

The second control unit 180 sets the traveling driving force output device 200, the braking device 210, and the steering device 220 so that the own vehicle M passes the target trajectory generated by the action plan generation unit 160 at the scheduled time. Control.

The second control unit 180 includes, for example, an acquisition unit 182, a speed control unit 184, and a steering control unit 186. The acquisition unit 182 acquires the information of the target trajectory (orbit point) generated by the action plan generation unit 160 and stores it in a memory (not shown). The speed control unit 184 controls the traveling driving force output device 200 or the braking device 210 based on the speed element associated with the target trajectory stored in the memory. The steering control unit 186 controls the steering device 220 according to the degree of bending of the target trajectory stored in the memory. The processing of the speed control unit 184 and the steering control unit 186 is realized by, for example, a combination of feedforward control and feedback control. As an example, the steering control unit 186 executes a combination of feedforward control according to the curvature of the road in front of the own vehicle M and feedback control based on the deviation from the target trajectory.

The traveling driving force output device 200 outputs a traveling driving force (torque) for traveling the vehicle to the drive wheels. The traveling driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, a transmission, and the like, and an ECU (Electronic Control Unit) that controls them. The ECU controls the above configuration according to the information input from the second control unit 180 or the information input from the operation operator 80.

The brake device 210 includes, for example, a brake caliper, a cylinder that transmits flood pressure to the brake caliper, an electric motor that generates flood pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to the information input from the second control unit 180 or the information input from the operation operator 80 so that the brake torque corresponding to the braking operation is output to each wheel. The brake device 210 may include, as a backup, a mechanism for transmitting the oil pressure generated by the operation of the brake pedal included in the operation operator 80 to the cylinder via the master cylinder. The brake device 210 is not limited to the configuration described above, and is an electronically controlled hydraulic brake device that controls an actuator according to information input from the second control unit 180 to transmit the oil pressure of the master cylinder to the cylinder. May be good.

The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor, for example, applies a force to the rack and pinion mechanism to change the direction of the steering wheel. The steering ECU drives the electric motor according to the information input from the second control unit 180 or the information input from the driving operator 80 to change the direction of the steering wheel.

[Estimation of future behavior]
The contents of the processing by the traffic participant monitoring unit 140 will be described below. FIG. 3 is a diagram schematically showing a situation around the own vehicle M recognized by the traffic participant monitoring unit 140. As shown in the figure, the conditions around the own vehicle M recognized by the traffic participant monitoring unit 140 include, for example, sidewalk SW, roadway RW, pedestrian crossing CR, building FC (specifically, its wall surface, entrance, etc.). There are obstacles such as OB. The traffic participant monitoring unit 140 recognizes the range they occupy in a coordinate system such as a horizontal coordinate system. Further, the traffic participant monitoring unit 140 recognizes the positions of the plurality of traffic participant TPs in time series based on the recognition result of the object position recognition unit 134. The traffic participant monitoring unit 140 performs the processing described below based on the results of these recognitions.

The traffic participant monitoring unit 140 includes, for example, a provisional goal determination unit 142, a desired speed estimation unit 144, a force estimation unit 146, and a simulation unit 148.

The provisional goal determination unit 142 determines a provisional goal, which is a destination for each of the plurality of traffic participant TPs to reach in the future. Temporary goal setting is repeated over time. The tentative goal will eventually approach the main goal. The tentative goal is a point that is predicted to be reached after the first period T1 (for example, 4 [sec]). The tentative goal is updated at each prediction step.

The provisional goal determination unit 142 includes, for example, a provisional goal candidate setting unit 142A and a provisional goal selection unit 142B.

The provisional goal candidate setting unit 142A has a history of the position of the traffic participant TP from the time when the second period T2 (for example, 2 [sec]) is traced back from the present, and the object recognition device 16 from the camera 10, the radar device 12, and the LIDAR 14. Based on the information input via the above and the information on the road structure recognized based on the map information, m-1 provisional goal candidates Gpc (p) (p = 1 to m-1) are set. For example, in the provisional goal candidate setting unit 142A, (1) the traffic participant TP keeps moving on the sidewalk SW, (2) the traffic participant TP crosses the pedestrian crossing CR, and (3) the traffic participant TP is the nearest facility. The provisional goal candidate Gpc (k) is determined based on a presumed scenario such as going to FC. For example, the provisional goal candidate setting unit 142A determines the provisional goal candidate Gpc (k) based on a discrete selection model (each selection includes a temporary goal selection). This discrete selection model is a set of possible selections (temporary) based on driving scenes classified by the content such as the vehicle M is at an intersection, has a pedestrian crossing, or is in a shopping district. Candidates for goals) are defined. The provisional goal candidate setting unit 142A is based on the existence of sidewalks, pedestrian crossings, intersections, etc. recognized by the object position recognition unit 134, and the positions of freeways, shopping streets, etc. recognized by the MPU 60, which discrete set (scenario). ) Is selected. In the example of the figure, the provisional goal candidate setting unit 142A sets one point of the sidewalk SW, which is ahead of the current movement direction of the traffic participant TP, as the provisional goal candidate Gpc (1) based on the scenario of (1). Based on the scenario of (2), the intersection of the sidewalk SW and the pedestrian crossing CR on the side opposite to the side where the traffic participant TP exists is set as the provisional goal candidate Gpc (2), and based on the scenario of (3), the facility FC Let the entrance of the tentative goal candidate Gpc (3). When the traffic participant TP is actually crossing the pedestrian crossing CR, the above two scenarios (2) or (4) turning back may be selected.

Further, the provisional goal candidate setting unit 142A extrapolates the prediction result of the Kalman filter processing in the object position recognition unit 134, and assumes that the traffic participant continues the current movement direction, and another provisional goal candidate Gpc (m). To set. As a result, the provisional goal candidate setting unit 142A sets a total of m provisional goal candidate Gpc (p).

The provisional goal selection unit 142B calculates the movement direction θ intended by the traffic participant TP, and obtains the reference goal position Gref using the movement direction θ. The moving direction θ is obtained by the equation (1). In the formula, w1, w2, w3, and w4 are coefficients. These coefficients may be fixed values or may be updated at any time by machine learning or the like. For example, the temporary goal selection unit 142B acquires information on the orientation and face orientation of the upper body by inputting the image captured by the camera 10 into the trained model that derives the orientation and face orientation of the upper body. Instead, the temporary goal selection unit 142B may acquire information on the orientation of the upper body and the orientation of the face by performing some rule-based processing on the image captured by the camera 10. For example, the temporary goal selection unit 142B calculates the face orientation by performing calculations using the bias of the position occupied by the featured portion such as the bridge of the nose in the face region and the direction of the line of sight based on the position of the black eye as input parameters. May be good. The recognition speed vector is information recognized by the object position recognition unit 134. The effect of the road structure is, for example, a correction term for canceling the component when the orientation of the upper body, the orientation of the face, and the recognition speed vector are directed in the direction deviating from the pedestrian crossing or the sidewalk.

Movement direction θ = w1 × (direction of upper body) + w2 × (direction of face) + w3 × (recognition speed vector) + w4 × (effect of road structure)… (1)

Instead of the above, the temporary goal selection unit 142B inputs the orientation of the upper body, the orientation of the face, the recognition speed vector, the influence of the road structure, etc., and the orientation of the upper body and the face with respect to the model that outputs the movement direction θ. The moving direction θ may be acquired by inputting the direction, the recognition speed vector, the influence of the road structure, and the like.

The provisional goal selection unit 142B obtains the reference goal position Greff on the assumption that the traffic participant TP moves in the movement direction θ at a predetermined speed during the first period T1. For example, the speed of the traffic participant TP acquired before a predetermined cycle (for example, two cycles) is used as the predetermined speed. Then, the provisional goal selection unit 142B inputs each provisional goal candidate Gpc (p) into the movement simulation model, and obtains the estimated position of the traffic participant TP at a plurality of time points during the first period T1 in the future. The movement simulation model is, for example, the same as the model used for the simulation executed by the simulation unit 148, which will be described later. Then, the provisional goal selection unit 142B connects the current position of the traffic participant TP to the reference goal position Gref with a straight line, a curve corresponding to the road structure, a polygonal line, or the like, and each provisional goal candidate Gpc (p). The provisional goal candidate Gpc (p) having the smallest deviation is determined as the provisional goal Gp by comparing with the estimated positions of the traffic participant TPs at a plurality of time points. FIG. 4 is a diagram for explaining the content of processing by the provisional goal selection unit 142B. For example, the provisional goal selection unit 142B squares the distance between the reference line and the estimated positions P1 to Pu of the traffic participant TP at a plurality of time points (for example, _{1 to u) calculated for each provisional goal candidate Gpc (p).} total (sum of squares) Σ _q ⁼ 1 u _{d q} is the smallest was provisional goal candidates Gpc (p) of the value, determined in the provisional goal Gp.

Instead of the above, the tentative goal determination unit 142 uses the surrounding situation of the own vehicle M as context information, and uses the SVM (Support Vector Machine) model and the model by the method of multivariate regression analysis to tentatively use the traffic participant TP. The goal Gp may be determined. This model is learned based on the behavior of the traffic participant TP collected in the actual traffic phase. The context information includes, for example, the distance between the pedestrian crossing and the traffic participant TP, the influence of the road structure, the positional relationship with the vehicle, the movement of the head of the traffic participant TP, and the like.

The desired speed estimation unit 144 estimates the desired speed vector → vi0 of the traffic participant TPi by using the history of the past position of the traffic participant TP. Hereinafter, "→" indicates a vector. Further, the i-th traffic participant TP recognized by the object position recognition unit 134 is referred to as a traffic participant TPi. i is the identification information of the traffic participant TP. In order to obtain the magnitude vi0 of the desired speed, the desired speed estimation unit 144 obtains a weighted moving average of the speeds in a plurality of steps for the past third period T3 (for example, 2 [sec]). FIG. 5 is a diagram for explaining processing by the desired speed estimation unit 144. The desired velocity estimation unit 144 increases, for example, the weight of the fourth period T4 (for example, 0.5 [sec]) near the end of the third period T3 (close to the present) as compared with the other periods. May be good. The desired speed estimation unit 144 sets the direction of the desired speed vector → vi0 as the direction from the current position of the traffic participant TPi toward the provisional goal GPi.

The force estimation unit 146 estimates a virtual force acting on each of the plurality of traffic participant TPs based on the surrounding environment of the plurality of traffic participant TPs. The force estimation unit 146 estimates, for example, the force received from itself, the force acting on each other, and the force exerted by an object for each traffic participant TP. These forces are virtual forces to be estimated. The force estimation unit 146 estimates the spontaneous force F1i, the social force F2i, and the physical force F3i, and totals them to obtain the force acting on the traffic participant TPi (hereinafter, "virtual" is omitted). Hereinafter, the contents of the calculation and processing performed by the force estimation unit 146 using mathematical formulas will be described. Equation (2) shows the force calculated by the force estimation unit 146 for the traffic participant TP. In the equation (2), the “→” indicating the vector is omitted. In the formula, k includes traffic participant TPs other than traffic participant TPi, vehicles, obstacles, walls, crossing conditions (the fact that they are crossing the roadway), and sidewalk boundaries (hereinafter, these are affected). It is assumed that there are n (referred to as factors). In the force estimation unit 146, the influencing factors that exert a force on the traffic participant TPi are limited to the influencing factors existing in the fan-shaped range centered on the front direction of the traffic participant TPi, and the influencing factors outside the range do not exert the force. Assuming that, the following calculation is performed. In equation (2), F ^motivation is a spontaneous force, F ^pedestrian is a force exerted by a pedestrian who ^{is an influencing factor, F vehicle is a force exerted by a vehicle which} is an influencing factor, and F ^obstacle is an influencing factor. F wall is the force exerted by an obstacle, F ^{wall is the force exerted by the wall which} is the influencing factor, F ^crosswalk is the force exerted by the crossing state which is the influencing factor, and F ^sidewalk is the force exerted by the sidewalk boundary which is the influencing factor. It is the power that comes from.

The spontaneous force F1i is a force that the traffic participant TPi acts on itself in order for the traffic participant TPi to move at a desired speed. The force estimation unit 146 calculates the spontaneous force F1i based on the desired velocity vector → vi0. The spontaneous force F1i expresses in the dimension of force that the traffic participant TPi accelerates or decelerates as a result of trying to move with the desired speed vector → vi0. Spontaneous force → F1i is expressed by, for example, equation (3). In the equation, → vi is the current velocity vector of the traffic participant TPi, and τ is the time required to match the velocity vector → vi with the desired velocity vector → vi0.

F1 (F ^motivation ) = (1 / τ) ・ (→ vi0− → vi)… (3)

Each of the social force F2i and the physical force F3i is estimated by obtaining the sum (or weighted sum) of the three components of the repulsive force, the compressive force, and the frictional force exerted by the influential factors.

In equation (2), F _k ^{pedestrian is the force exerted by the pedestrian, which} is the kth influencing factor, F _k ^{wall is the force exerted by the wall, which} is the kth influencing factor, and F _k ^obstacle is the force exerted. It is the force exerted by the obstacle, which is the kth influencing factor. These are represented by the equation (4). In the formula, repulsion represents the repulsive force, compression represents the compression force, and friction represents the frictional force.

^Each term of F _k pedestrian is expressed by the formula (5). In the formula, Aped, Bped, Cped, and Kped are conforming values obtained by experiments and the like. r _ik is the sum of the radii of the private space preset for each of the traffic participant TPi and the influencing factor k. The private space is a space in which the traffic participant TP behaves so as to prevent others from entering the space. If the influencing factor is not a traffic participant, a pseudo private space is set. _dik is the Euclidean distance between the traffic participant TPi and the influencing factor k. n _ik is represented by the formula (6), and _tic is represented by the formula (7). In equation (7), (1) represents an x element and (2) represents a y element. Φ is the angle of the difference between the traffic participant TPi and the influencing factor k in the moving direction (direction of the velocity vector). Further, dv _ik is a difference vector between the speed vector of the traffic participant TPi and the speed vector of the influencing factor k. ^Each term of F _k pedestrian may be replaced with zero if it becomes negative. Bped is a coefficient of magnitude of force generated by the Exp function. First, define the minimum and maximum values of the force generated by the social forces of each element, by adjusting the Bped, it is possible to control the rate of change in society force with respect to a change in r _ik -d _ik. In addition, Bped acts in the same manner in equations (8) and (9).

Regarding Aped in the equation (5), the force estimation unit 146 estimates in advance whether or not the traffic participant TPi and the influencing factor k (in this case, other traffic participants) form a group, and the same group. The value may be smaller (near zero or even negative) among traffic participants presumed to belong to the same group as compared to those not presumed to belong to the same group. This is because the traffic participants forming a group often move while maintaining a state in which the distance is within a certain distance. The force estimation unit 146 is used when the positions of the two traffic participants in the past fixed period are within the predetermined range and the difference in the moving direction (velocity vector direction) in the fixed period is less than the threshold value. It is presumed that human traffic participants form a group, and in the case of three or more people, a ripple estimation may be made in the same manner.

Bicycles may be treated as ^pedestrians and included in the _{F k pedestrian.} However, the radius of the private space shall be 1.5 times that of a pedestrian.

^Each term of F _k obstacle is represented by the equation (8). In the formula, Aobs, Cobs, and Kobs are conforming values obtained by experiments and the like. ^Each term of F _k obstacle may be replaced with zero if it becomes negative.

^Each term of F _k wall is represented by the equation (9). In the formula, Awall, Cwall, and Kwall are conforming values obtained by experiments and the like. ^Each term of F _k wall may be replaced with zero if it becomes negative. When obtaining the n _ik and t _ik for walls, the force estimating section 146, for example, the point of intersection to move the traffic participants TPi obtained by extrapolation may be used as the reference.

The F _k ^sidewalk is the force exerted from the wall, which is the kth influencing factor. The F _k ^sidewalk is represented by the equation (10). In the formula, K _side is a conforming value obtained by an experiment or the like. n _sk is a unit vector of the direction in which the traffic participant TPi should be moved in order to fit in the sidewalk. d _{safe_s} is the clearance distance set when the boundary of the sidewalk is not indicated by a physical boundary such as a curb or a guardrail. F _k ^crossswalk is the force exerted from the kth influential factor, the crossing state. F _k ^{crosss walk} is represented by the equation (11). In the formula, K _cross is a conforming value obtained by an experiment or the like. n _ck is a unit vector from the position of the traffic participant TPi toward the sidewalk closer to it. d _{safe_c} is the clearance distance set when the boundary of the pedestrian crossing is not indicated by a physical boundary such as a curb or a guardrail. F _k vehicle is the force exerted by the ^{vehicle, which is the kth influencing factor.} The F _k ^vehicle is represented by the equation (12). However, the formula (12) is applied to a moving vehicle, and a parked or stopped vehicle is treated as an obstacle, and the formula (8) is applied. In the formula, n _kcar is a unit vector toward the sidewalk closer to the position of the traffic participant when the traffic participant TPi is on the roadway. TTC is a time to collsion, that is, a collision time calculated between the position of a traffic participant projected on the roadway and the position of the vehicle. h is a parameter for normalizing the TTC, and is set to the maximum value of the assumed TTC. h is a parameter (user-set parameter) experimentally obtained by repeating data analysis and simulation.

When the provisional goal Gp, the desired speed → vi0, and various forces are obtained in this way, the simulation unit 148 determines the provisional goal Gp for all the traffic participants and vehicles recognized by the object position recognition unit 134. , Desired velocity → vi0, and various forces are input to the movement model, and the position at a plurality of steps (time points) during the first period T1 in the future is obtained by simulation. During the second period T2, the provisional goal Gp and the desired speed → vi0 are fixed. When the second period T2 elapses, the simulation unit 148 selects the provisional goal Gp in the same manner as the provisional goal candidate setting unit 142A and the provisional goal selection unit 142B, and the traffic participants assume that they move toward the provisional goal Gp. Simulate movement. In this simulation, the traffic participant moves toward the temporary goal Gp by the shortest path if no force is applied, and if the force is applied, for example, the direction toward the temporary goal Gp and the direction of the force. , Moves in the combined direction at a ratio according to the magnitude of the force. That is, the simulation unit 148 uses a movement model to estimate the future behavior of each of the plurality of traffic participants, and each of the plurality of traffic participants has a provisional goal determined by the provisional goal determination unit 142. Simulate the process of movement toward Gp. The moving model is, for example, the human locomotion model described in Non-Patent Document 2. Not limited to this, the movement model may be any model as long as it estimates the position at a plurality of future time points by simulation based on the above input information. At this time, the simulation unit 148 may perform the calculation by converting the direction in which the force acting due to the approach of another person acts on the traffic participant as it is, but converting it into the force of the local frame. FIG. 6 is a diagram for explaining a process of obtaining the force of the local frame. In the figure, → vi is the velocity vector of the traffic participant TPi, → vk is the velocity vector of the influencing factor k, and → F _k is the force exerted by the influencing factor k on the traffic participant TPi. In this case, since → vi and → F _k are oriented in substantially opposite directions, the simulation unit 148 wraps around the _{force → F k instead of acting the force → F k as it} is on the traffic participant TPi. It may be corrected in the direction (for example, → Fk * in the figure). In this way, it is possible to suppress the simulation result that the traffic participants come and go unnaturally.

FIG. 7 is a flowchart showing an example of the flow of processing executed by the automatic operation control device 100. In this flowchart, attention is paid to the part of predicting future behavior of traffic participants, and the description of processing such as own vehicle position recognition is omitted.

First, the object position recognition unit 134 recognizes the position of an object including a traffic participant (step S100).

Next, the provisional goal determination unit 142 and the desired speed estimation unit 144 execute the processes of steps S102 to S108 for each traffic participant (step S102). First, the provisional goal candidate setting unit 142A sets the provisional goal candidate Gpc (p), and the provisional goal selection unit 142B executes a simulation for each provisional goal candidate Gpc (p) (step S104). The provisional goal selection unit 142B selects Gpc (p) having the smallest deviation from the route to the reference goal position Gref as the provisional goal (step S106). Further, the desired speed estimation unit 144 estimates the desired speed (step S108).

Next, the force estimation unit 146 estimates the force acting on each traffic participant based on the current position recognized in step S100 or the future position obtained in the next step S112 (step S110). The simulation unit 148 performs a simulation based on the provisional goal Gp and the force using the movement model, and obtains the future position of the traffic participant after one cycle (step S112). The traffic participant monitoring unit 140 determines whether or not the processes of steps S110 and S112 have been executed for a predetermined number of cycles (step S114). If it is determined that the process has not been executed for a predetermined number of cycles, the process is returned to step S110. When it is determined that the action plan has been executed for a predetermined number of cycles, the action plan generation unit 160 controls the vehicle based on the current position and the future position of the traffic participants and the like (step S116). For example, when determining the future action of the own vehicle M, the action plan generation unit 160 sets a risk area centered on the position of a traffic participant or the like at a future time point, and suppresses the approach to the risk area. Generate a target trajectory.

[Hardware configuration]
FIG. 8 is a diagram showing an example of the hardware configuration of the automatic operation control device 100 of the embodiment. As shown in the figure, the automatic operation control device 100 includes a communication controller 100-1, a CPU 100-2, a RAM (Random Access Memory) 100-3 used as a working memory, a ROM (Read Only Memory) for storing a boot program, and the like. The configuration is such that 100-4, a storage device 100-5 such as a flash memory or an HDD (Hard Disk Drive), a drive device 100-6, and the like are connected to each other by an internal bus or a dedicated communication line. The communication controller 100-1 communicates with a component other than the automatic operation control device 100. The storage device 100-5 stores a program 100-5a executed by the CPU 100-2. This program is expanded into RAM 100-3 by a DMA (Direct Memory Access) controller (not shown) or the like, and is executed by CPU 100-2. As a result, a part or all of the recognition unit 130, the action plan generation unit 160, and the second control unit 180 is realized.

According to the embodiment described above, each of the plurality of traffic participants reaches the future based on the recognition result of the object position recognition unit (134) that recognizes the positions of the plurality of traffic participants and the object position recognition unit. A tentative goal determination unit (142) that determines the tentative goal to be attempted, and a movement model are used to estimate the future behavior of each of the plurality of traffic participants, and each of the plurality of traffic participants has a tentative goal. By providing a simulation unit (148) that simulates the movement process toward the temporary goal determined by the determination unit, it is possible to make a prediction according to an actual traffic scene.

The embodiment described above can be expressed as follows.
A storage device that stores programs and
With a hardware processor,
When the hardware processor executes a program stored in the storage device,
Recognize the location of multiple traffic participants and
Based on the result of the recognition, each of the plurality of traffic participants determines a tentative goal to reach in the future.
In order to estimate the future behavior of each of the plurality of traffic participants, a movement model is used to simulate the movement process of each of the plurality of traffic participants toward the provisional goal determined by the provisional goal determination unit. To
A future behavior estimator configured to.

Although the embodiments for carrying out the present invention have been described above using the embodiments, the present invention is not limited to these embodiments, and various modifications and substitutions are made without departing from the gist of the present invention. Can be added.

1 Vehicle system 10 Camera 12 Radar device 14 LIDAR
16 Object recognition device 100 Automatic operation control device 120 First control unit 130 Recognition unit 132 Own vehicle position recognition unit 134 Object position recognition unit 140 Traffic participant monitoring unit 142 Temporary goal determination unit 142A Temporary goal candidate setting unit 142B Temporary goal selection unit 144 Desired speed estimation unit 146 Force estimation unit 148 Simulation unit 160 Action plan generation unit 180 Second control unit

Claims (9)

An object position recognition unit that recognizes the positions of multiple traffic participants,
Based on the recognition result of the object position recognition unit, the provisional goal determination unit, which determines the provisional goal that each of the plurality of traffic participants intends to reach in the future,
In order to estimate the future behavior of each of the plurality of traffic participants, a movement model is used to simulate the movement process of each of the plurality of traffic participants toward the provisional goal determined by the provisional goal determination unit. The simulation part to be operated and
A future behavior estimation device equipped with. The provisional goal determination unit determines each of the plurality of traffic participants.
A plurality of provisional goal candidates are set based on the history of the positions of the plurality of traffic participants recognized by the object position recognition unit and the information indicating the road structure.
The deviation between the result of the simulation in which the traffic participant heads for each of the plurality of provisional goal candidates and the movement direction based on the posture of the traffic participant is obtained, and the provisional goal candidate having the smallest deviation is selected as the provisional goal candidate. Determine as a goal,
The future behavior estimation device according to claim 1. The movement model simulates the movement process of the traffic participants in each future step by reflecting the virtual force acting on each of the plurality of traffic participants.
A force estimation unit that estimates the virtual force based on the surrounding environment of the plurality of traffic participants is further provided.
The future behavior estimation device according to claim 1 or 2. In the force estimation unit, the influencing factors that exert the virtual force on each of the plurality of traffic participants are limited to the influencing factors existing in a fan-shaped range centered on the front direction of each traffic participant. The hypothetical force is estimated assuming that the influencing factors outside the range do not exert the force.
The future behavior estimation device according to claim 3. The virtual force estimated by the force estimation unit includes a force for the traffic participant himself to accelerate or decelerate in an attempt to move at a desired speed.
Further provided is a desired speed estimation unit that estimates the desired speed based on the history of the past positions of the traffic participants.
The future behavior estimation device according to claim 3 or 4. The virtual force estimated by the force estimation unit includes a force that repels each other among the traffic participants.
The force estimation unit estimates the traffic participants forming a group among the plurality of traffic participants, and among the traffic participants presumed to belong to the same group, the forces that repel each other are exerted. Smaller than among traffic participants who are not presumed to belong to the same group,
The future behavior estimation device according to any one of claims 3 to 5. The future behavior estimation device according to any one of claims 1 to 6 and
A driving control unit that controls the running of the vehicle based on the future behavior of each of the plurality of traffic participants estimated by the future behavior estimation device.
Vehicle control device. A method of estimating future behavior that is executed using a computer.
Recognizing the location of multiple traffic participants and
Based on the result of the recognition, each of the plurality of traffic participants decides a tentative goal to reach in the future.
In order to estimate the future behavior of each of the plurality of traffic participants, a movement model is used to simulate the movement process of each of the plurality of traffic participants toward the determined tentative goal.
Future behavior estimation method. On the computer
Recognizing the location of multiple traffic participants and
Based on the result of the recognition, each of the plurality of traffic participants decides a tentative goal to reach in the future.
In order to estimate the future behavior of each of the plurality of traffic participants, a movement model is used to simulate the movement process of each of the plurality of traffic participants toward the determined tentative goal.
A program that executes.

Images