JP7434714B2 - Vehicle computing system - Google Patents

Description

The technology disclosed herein relates to a vehicle calculation system used for automatic driving of a vehicle, for example.

Patent Document 1 discloses a system that controls a plurality of on-vehicle devices such as an engine and a steering wheel mounted on a vehicle. This control system has a hierarchical configuration of an integrated control section, a domain control section, and a device control section in order to control a plurality of in-vehicle devices.

JP2017-061278A

In order to achieve high-precision autonomous driving, vehicle motion must be controlled through comprehensive judgment based not only on the environment around the vehicle, but also on various information such as the driver's condition and the vehicle's condition. It won't happen. To do this, it is necessary to process enormous amounts of data from cameras, sensors, external networks, etc. at high speed, determine the optimal movement of the vehicle at each moment, and operate each actuator. It is necessary to build a system.

The technology disclosed herein has been developed in view of this point, and its purpose is to provide a vehicular calculation system for realizing highly accurate automatic driving.

Specifically, the technology disclosed herein is a vehicle calculation system that is installed in a vehicle and executes calculations to control the running of the vehicle, and which receives the output of a means for acquiring information about the environment outside the vehicle and performs calculations on the road. and an external environment estimating unit that estimates an external environment including obstacles, and a driving route for the vehicle that avoids the estimated obstacles on the estimated road based on the output of the external environment estimating unit. A route generation unit, and a planar movement of the vehicle when traveling along the travel route generated by the route generation unit based on the output of the route generation unit, and a vertical posture change of the vehicle body. a target motion determination unit that determines a target motion; an energy management unit that calculates driving force, braking force, and steering angle to realize the target motion determined by the target motion determination unit; and a measurement unit that measures the state of the driver. and a driver condition estimating section that receives the output of the means for estimating the driver's condition including the health condition, and the route generating section generates a driving route that matches the driver condition estimated by the driver condition estimating section. Furthermore, the route generation unit uses the data representing the driver's emotion outputted from the driver state estimation unit to determine the driving route from a plurality of route candidates, with reference to the degree of change in the behavior of the vehicle in each route candidate. Select. Further, the driver state estimating unit includes a human body model that outputs an expected subjective feeling of the occupant when the movement of the vehicle body is input, and the route generating unit includes a human body model that outputs an expected subjective feeling of the occupant based on the human body model. Remove driving routes that involve vehicle behavior from selection.

According to this configuration, in the vehicle calculation system, the vehicle exterior environment estimating section receives the output of a means for acquiring information on the vehicle exterior environment, such as a camera or radar mounted on the vehicle, and estimates the vehicle exterior environment including the road and obstacles. presume. The route generation unit generates a travel route for the vehicle that avoids the estimated obstacles on the estimated road. The target motion determination unit determines a target motion of the vehicle, which includes a planar motion and a vertical posture change of the vehicle body, when traveling along the travel route generated by the route generation unit. The energy management section calculates the driving force, braking force, and steering angle to achieve the target motion. Thereby, the vehicle calculation system can control the movement of the vehicle with high precision according to the environment around the vehicle. Furthermore, the route generation section determines a travel route that matches the driver state estimated by the driver state estimation section. With this, it is possible to make a comprehensive judgment regarding route generation based not only on the environment around the vehicle but also on the driver's condition.

In the above-described vehicle calculation system, the energy management section may generate operation signals for each actuator to generate the driving force, braking force, and steering angle.

With this configuration, the vehicle arithmetic system can use the energy management section to generate operation signals for each actuator according to the output of the target motion determination section.

According to the present disclosure, the vehicle computing system can control the motion of the vehicle with high precision according to the environment around the vehicle.

Functional configuration of vehicle calculation system according to embodiment Specific examples of vehicle actuators and their control devices

FIG. 1 is a block diagram showing the functional configuration of a vehicle calculation system according to an embodiment. As shown in FIG. 1, the vehicle computing system includes an information processing unit 1 mounted on a vehicle. The information processing unit 1 receives various signals and data related to the vehicle, and based on these signals and data, performs arithmetic processing using, for example, a trained model generated by deep learning, and determines the target of the vehicle. Decide on exercise. Then, based on the determined target motion, operation signals for each actuator 200 of the vehicle are generated.

The functions of the information processing unit 1 may be realized by a single chip, or may be realized by a plurality of chips. When implemented using multiple chips, the multiple chips may be mounted on a common substrate or on different substrates. However, in this embodiment, the information processing unit 1 is configured within a single housing.

<Example of input for information processing unit>
The information processing unit 1 receives outputs from cameras, sensors, and switches mounted on the vehicle, as well as signals and data from outside the vehicle. For example, the output of a camera 101, radar 102, etc. mounted on the vehicle, which are examples of means for acquiring information on the environment outside the vehicle, the signal 111 of a positioning system such as GPS, the signal 111 of a positioning system such as GPS, the signal 111 for navigation transmitted from a network outside the vehicle, etc. Data 112, the output of a camera 120 installed in the vehicle interior, which is an example of means for acquiring driver information, the output of sensors 130 that detect the behavior of the vehicle, and the sensors 140 that detect driver operations. Let the output of be the input.

A camera 101 mounted on a vehicle captures images of the surroundings of the vehicle and outputs captured image data. A radar 102 mounted on a vehicle transmits radio waves toward the surroundings of the vehicle and receives reflected waves from objects. Then, the radar 102 measures the distance from the vehicle to the object and the relative speed of the object with respect to the vehicle based on the transmitted wave and the received wave. Note that other means for acquiring information about the environment outside the vehicle include, for example, a laser radar and an ultrasonic sensor.

In addition to the camera 120 installed in the vehicle interior, means for acquiring information about the driver include, for example, biological information sensors such as a skin temperature sensor, a heartbeat sensor, a blood flow sensor, and a sweat sensor.

Examples of the sensors 130 that detect vehicle behavior include a vehicle speed sensor, an acceleration sensor, and a yaw rate sensor. Examples of the sensors 140 that detect the driver's operation include a steering angle sensor, an accelerator sensor, and a brake sensor.

<Example of output of information processing unit>
The information processing unit 1 outputs an operation signal to a control device that controls each actuator 200 of the vehicle. Examples of the control device include an engine control device, a brake control device, and a steering control device. Each control device is realized as, for example, an ECU (Electronic Control Unit), and the information processing unit 1 and the ECU are connected via an in-vehicle network such as a CAN (Controller Area Network).

FIG. 2 is a diagram showing a specific example of the actuator. In FIG. 2, 201 is an engine, 202 is a transmission, 203 is a brake, and 204 is a steering wheel. A power train ECU 211, a DSC (Dynamic Stability Control) microcomputer 212, a brake microcomputer 213, and an EPAS (Electric Power Assist Steering) microcomputer 214 are examples of control devices. The information processing unit 1 calculates the driving force, braking force, and steering angle of the vehicle to realize the determined target motion. For example, the power train ECU 211 controls the ignition timing and fuel injection amount of the engine 201 according to the calculated driving force. Alternatively, the EPAS microcomputer 214 controls the steering of the steering wheel 204 according to the calculated steering angle.

Note that examples of control devices that control other actuators include a body microcomputer 221 that controls bodies such as airbags and doors, and a driver support HMI (Human Machine Interface) unit 223 that controls a vehicle interior display 222. There is.

The functional configuration of the information processing unit 1 shown in FIG. 1 will be described in detail. The information processing unit 1 executes so-called model predictive control (MPC) in processing such as route generation, for example. Simply put, model predictive control involves having an evaluation function that takes multivariate input and producing multivariate output, and solving this using a convex function (multivariate analysis: mathematics for efficiently solving multivariate problems). The method is to extract a good balance. The relational expression (called a model) for obtaining multivariate output from multivariate input is first created by the designer based on the physical phenomenon of interest. Next, this relational expression is evolved through neural learning (so-called unsupervised learning). Alternatively, the relational expression can be evolved by tuning the relational expression by statistically looking at the input and output.

When a vehicle is shipped, the model developed by the manufacturer is implemented. Then, the installed model may evolve to suit the user in accordance with the user's driving of the vehicle. Alternatively, the model may be updated by software update at a dealer or the like.

Here, outputs from a camera 101 and a radar 102 mounted on the vehicle are sent to an external environment estimation section 10. Signals 111 of a positioning system such as GPS and navigation data 112, for example, transmitted from a network outside the vehicle are sent to a route search unit 61. The output of the camera 120 installed in the vehicle interior is sent to the driver state estimation section 20. The output of the sensors 130 that detect the behavior of the vehicle is sent to the vehicle condition measurement section 62. The output of the sensors 140 that detect the driver's operation is sent to the driver operation recognition section 63.

<Vehicle external environment estimation department>
The vehicle exterior environment estimation unit 10 receives outputs from a camera 101, a radar 102, etc. mounted on the vehicle, and estimates the vehicle exterior environment. The estimated external environment of the vehicle includes at least roads and obstacles. Here, the vehicle exterior environment estimation unit 10 estimates the vehicle environment including roads and obstacles by comparing three-dimensional information around the vehicle with the vehicle exterior environment model 15 based on data from the camera 101 and the radar 102. shall be estimated. The vehicle exterior environment model 15 is a trained model generated, for example, by deep learning, and can recognize roads, obstacles, etc. from three-dimensional information around the vehicle.

For example, the object recognition/map generation unit 11 identifies a free space, that is, an area where no object exists, from the image captured by the camera 101 through image processing. For example, a trained model generated by deep learning is used for image processing here. A two-dimensional map representing the free space is then generated. Further, the object recognition/map generation unit 11 acquires information on targets existing around the vehicle from the output of the radar 102. This information includes the position and speed of the target.

The estimation unit 12 combines the two-dimensional map output from the object recognition/map generation unit 11 and the target object information to generate a three-dimensional map representing the surroundings of the vehicle. Here, information on the installation position and imaging direction of the camera 101 and information on the installation position and transmission direction of the radar 102 are used. The estimation unit 12 estimates the vehicle environment including roads and obstacles by comparing the generated three-dimensional map and the vehicle exterior environment model 15.

<Driver state estimation unit>
The driver condition estimating unit 20 estimates the driver's health condition, emotions, or physical behavior from an image captured by a camera 120 installed in the vehicle interior. Examples of the health condition include good health, mild fatigue, poor physical condition, and decreased consciousness. Emotions include, for example, fun, normal, boredom, irritation, and discomfort.

For example, the driver state measurement unit 21 extracts a facial image of the driver from an image captured by a camera 120 installed in the vehicle interior, and identifies the driver. The extracted face image and the identified driver information are given as input to the human model 25. The human model 25 is a trained model generated, for example, by deep learning, and outputs the health condition and emotion of each person who may be the driver of the vehicle from the facial image. The estimation unit 22 outputs the driver's health condition and emotions output by the human model 25.

In addition, when a biological information sensor such as a skin temperature sensor, heart rate sensor, blood flow sensor, sweat sensor, etc. is used as a means for acquiring information about the driver, the driver condition measurement unit 21 uses the output of the biological information sensor. , measure the driver's biometric information. In this case, the human model 25 inputs the biometric information of each person who may be the driver of the vehicle, and outputs the health condition and emotion. The estimation unit 22 outputs the driver's health condition and emotions output by the human model 25.

Further, as the human model 25, a model that estimates the emotions that humans have regarding the behavior of the vehicle may be used for each person who may be the driver of the vehicle. In this case, a model must be constructed by managing the outputs of the sensors 130 that detect the behavior of the vehicle, the outputs of the sensors 140 that detect the driver's operations, the driver's biological information, and the estimated emotional state in chronological order. Bye. This model makes it possible to predict, for example, the relationship between the driver's heightened emotion (arousal level) and the behavior of the vehicle.

Further, the driver state estimation unit 20 may include a human body model as the human model 25. The human body model specifies, for example, the head mass (eg, 5 kg) and the muscular strength around the neck that supports front, back, left, and right G. When the human body model inputs the movement of the vehicle body (acceleration G and jerk), it outputs the expected physical and subjective state of the occupant. The occupant's physical condition may be, for example, comfortable/moderate/uncomfortable, and the subjective condition may be, for example, unexpected/predictable. By referring to the human body model, for example, it is possible to avoid selecting a vehicle body behavior that causes the head to tilt back even slightly, since this would be uncomfortable for the occupants. On the other hand, since the vehicle body behavior in which the head moves forward as if bowing makes it easier for the occupant to take a posture that resists this behavior and does not cause immediate discomfort, the vehicle can select that travel route. Alternatively, by referring to a human body model, it is possible to determine the target motion, for example, so that the occupant's head does not shake or dynamically so as to make the occupant's head appear lively.

<Route search section>
The route search unit 61 searches for a wide-area route for the vehicle using signals 111 from a positioning system such as GPS, and navigation data 112, for example, transmitted from a network outside the vehicle.

<Vehicle condition measurement section>
The vehicle state measurement unit 62 measures the state of the vehicle from the outputs of sensors 130 that detect the behavior of the vehicle, such as a vehicle speed sensor, acceleration sensor, and yaw rate sensor. Then, an in-vehicle environment model 65 representing the in-vehicle environment is generated. This in-vehicle environment includes physical quantities such as humidity, temperature, shaking, vibration, and acoustic noise that particularly affect the physical condition of the occupants. The in-vehicle environment estimating unit 64 estimates the in-vehicle environment based on the in-vehicle environment model 65 and outputs it.

<Driver operation recognition unit>
The driver operation recognition unit 63 recognizes the driver's operation from the outputs of sensors 140 that detect the driver's operation, such as a steering angle sensor, an accelerator sensor, and a brake sensor.

<Route generation section>
The route generation unit 30 generates a travel route for the vehicle based on the output of the external environment estimation unit 10 and the output of the route search unit 61. For example, the route generation unit 30 generates a driving route that avoids obstacles estimated by the vehicle exterior environment estimation unit 10 on the road estimated by the vehicle exterior environment estimation unit 10 . The output of the vehicle external environment estimation unit 10 includes, for example, travel route information regarding the travel route on which the vehicle travels. The driving path information includes information regarding the shape of the driving path itself and information regarding objects on the driving path. The information regarding the driving road shape includes the driving road shape (straight line, curve, curve curvature), the driving road width, the number of lanes, the width of each lane, and the like. The information regarding the object includes the relative position and velocity of the object with respect to the vehicle, the attributes (type, direction of movement), etc. of the object. Examples of the types of objects include vehicles, pedestrians, roads, and lane markings.

Here, the route generation unit 30 calculates a plurality of route candidates using the state lattice method, and selects one or more route candidates from among them based on the route cost of each route candidate. do. However, route generation may be performed using other methods.

The route generation unit 30 sets a virtual grid area on the travel route based on the travel route information. This grid area has multiple grid points. Each grid point specifies a position on the road. The route generation unit 30 uses the output of the route search unit 61 to set a predetermined grid point as a target arrival position. Then, a plurality of route candidates are calculated by route searching using a plurality of grid points within the grid area. In the state lattice method, a route branches from a certain grid point to an arbitrary grid point in front of the vehicle in the direction of travel. Therefore, each route candidate is set to sequentially pass through a plurality of grid points. Each route candidate also includes time information representing the time it takes to pass through each grid point, speed information regarding speed, acceleration, etc. at each grid point, and other information regarding vehicle motion.

The route generation unit 30 selects one or more travel routes from a plurality of route candidates based on the route cost. The route cost here includes, for example, the degree of lane centering, the acceleration of the vehicle, the steering angle, and the possibility of collision. Note that when the route generating section 30 selects a plurality of running routes, the target exercise determining section 40 and the energy management section 50, which will be described later, select one running route.

<Target exercise determination section>
The target motion determination unit 40 determines a target motion for the travel route selected by the route generation unit 30. The target motion refers to steering and acceleration/deceleration that trace the travel route. Further, the target motion determination unit 40 refers to the six-axis vehicle model 45 and calculates the movement of the vehicle body regarding the travel route selected by the route generation unit 30.

Here, the 6-axis vehicle model 45 refers to the acceleration in the 3-axis directions of "front and back," "left and right," and "up and down" of a running vehicle, and the angular velocity in the three-axis directions of "pitch," "roll," and "yaw." It is modeled. In other words, instead of capturing the movement of the vehicle only on the classical plane of vehicle dynamics engineering (only the vehicle's front, back, left and right movements (X-Y movement) and yaw movement (Z-axis)), This is a numerical model that reproduces the behavior of a vehicle using a total of six axes: the pitching (Y-axis) and roll (X-axis) movements of the vehicle body you are riding on, and the Z-axis movement (vertical movement of the vehicle body).

The target motion determination unit 40 calculates the movement of the vehicle body with reference to the six-axis vehicle model 45, and determines the target motion using the calculation result. That is, the target motion determination unit 40 uses the vehicle six-axis model 45 to calculate the planar movement of the vehicle and the vertical posture change of the vehicle body that occurs when traveling along the travel route generated by the route generation unit 30. The estimated planar movement and vertical posture change of the vehicle body are determined as the target movement of the vehicle. Thereby, for example, a so-called diagonal roll state can be generated during cornering.

For example, the target motion determining unit 40 inputs the vehicle body motion (acceleration G and jerk) calculated with reference to the six-axis vehicle model 45 into the human body model described above, and calculates the expected physical and subjective behavior of the occupant. may be obtained. For example, if the route generation unit 30 selects a plurality of travel routes, the target motion determination unit 40 may select one travel route based on the predicted physical condition and subjectivity of the occupant. good.

Furthermore, when the driver operation recognition unit 63 recognizes the driver's operation, the target movement determination unit 40 determines a target movement according to the driver's operation without following the travel route selected by the route generation unit 30. .

<Energy Management Department>
The energy management section 50 calculates the driving force, braking force, and steering angle for realizing the target motion determined by the target motion determining section 40. Then, operation signals for each actuator 200 are generated to generate the calculated driving force, braking force, and steering angle.

For example, the vehicle kinetic energy operation unit 51 may request torque from the drive system (engine, motor, transmission), steering system (steering), control system (brake), etc. for the target motion determined by the target motion determining unit 40. Calculate the physical quantity of The control amount calculation unit 52 calculates the control amount of each actuator so that the energy efficiency is the highest in achieving the target movement determined by the target movement determination unit 40. For example, the opening/closing timing of the intake and exhaust valves, the fuel injection timing of the injector, etc. are calculated so as to achieve the engine torque determined by the vehicle kinetic energy operating section 51 and to improve fuel efficiency the most. A vehicle thermal model 55 and a vehicle energy model 56 are used for energy management here. For example, each calculated physical quantity is compared with the vehicle energy model 56, and the momentum of each actuator is distributed so that energy consumption becomes smaller.

Specifically, for example, the energy management unit 50 calculates operating conditions that minimize energy loss for the travel route selected by the route generation unit 30, based on the target motion determined by the target motion determination unit 40. For example, the energy management unit 50 calculates the running resistance of the vehicle with respect to the running route selected by the route generating unit 30, and calculates the loss of the route. Running resistance includes tire friction, drivetrain loss, and air resistance. Then, the operating conditions for generating the driving force necessary to overcome this loss are determined. For example, the injection/ignition timing that results in the lowest fuel consumption in an internal combustion engine, the shift pattern that minimizes energy loss in the transmission, and the operating conditions for lock-up control of torque control are determined. Alternatively, if deceleration is required, a combination of the vehicle model's foot brake, engine brake, and drive auxiliary motor regeneration model that realizes the deceleration profile is calculated to determine operating conditions that minimize energy loss.

Then, the energy management unit 50 generates an operation signal for each actuator 200 according to the determined operating conditions, and outputs it to the control device of each actuator 200.

(Examples of other controls)
The route generation unit 30 may generate a travel route for the vehicle using the output of the driver state estimation unit 20. For example, the driver state estimation unit 20 outputs data representing the driver's emotions to the route generation unit 30, and the route generation unit 30 selects a travel route using the data representing the emotions. For example, when the emotion is "fun", a route with smooth vehicle behavior is selected, and when the emotion is "boring", a route with large changes in vehicle behavior is selected.

Alternatively, the route generating unit 30 may refer to the human model 25 included in the driver state estimating unit 20 and select a route that will most likely enhance the driver's emotions (highest level of arousal) from among the multiple route candidates. .

Further, when the route generation unit 30 determines that the vehicle is in danger from the vehicle exterior environment estimated by the vehicle exterior environment estimation unit 10, the route generation unit 30 may generate a route for emergency avoidance regardless of the driver state. . In addition, when the route generating unit 30 determines that the driver is unable or has difficulty driving based on the output of the driver state estimating unit 20 (for example, the driver is unconscious), the route generating unit 30 determines that the driver is unable to drive or that it is difficult to drive (for example, the driver has lost consciousness). A route may also be generated.

Further, when the target motion determining unit 40 determines that the driver is unable or has difficulty driving based on the output of the driver state estimating unit 20 (for example, the driver is unconscious), the target motion determining unit 40 instructs the driver to evacuate the vehicle to a safe location. The target motion may be determined accordingly. In this case, the route generation unit 30 generates a plurality of travel routes including a route for evacuating the vehicle to a safe place, and the target movement determination unit 40 determines that the driver is unable or difficult to drive. When this occurs, a route for evacuating the vehicle to a safe location may be selected (override).

(Other embodiments)
In the embodiment described above, the single information processing unit 1 determines the target motion of the vehicle based on various signals and data related to the vehicle, and based on the determined target motion, operates the operation signal for each actuator 200 of the vehicle. was assumed to be generated. However, for example, the information processing unit 1 may perform up to the determination of the target motion, and another information processing unit may generate the operation signals for each actuator 200 of the vehicle. In this case, the single information processing unit 1 determines the target motion of the vehicle based on various signals and data regarding the vehicle, and outputs data indicating the determined target motion. Then, another information processing unit receives the data output from the information processing unit 1 and generates operation signals for each actuator 200 of the vehicle.

1 Information processing unit 10 External environment estimation section 20 Driver state estimation section 30 Route generation section 40 Target motion determination section 50 Energy management section

Claims (2)

A vehicle calculation system that is installed in a vehicle and executes calculations to control the running of the vehicle,
an external environment estimation unit that receives an output from the means for acquiring information on the external environment and estimates the external environment including roads and obstacles;
a route generation unit that generates a travel route for the vehicle that avoids the estimated obstacles on the estimated road based on the output of the vehicle external environment estimation unit;
A target motion of the vehicle, which includes a planar motion and a vertical attitude change of the vehicle body, when the vehicle travels along the travel route generated by the route generator based on the output of the route generator. a target motion determining unit that determines the
an energy management unit that calculates a driving force, a braking force, and a steering angle to realize the target motion determined by the target motion determining unit ;
a driver state estimation unit that receives the output of the means for measuring the driver's state and estimates the driver's state including the health state;
The route generation unit generates a travel route that matches the driver state estimated by the driver state estimation unit,
The route generation unit selects a driving route from a plurality of route candidates by referring to the degree of change in vehicle behavior in each route candidate using the data representing the driver's emotion output from the driver state estimation unit. and
The driver state estimating unit includes a human body model that outputs an expected subjective feeling of the occupant when the movement of the vehicle body is input, and the route generating unit calculates the vehicle body behavior that is expected to be uncomfortable for the occupant based on the human body model. Remove the driving route that involves from the selection.
A vehicle computing system characterized by: The vehicle calculation system according to claim 1,
A computing system for a vehicle, wherein the energy management unit generates operation signals for each actuator to generate the driving force, braking force, and steering angle.

Images