JP7177641B2 - VEHICLE CONTROL DEVICE, VEHICLE CONTROL METHOD, AND PROGRAM - Google Patents

Description

The present invention relates to a vehicle control device, a vehicle control method, and a program.

After stopping the vehicle, when opening the tailgate to take out luggage, there was not enough space at the rear of the vehicle, and if the tailgate could not be opened, it was necessary to get in again and move the vehicle forward a little. . Therefore, in order to reduce this kind of trouble, a command to unlock the rear door is sent to the vehicle by remote control operation, and when the vehicle detects an obstacle within the movable range of the rear door, the vehicle is automatically A moving device is known (see Patent Document 1, for example).

JP 2015-20495 A

However, with the above technology, the user has to check the situation around the vehicle, and there are cases where the user cannot operate the remote control while standing in a place where it is easy to check, which is inconvenient.

SUMMARY OF THE INVENTION The present invention has been made in consideration of such circumstances, and one of the objects thereof is to provide a vehicle control device, a vehicle control method, and a program capable of improving user convenience.

(1): A vehicle control device according to an aspect of the present invention includes an imaging unit that captures an image of the surroundings of a vehicle, a recognition unit that recognizes the surrounding environment of the vehicle based on the image captured by the imaging unit, and A driving control unit that controls at least one of steering or acceleration/deceleration of the vehicle, an external operator that is operated from outside the vehicle, and an operation received by the external operator based on output information from the recognition unit. an acquisition unit that acquires, the operation control unit determines a control amount for the vehicle based on the operation content acquired by the acquisition unit, and the recognition unit determines the control amount based on the recognition result is given, the operation control unit determines whether or not the behavior of the vehicle satisfies a predetermined condition, and cancels the control according to the control amount when the predetermined condition is not satisfied. be.

(2): In the aspect of (1) above, the operation control unit performs control of steering and acceleration/deceleration of the vehicle as control according to the control amount.

(3): In the aspect (1) or (2) above, the operation control unit uses automatic operation control based on the recognition result of the recognition unit to execute control according to the control amount.

(4): In any one of the aspects (1) to (3) above, the external operator accepts at least one of an operation for driving the vehicle forward and an operation for driving the vehicle in reverse. .

(5): In any one of the aspects (1) to (4) above, when the recognized obstacle exists in the traveling direction of the vehicle when the control amount is given, the recognition unit detects the It is determined that the predetermined condition is not satisfied.

(6): In any one of the aspects (1) to (5) above, further comprising a communication unit that communicates with the key of the vehicle, and the recognition unit further includes a communication unit that communicates with the key of the vehicle. It is determined that the predetermined condition is not satisfied when the condition is not met.

(7): In any one of the above (1) to (6), the operation control unit causes the external operator to be in a state in which it does not accept the operation when the predetermined condition is not satisfied.

(8): In any one of the above aspects (1) to (7), an output unit provided outside the vehicle is further provided, and the operation control unit performs processing based on operation details received by the external operator. is being executed, the output unit outputs that the automatic driving function is in operation.

(9): In any one of the above (1) to (8), the operation control section determines the speed of the vehicle based on the magnitude of the force applied to the external operator.

(10): In any one of the aspects (1) to (3) above, the operation control section determines the traveling direction of the vehicle based on the displacement direction of the external operator.

(11): A vehicle control method according to an aspect of the present invention, in which a computer mounted on a vehicle recognizes the surrounding environment of the vehicle based on an image captured by an imaging unit that captures an image of the surroundings of the vehicle; When an operation received by an external operator operated from outside the vehicle is acquired, a control amount for the vehicle is determined based on the acquired operation, and the control amount is given based on the recognition result. Then, it is determined whether or not the behavior of the vehicle satisfies a predetermined condition, and if the predetermined condition is not satisfied, at least one of steering or acceleration/deceleration of the vehicle is controlled according to the control amount. Cancel.

(12): A program according to an aspect of the present invention causes a computer mounted on a vehicle to recognize the surrounding environment of the vehicle based on an image captured by an image capturing unit that captures the surroundings of the vehicle. Acquire the operation content accepted by an external operator operated from the outside, determine the control amount for the vehicle based on the acquired operation content, and determine the control amount based on the recognition result when the control amount is given. It is determined whether or not the behavior of the vehicle satisfies a predetermined condition, and if the predetermined condition is not satisfied, control of at least one of steering or acceleration/deceleration of the vehicle is canceled according to the control amount. .

According to (1) to (12), user convenience can be improved.

1 is a configuration diagram of a vehicle system 1 using a vehicle control device according to an embodiment; FIG. 4 is a diagram showing an example of an installation location of an external operator 90. FIG. 4 is a diagram for explaining a first configuration example of an external operator 90; FIG. FIG. 4 is a diagram for explaining an example of the movement of an external operator 90 and the movement of a vehicle; 3 is a functional configuration diagram of a recognition unit 130 and an action plan generation unit 140; FIG. 4 is a flowchart showing an example of control amount determination processing by a first control unit 120; 4 is a flowchart showing an example of vehicle control processing by a first control unit 120; FIG. 9 is a diagram for explaining a second configuration example of the external operator 90; FIG. 11 is a diagram for explaining a third configuration example of the external operator 90; It is a figure showing an example of hardware constitutions of automatic operation control device 100 of an embodiment.

Hereinafter, embodiments of a vehicle control device, a vehicle control method, and a program according to the present invention will be described with reference to the drawings.

[overall structure]
FIG. 1 is a configuration diagram of a vehicle system 1 using a vehicle control device according to an embodiment. A vehicle in which the vehicle system 1 is installed is, for example, a four-wheeled vehicle, and its drive source is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates using electric power generated by a generator connected to the internal combustion engine, or electric power discharged from a secondary battery or a fuel cell.

The vehicle system 1 includes, for example, a camera 10, a radar device 12, a viewfinder 14, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, a vehicle sensor 40, and an internal operator 80. , an external operator 90 , an external output unit 92 , a key communication unit 94 , an automatic driving control device 100 , a driving force output device 200 , a braking device 210 , and a steering device 220 . These apparatuses and devices are connected to each other by multiplex communication lines such as CAN (Controller Area Network) communication lines, serial communication lines, wireless communication networks, and the like. Note that the configuration shown in FIG. 1 is merely an example, and a part of the configuration may be omitted, or another configuration may be added.

The camera 10 is, for example, a digital camera using a solid-state imaging device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 is attached to an arbitrary location of a vehicle (hereinafter referred to as own vehicle M) on which the vehicle system 1 is mounted. When imaging the front, the camera 10 is attached to the upper part of the front windshield, the rear surface of the rearview mirror, or the like. The camera 10, for example, periodically and repeatedly captures the surroundings of the own vehicle. Camera 10 may be a stereo camera.

The radar device 12 radiates radio waves such as millimeter waves around the vehicle M and detects radio waves (reflected waves) reflected by an object to detect at least the position (distance and direction) of the object. The radar device 12 is attached to any location of the own vehicle M. As shown in FIG. The radar device 12 may detect the position and velocity of an object by the FM-CW (Frequency Modulated Continuous Wave) method.

The finder 14 is a LIDAR (Light Detection and Ranging). The finder 14 irradiates light around the vehicle M and measures scattered light. The finder 14 detects the distance to the object based on the time from light emission to light reception. The irradiated light is, for example, pulsed laser light. The viewfinder 14 is attached to any location on the vehicle.

The object recognition device 16 performs sensor fusion processing on the detection results of some or all of the camera 10, the radar device 12, and the finder 14, and recognizes the position, type, speed, etc. of the object. The object recognition device 16 outputs recognition results to the automatic driving control device 100 . Object recognition device 16 may output the detection result of camera 10, radar device 12, and finder 14 to automatic operation control device 100 as it is. The object recognition device 16 may be omitted from the vehicle system 1 .

The communication device 20 uses, for example, a cellular network, a Wi-Fi network, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), or the like, to communicate with other vehicles existing in the vicinity of the own vehicle, or to communicate with a wireless base. It communicates with various server devices through the station.

The HMI 30 presents various types of information to the occupants of the own vehicle and accepts input operations by the occupants. The HMI 30 includes various display devices, speakers, buzzers, touch panels, switches, keys, and the like.

The vehicle sensors 40 include a vehicle speed sensor that detects the speed of the vehicle, an acceleration sensor that detects acceleration, a yaw rate sensor that detects angular velocity about a vertical axis, a direction sensor that detects the direction of the vehicle M, and the like.

The internal controls 80 include, for example, accelerator pedals, brake pedals, shift levers, steering wheels, modified steering wheels, joysticks, and other controls. A sensor that detects the amount of operation or the presence or absence of operation is attached to the internal operator 80, and the detection result is the automatic driving control device 100, or the driving force output device 200, the brake device 210, and the steering device. 220 to some or all of them.

The external operator 90 is an operator operated from outside the vehicle, and includes, for example, an operator such as a joystick. A sensor that detects the amount of operation or the presence or absence of operation is attached to the external operator 90 , and the detection result is output to the automatic operation control device 100 . The external operator 90 receives at least an operation for moving the vehicle forward and an operation for moving the vehicle backward. The external operator 90 may also receive an operation for changing the steering angle of the vehicle or an operation for changing the speed.

The external output unit 92 includes various display devices provided at a position visible from the outside of the vehicle, a speaker for outputting sound to the outside of the vehicle, and the like.

The key communication unit 94 is a communication unit that communicates with the vehicle key, and the communication result and the like are output to the automatic driving control device 100 .

Automatic operation control device 100 is provided with the 1st control part 120 and the 2nd control part 160, for example. The first control unit 120 and the second control unit 160 are each implemented by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components are hardware (circuits) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), and GPU (Graphics Processing Unit). (including circuitry), or by cooperation of software and hardware. The program may be stored in advance in a storage device such as the HDD or flash memory of the automatic operation control device 100, or stored in a removable storage medium such as a DVD or CD-ROM, and the storage medium is a drive device. It may be installed in the HDD or flash memory of the automatic operation control device 100 by being attached to the . The 1st control part 120 and the 2nd control part 160 perform automatic operation control demonstrated below.

The 1st control part 120 is provided with the recognition part 130 and the action plan production|generation part 140, for example. The first control unit 120, for example, realizes in parallel a function based on AI (Artificial Intelligence) and a function based on a model given in advance. For example, the "recognition of intersections" function performs recognition of intersections by deep learning, etc., and recognition based on predetermined conditions (signals that can be pattern-matched, road markings, etc.) in parallel. It may be realized by scoring and evaluating comprehensively. This ensures the reliability of automated driving.

The recognition unit 130 recognizes the position, speed, acceleration, and other states of objects around the vehicle based on information input from the camera 10, the radar device 12, and the viewfinder 14 via the object recognition device 16. do. The position of the object is recognized, for example, as a position on absolute coordinates with a representative point (the center of gravity, the center of the drive shaft, etc.) of the own vehicle as the origin, and used for control. The position of an object may be represented by a representative point such as the center of gravity or a corner of the object, or may be represented by a represented area. The "state" of the object may include acceleration or jerk of the object, or "behavioral state" (eg, whether it is changing lanes or about to change lanes).

In principle, the action plan generation unit 140 drives the recommended lane determined by the recommended lane determination unit (not shown), and automatically ( Generates a target trajectory to be traveled in the future (independent of the driver's operation). The target trajectory includes, for example, velocity elements. For example, the target trajectory is represented by arranging points (trajectory points) that the vehicle should reach in order. A trajectory point is a point to be reached by the own vehicle M for each predetermined travel distance (for example, about several [m]) along the road. ) are generated as part of the target trajectory. Also, the trajectory point may be a position that the vehicle M should reach at each predetermined sampling time. In this case, the information on the target velocity and target acceleration is represented by the intervals between the trajectory points.

The action plan generator 140 may set an automatic driving event when generating the target trajectory. Autonomous driving events include constant-speed driving events, low-speed following driving events, lane change events, branching events, merging events, and takeover events. The action plan generator 140 generates a target trajectory according to the activated event.

Further, when receiving an operation from the external operator 90, the action plan generator 140 may set an external operation event. When the external operation event is activated, the action plan generation unit 140 determines the amount of control according to the content of the operation received by the external operator 90, and executes control according to the determined amount of control. The control according to the control amount is, for example, vehicle operation and acceleration/deceleration control, and is executed by the action plan generator 140 using the above-described automatic driving control. Details will be described later.

The second control unit 160 controls the driving force output device 200, the braking device 210, and the steering device 220 so that the vehicle M passes the target trajectory generated by the action plan generating unit 140 at the scheduled time. Control.

The second control unit 160 includes an acquisition unit 162, a speed control unit 164, and a steering control unit 166, for example. The acquisition unit 162 acquires information on the target trajectory (trajectory point) generated by the action plan generation unit 140 and stores it in a memory (not shown). Speed control unit 164 controls running driving force output device 200 or brake device 210 based on the speed element associated with the target trajectory stored in the memory. The steering control unit 166 controls the steering device 220 according to the curve of the target trajectory stored in the memory. The processing of the speed control unit 164 and the steering control unit 166 is realized by, for example, a combination of feedforward control and feedback control. As an example, the steering control unit 166 performs a combination of feedforward control according to the curvature of the road ahead of the host vehicle M and feedback control based on deviation from the target trajectory.

The running driving force output device 200 outputs running driving force (torque) for running the vehicle to the drive wheels. Traveling driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, a transmission, and the like, and an ECU controlling these. The ECU controls the above configuration according to information input from the second control unit 160 or information input from the internal operator 80 .

The brake device 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to information input from the second control unit 160 or information input from the internal operator 80 so that brake torque corresponding to the braking operation is output to each wheel. The brake device 210 may include, as a backup, a mechanism that transmits hydraulic pressure generated by operating the brake pedal included in the internal operator 80 to the cylinders via the master cylinder. The brake device 210 is not limited to the configuration described above, and is an electronically controlled hydraulic brake device that controls the actuator according to information input from the second control unit 160 to transmit the hydraulic pressure of the master cylinder to the cylinder. good too.

The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor, for example, applies force to a rack and pinion mechanism to change the orientation of the steered wheels. The steering ECU drives the electric motor according to information input from the second control unit 160 or information input from the internal operator 80 to change the direction of the steered wheels.

Next, an example of the external operator 90 will be described. FIG. 2 is a diagram showing an example of an installation location of the external operator 90. As shown in FIG. The external operator 90 is installed, for example, in either or both of a front area 5A located in front of the vehicle body 5 and a rear area 5B located in the rear of the vehicle body 5 . The external operator 90 is arranged, for example, inside a cover such as a fuel filler port. The external operator 90 is not limited to this, and may be attached outside the vehicle body like an antenna, and may be installed, for example, in the upper surface area 5C located near the upper surface of the vehicle. In FIG. 2, the longitudinal direction of the vehicle body 5 is defined as the X axis, and the horizontal direction of the vehicle body 5 is defined as the Y axis.

FIG. 3 is a diagram for explaining a first configuration example of the external operator 90. As shown in FIG. The external operator 90 includes an operation lever 90A and an operation base 90B. The operating lever 90A is movable in the X-axis direction, and is also movable in an arc around the Z-axis with the operation table 90B as the center. The direction in which the control lever 90A is moved is detected as the traveling direction of the vehicle. Further, the force with which the operation lever 90A is pushed in or the force with which it is pulled is detected as the speed of the vehicle.

In this manner, the vehicle is moved in conjunction with the amount of operation detected by the external operator 90 . In other words, the action plan generator 140 determines the traveling direction of the vehicle based on the displacement direction of the control lever 90A. Also, the action plan generator 140 determines the speed of the vehicle based on the magnitude of the force applied to the external operator 90 .

FIG. 4 is a diagram for explaining an example of the movement of the external operator 90 and the movement of the vehicle. As shown in FIG. 4A, when the operating lever 90A is pushed in the X-axis direction, the vehicle body 5 moves forward in the X-axis direction. As shown in FIG. 4B, when the operating lever 90A is pulled in the X-axis direction, the vehicle body 5 moves backward in the X-axis direction.

As shown in FIG. 4(c), when the operation lever 90A is pushed while being tilted to the left side of the vehicle body 5, the vehicle body 5 moves forward while turning the steering wheel to the right. As shown in FIG. 4(d), when the operation lever 90A is pulled while being tilted to the left side of the vehicle body 5, the vehicle body 5 moves backward while steering to the left.

As shown in FIG. 4(e), when the operation lever 90A is pushed in while being tilted to the right side of the vehicle body 5, the vehicle body 5 moves forward while steering to the left. As shown in FIG. 4(f), when the operation lever 90A is pulled while being tilted to the right side of the vehicle body 5, the vehicle body 5 moves backward while steering to the right.

As shown in FIG. 4(g), when the operation lever 90A is tilted to the right side of the vehicle body 5 and pushed in with a stronger force than in FIG. 4(e), the vehicle body 5 steers to the left and Move forward faster than As shown in FIG. 4(h), when the operation lever 90A is tilted to the right side of the vehicle body 5 and pulled with a force stronger than that in FIG. move backward at a faster speed.

With such a configuration, the vehicle can be moved in conjunction with the movement of the external operator 90 .

Next, other functional configurations of the recognition unit 130 and the action plan generation unit 140 will be described. FIG. 5 is a functional configuration diagram of the recognition unit 130 and the action plan generation unit 140. As shown in FIG. The functional configuration unit shown in FIG. 5 executes the following processing in accordance with the setting of the external operation event described above.

The recognition unit 130 includes a determination unit 131, for example. The determination unit 131 determines whether or not a predetermined driving condition is satisfied based on the result of recognition by the recognition unit 130 . For example, when an external operation event is set by the action plan generation unit 140, the determination unit 131 determines whether or not the behavior of the vehicle satisfies the first condition when the determined control amount is given. The first condition includes, for example, that no obstacle is recognized in the direction in which the vehicle moves, that the obstacle is recognized in the direction in which the vehicle moves, and that the distance between the obstacle and the vehicle is greater than or equal to a certain amount. included. Obstacles include other vehicles, pedestrians, utility poles, etc., but do not include things that vehicles may come into contact with, such as tire chocks.

The recognition unit 130 may determine whether or not the situation in which the external operator 90 is operated satisfies the second condition. The recognition unit 130 may determine that the drive condition is satisfied when the second condition is satisfied in addition to the first condition. The second condition includes, for example, that the external operator 90 is operated within several seconds (or several minutes) after the key communication unit 94 communicates with the vehicle key, and within several minutes after the vehicle engine is turned off. includes that the external operator 90 is operated.

The action plan generation unit 140 includes, for example, an acquisition unit 141, a determination unit 142, an external operation control unit 143, an operator control unit 144, and an output control unit 145.

Acquisition unit 141 acquires operation content accepted by external operator 90 based on the output from external operator 90 . For example, the acquisition unit 141 acquires information indicating the amount of operation (traveling direction and speed of the vehicle) detected by the external operator 90 .

The determination unit 142 determines a control amount for the vehicle based on the operation details acquired by the acquisition unit 141 . For example, when the acquiring unit 141 acquires that the traveling direction of the vehicle is the positive direction of the X-axis and the speed is level 1 as the operation content, the determining unit 142 determines to move forward at a speed of 5 km/h. do.

The external operation control section 143 controls the second control section 160 according to the determination result of the determination section 131 . For example, when the determination unit 131 determines that the drive condition is satisfied, the external operation control unit 143 controls the second control unit 160 so that the vehicle travels with the control amount determined by the determination unit 142 . On the other hand, when the determination unit 131 determines that the drive condition is not satisfied, the external operation control unit 143 controls the second control unit 160 to cancel the control according to the control amount. "Controlling the second control unit 160 so as to cancel the control according to the control amount" includes, for example, controlling the braking device 210 to stop the vehicle, controlling the steering device 220 to fix the steering. When the vehicle is in a stopped state, the state is maintained as it is.

The manipulator control section 144 controls the state of the external manipulator 90 according to the determination result of the determination section 131 . For example, when the determination unit 131 determines that the drive condition is satisfied, the operator control unit 144 puts the external operator 90 into a state of accepting an operation. On the other hand, when the determination unit 131 determines that the drive condition is not satisfied, the operator control unit 144 causes the external operator 90 to be in a state in which it does not accept the operation. The "state in which no operation is accepted" includes, for example, disabling operation levers and disabling buttons and knobs.

The output control unit 145 causes the external output unit 92 to output "that the automatic driving function is in operation" while the process based on the operation content received by the external operator 90 is being executed. For example, the output control unit 145 causes the display unit to display a message indicating that "the automatic driving function is in operation", or causes the speaker to output the message.

[flowchart]
FIG. 6 is a flow chart showing an example of control amount determination processing by the first control unit 120 . First, the action plan generator 140 determines whether or not an external operation event has been set (step S101). When the external operation event is set, the recognition unit 130 starts recognition processing for recognizing the surroundings of the vehicle (step S103). Next, when the external operator is operated (step S105), the acquisition unit 141 acquires information indicating the amount of operation detected by the external operator 90 (step S107). Then, the determination unit 142 determines the control amount for the vehicle based on the information indicating the operation amount acquired by the acquisition unit 141 (step S109). When the end of the external operation event is instructed (step S111), the action plan generator 140 ends the process based on the external operation event. When the termination of the external operation event is instructed, for example, when a predetermined time elapses after the operation of the external operator 90, or when the tailgate is opened, the cover housing the external operator 90 is closed. This includes cases where

If the external operator 90 is not operated in step S105, or if the end of the external operation event is not instructed in step S111, the process returns to step S103 to repeat the process.

FIG. 7 is a flowchart showing an example of vehicle control processing by the first control unit 120. As shown in FIG. The following processing is executed each time the external operator 90 is operated. The external operation control unit 143 determines whether or not the control amount has been determined by the determination unit 142 (step S201). When the control amount is determined by the determination unit 142, the external operation control unit 143 determines whether or not the amount of change in the control amount determined in step S201 exceeds the threshold (step S203). The amount of change in the controlled variable is the difference between the previously determined controlled variable and the currently determined controlled variable. Examples of cases where the amount of change in the determined control amount exceeds the threshold include cases where the change in vehicle speed per hour exceeds the threshold, cases where the steering angle exceeds the threshold, and the like. If the control amount is not determined by the determining unit 142, or if the amount of change in the determined control amount does not exceed the threshold, the process returns to step S201 and repeats the process.

When the amount of change in the control amount determined in step S203 exceeds the threshold, the determination unit 131 determines whether or not the drive condition is satisfied (step S205). When it is determined that the drive condition is satisfied, the external operation control section 143 controls the second control section 160 so that the vehicle travels with the control amount determined by the determination section 142 (step S207). Then, the output control unit 145 causes the external output unit 92 to output "that the automatic driving function is in operation" (step S209).

When it is determined in step S205 that the drive condition is not satisfied, the external operation control section 143 controls the second control section 160 to cancel the control according to the control amount (step S211). Then, the manipulator control unit 144 puts the external manipulator 90 into a state in which it does not accept an operation (step S213).

After step S209 or step S213, the first control unit 120 returns to step S205 and repeats the process until an operation end instruction is received. This makes it possible to stop the vehicle when the driving condition is no longer satisfied while the vehicle is in motion. Vehicles can be moved when they meet the requirements.

According to the automatic driving control device 100 of the present embodiment described above, when the external operator 90 is operated to move the vehicle, the surroundings of the vehicle are checked using the automatic driving function, and the driving condition is not satisfied. In some cases, control over the vehicle can be canceled. By doing so, the vehicle can be easily moved to a place desired by the user while safety is ensured. Since the vehicle side checks the surroundings of the vehicle, the user's convenience is improved.

Further, since the vehicle can be moved by operating the external operator 90, the vehicle can be moved while the user visually confirms the space required for opening the tailgate. Therefore, it is possible to easily adjust the movement of the vehicle compared to the method of getting on the vehicle and moving the vehicle.

In addition, since the user can determine the control amount of the vehicle using the external operator 90, adjustment is easy even when the distance to move the vehicle is desired to be changed according to the size of the luggage.

In addition, by outputting a message indicating that the automatic driving function is in operation from the external output unit 92, the user who operates the external operator 90 can move the vehicle with peace of mind.

Further, when it is determined that the drive condition is not satisfied, the external operator 90 is brought into a state in which the operation is not accepted, thereby notifying the external operator that the vehicle cannot be moved by operating the external operator 90 . This can be communicated to the user who operates the manipulator 90 .

Further, by enabling operation using the external operator 90 when the second condition is satisfied in addition to the first condition, it is possible to prevent a situation in which the vehicle is moved without permission from the owner of the vehicle. can.

Next, another configuration example of the external operator 90 will be described with reference to FIGS. 8 and 9. FIG. FIG. 8 is a diagram for explaining a second configuration example of the external operator 90. As shown in FIG. FIG. 9 is a diagram for explaining a third configuration example of the external operator 90. As shown in FIG.

FIG. 8A is a side view of a second configuration example of the external operator 90. FIG. FIG. 8(b) is a top view of the second configuration example of the external operator 90. As shown in FIG. The external operator 90 includes, for example, an operation lever 90C, a movable table 90X, and a movable table 90Y. Note that FIGS. 8A and 8B show a state in which the operating lever 90C is positioned at the reference position. The movable table 90X is a member that moves the operating lever 90C along the X-axis direction. The movable table 90Y is a member that moves the operating lever 90C along the Y-axis direction. Note that when the operating lever 90C is operated in the direction between the X axis and the Y axis, both the movable table 90X and the movable table 90Y move the operating lever 90C in the operated direction. The direction in which the operation lever 90C is moved on the XY plane is detected as the traveling direction of the vehicle. Further, the moving distance of the operating lever 90C from a predetermined reference position is detected as the speed of the vehicle.

FIG. 9A is a side view of a third configuration example of the external operator 90. FIG. FIG. 9(b) is a top view of the third configuration example of the external operator 90. As shown in FIG. The external operator 90 includes an operation lever 90D and an operation base 90E. Note that FIGS. 9A and 9B show a state in which the operating lever 90D is vertical. The operating lever 90D is configured to tilt in any direction of 360 degrees around the Z axis. The direction in which the operation lever 90D is tilted on the XY plane is detected as the traveling direction of the vehicle. Also, the tilted angle 90F of the control lever 90D is detected as the speed of the vehicle.

Note that the external operator 90 is not limited to these configuration examples. For example, it may be a button or knob for inputting the traveling direction or speed. Also, the external operator 90 may be a capacitance sensor. With this configuration, the user can push the vehicle simply by touching the external operator 90, which is a capacitance sensor, so that even a weak user can easily move the vehicle. Further, the external operator 90 may receive adjustment of the traveling direction and speed according to the contact position of the capacitance sensor. With this configuration, the user can move the vehicle in a desired direction and speed by touching a predetermined position of the capacitance sensor.

[Hardware configuration]
Drawing 10 is a figure showing an example of hardware constitutions of automatic operation control device 100 of an embodiment. As shown, the automatic operation control device 100 includes a communication controller 100-1, a CPU 100-2, a RAM (Random Access Memory) 100-3 used as a working memory, a ROM (Read Only Memory) for storing a boot program, etc. 100-4, a storage device 100-5 such as a flash memory or a HDD (Hard Disk Drive), and a drive device 100-6 are interconnected by an internal bus or a dedicated communication line. The communication controller 100-1 communicates with components other than the automatic operation control device 100. FIG. The storage device 100-5 stores a program 100-5a executed by the CPU 100-2. This program is developed in RAM 100-3 by a DMA (Direct Memory Access) controller (not shown) or the like and executed by CPU 100-2. Thereby, part or all of the functional configurations included in first control unit 120 and second control unit 160 are realized.

The embodiment described above can be expressed as follows.
a storage device storing a program;
a hardware processor;
By executing the program stored in the storage device, the hardware processor
recognizing the surrounding environment of the vehicle based on an image captured by an imaging unit that captures the surroundings of the vehicle;
controlling at least one of steering or acceleration/deceleration of the vehicle based on the recognition result;
Acquiring operation details received by an external operator operated from the outside of the vehicle;
determining a control amount for the vehicle based on the acquired operation content;
determining whether the behavior of the vehicle satisfies a predetermined condition when the control amount is given, based on the recognition result;
canceling the control according to the control amount if the predetermined condition is not satisfied;
A vehicle control device configured to:

As described above, the mode for carrying out the present invention has been described using the embodiments, but the present invention is not limited to such embodiments at all, and various modifications and replacements can be made without departing from the scope of the present invention. can be added.

DESCRIPTION OF SYMBOLS 1... Vehicle system, 10... Camera, 12... Radar apparatus, 14... Finder, 16... Object recognition apparatus, 20... Communication apparatus, 30... HMI, 40... Vehicle sensor, 80... Internal operator, 90... External operator, 92... External output unit, 94... Key communication unit, 100... Automatic operation control device, 120... First control unit, 130... Recognition unit, 131... Judgment unit, 140... Action plan generation unit, 141... Acquisition unit, 142... Determination unit 143 External operation control unit 144 Manipulator control unit 145 Output control unit 160 Second control unit 200 Driving force output device 210 Brake device 220 Steering device

Claims (13)

an imaging unit that images the surroundings of the vehicle;
a recognition unit that recognizes the surrounding environment of the vehicle based on the image captured by the imaging unit;
a driving control unit that controls at least one of steering or acceleration/deceleration of the vehicle based on the output information of the recognition unit;
an external operator attached to an exterior portion of the vehicle body of the vehicle and operated by a person outside and in the vicinity of the vehicle;
an acquisition unit that acquires operation content received by the external operator;
a determining unit that determines, based on the recognition result of the recognizing unit, whether or not a predetermined condition including a condition related to behavior of the vehicle is satisfied when a control amount for the vehicle is given ;
The operation control unit determines the control amount based on the operation content acquired by the acquisition unit, and determines that the predetermined condition is not satisfied by the determination unit , according to the control amount cancel control,
Vehicle controller. The operation control unit controls steering and acceleration/deceleration of the vehicle as control according to the control amount.
The vehicle control device according to claim 1. The operation control unit uses automatic operation control based on the recognition result of the recognition unit to perform control according to the control amount.
The vehicle control device according to claim 1 or 2. The external operator accepts at least one of an operation for moving the vehicle forward and an operation for moving the vehicle in reverse.
The vehicle control device according to any one of claims 1 to 3. The operation control unit cancels the control according to the control amount when an obstacle recognized by the recognition unit exists in the traveling direction of the vehicle when the control amount is given.
The vehicle control device according to any one of claims 1 to 4. further comprising a communication unit that communicates with a key for permitting unlocking and/or starting of the vehicle;
The operation control unit cancels the control according to the control amount when the communication unit is not in communication with the vehicle key.
The vehicle control device according to any one of claims 1 to 5. When the predetermined condition is not satisfied, the operation control unit puts the external operator into a state in which it does not accept an operation.
The vehicle control device according to any one of claims 1 to 6. further comprising a display unit attached to the exterior of the vehicle body of the vehicle and displaying an image visible from the outside of the vehicle;
The operation control unit causes the display unit to display an image indicating that the automatic operation function is in operation while the processing based on the operation content received by the external operator is being executed.
The vehicle control device according to any one of claims 1 to 7. further comprising a speaker that is attached to the exterior of the vehicle body of the vehicle and outputs audio to the outside of the vehicle;
The driving control unit causes the speaker to output a sound indicating that the automatic driving function is in operation while the process based on the operation content received by the external operator is being executed.
The vehicle control device according to any one of claims 1 to 8. The operation control unit determines the speed of the vehicle based on the magnitude of the force applied to the external operator.
The vehicle control device according to any one of claims 1 to 9. The operation control unit determines the traveling direction of the vehicle based on the displacement direction of the external operator.
The vehicle control device according to any one of claims 1 to 10. A computer installed in the vehicle
recognizing the surrounding environment of the vehicle based on an image captured by an imaging unit that captures the surroundings of the vehicle;
an external operator attached to the exterior of the vehicle body of the vehicle and operated by a person outside and in the vicinity of the vehicle;
determining whether or not a predetermined condition including a condition related to behavior of the vehicle is satisfied when a control amount for the vehicle is given based on the result of recognition of the surrounding environment;
determining the control amount based on the obtained operation content;
When it is determined that the predetermined condition is not satisfied, at least one control of steering or acceleration/deceleration of the vehicle is canceled in accordance with the control amount.
Vehicle control method. computer on board the vehicle,
recognizing the surrounding environment of the vehicle based on the image captured by the imaging unit that captures the surroundings of the vehicle;
An external operator attached to the exterior of the vehicle body of the vehicle and operated by a person outside and in the vicinity of the vehicle Acquires operation details received by an external operator,
determining whether or not a predetermined condition including a condition related to the behavior of the vehicle is satisfied when a control amount for the vehicle is given based on the recognition result of the surrounding environment;
determining the control amount based on the acquired operation content;
When it is determined that the predetermined condition is not satisfied, at least one control of steering or acceleration/deceleration of the vehicle is canceled in accordance with the control amount;
program.

Images