JP2019172052A - Vehicle control device and vehicle control system - Google Patents

Abstract

To provide a vehicle control device etc., capable of controlling a vehicle as instructed by an instructing person who is outside the vehicle and does not have a terminal device.SOLUTION: An ECU (vehicle control device) comprises an instructing person detection part 1011, an instructing action detection part 1014, and a first vehicle control part 1015. The instructing person detection part 1011 detects an instructing person from an image picked up by a camera 200. The instructing action detection part 1014 detects an instructing action indicative of an instruction by the instructing person from the image picked up by the camera 200. The first vehicle control part 1015 controls, when the instructing action detection part 1014 detects the instructing action, a vehicle 400 according to the instruction that the instructing action indicates.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a vehicle control device and a vehicle control system.

  A vehicle control system that can reduce anxiety given to people around the vehicle is known (see, for example, Patent Document 1). Patent Document 1 states that “a vehicle control system performs at least one of vehicle speed control and steering control based on a detection unit that detects a vehicle peripheral state and a vehicle peripheral state detected by the detection unit. An automatic driving control unit that performs automatic driving automatically, a recognition unit that recognizes a moving object that exists in the vicinity of the vehicle based on the peripheral state of the vehicle detected by the detection unit, and a recognition unit that is recognized A transmitting unit that transmits information for causing the terminal device to call attention in a direction other than the direction of the moving object. "

JP 2017-207964

  In the technology disclosed in Patent Document 1, in order to receive information for alerting, people around the vehicle need to have a terminal device. Also, with such technology, people around the vehicle cannot control the vehicle.

  The objective of this invention is providing the vehicle control apparatus etc. which can control a vehicle according to the instruction | indication of the instructor who does not have the terminal device outside the vehicle.

  In order to achieve the above object, the present invention provides an instructor detection unit for detecting an instructor, an instruction operation detection unit for detecting an instruction operation indicating an instruction of the instructor, and when the instruction operation is detected, A first vehicle control unit that controls the vehicle according to the instruction indicated by the instruction operation.

  ADVANTAGE OF THE INVENTION According to this invention, a vehicle can be controlled according to the instruction | indication of the instructor who does not have the terminal device outside the vehicle. Problems, configurations, and effects other than those described above will be clarified by the following description of embodiments.

1 is a configuration diagram of a vehicle control system including an ECU as a vehicle control device according to a first embodiment of the present invention. It is a block diagram which shows the function of CPU used for ECU by the 1st Embodiment of this invention. It is a flowchart which shows operation | movement of CPU used for ECU by the 1st Embodiment of this invention. It is a figure for demonstrating the modification 1, and shows the focal distance when a field angle of a camera is wide. It is a figure for demonstrating the modification 1, and shows the focal distance when a field angle of a camera is narrow. It is a flowchart which shows operation | movement of the modification 2 of 1st Embodiment. It is a flowchart which shows the operation | movement of the 2nd Embodiment of this invention. It is a flowchart which shows the operation | movement of the 3rd Embodiment of this invention. It is a figure which shows taking over of the instruction | indication from a 1st instructor to a 2nd instructor. It is a figure for demonstrating the modification of the 3rd Embodiment of this invention. It is a figure for demonstrating the example which changes authority according to the thing which an instructor hold | maintains.

  Hereinafter, the configuration and operation of a vehicle control system including a vehicle control device according to first to third embodiments of the present invention will be described with reference to the drawings. In each figure, the same numerals indicate the same parts.

(First embodiment)
First, the configuration of the vehicle control system will be described with reference to FIG. FIG. 1 is a configuration diagram of a vehicle control system 1 including an ECU 100 (Electronic Control Unit) as a vehicle control device according to a first embodiment of the present invention.

(System configuration)
The ECU 100 includes a CPU 101 (Central Processing Unit) as a processor, a memory 102 as a storage device, an I / F 103 as an input / output circuit, and the like. ECU 100 is mounted on the vehicle.

  The camera 200 captures an image and temporarily stores the captured image in a memory. The camera 200 is connected to the I / F 103 and is controlled by the CPU 101. The camera 200 is a stereo camera, for example.

  The light source, the sound source, or the display 300 sends a signal (first signal) for confirming whether there is an intention to control the vehicle to an instructor (human) around the vehicle. A light source, a sound source, or a display 300 is connected to the I / F 103 and controlled by the CPU 101. The light source is, for example, a vehicle light, the sound source is, for example, a horn, and the display is, for example, an electric bulletin board provided on the front surface of a bus or the like. A light source, a sound source, or a display 300 is connected to the I / F 103 and controlled by the CPU 101.

  The vehicle 400 includes a drive mechanism that drives the vehicle body, a braking mechanism that brakes the vehicle body, a steering mechanism that steers the vehicle body, and the like.

  Next, the function of the CPU 101 will be described with reference to FIG. FIG. 2 is a block diagram showing functions of the CPU 101 used in the ECU 100 according to the first embodiment of the present invention.

(function)
The CPU 101 executes a predetermined program stored in advance in the memory 102 to thereby perform an instructor detection unit 1011, a signal transmission unit 1012, a response operation detection unit 1013, an instruction operation detection unit 1014, a first vehicle control unit 1015, It functions as the second vehicle control unit 1016. Note that these functions may be realized by a circuit such as an FPGA (Field-Programmable Gate Array).

  The instructor detection unit 1011 detects an instructor (human) from the image captured by the camera 200.

  When the instructor is detected by the instructor detection unit 1011, the signal transmission unit 1012 confirms whether the instructor has an intention to control the vehicle by controlling at least one of the light source, the sound source, and the display. Signal (first signal) is sent out. For example, the signal transmission unit 1012 lights a vehicle light, sounds a horn, or displays a message “Do you want to control the vehicle?” On an electric bulletin board for a predetermined period.

  The instructor performs a predetermined operation when he wants to control the vehicle. The predetermined operation (response operation) is, for example, an operation of raising both arms, an operation of raising a flag, or the like.

  The response operation detection unit 1013 detects a response operation indicating the response of the instructor from the image captured by the camera 200. Next, the response operation notifies that the instruction operation indicating the instruction from the instructor will continue.

  The instruction operation detection unit 1014 detects an instruction operation from an image captured by the camera 200. For example, if the instructor wants to stop the vehicle, both arms are leveled, and if the vehicle is to be parked, after pointing the parking position, the instructor turns one arm downward.

  The first vehicle control unit 1015 controls the vehicle 400 according to the instruction indicated by the instruction operation when the instruction operation detection unit 1014 detects the instruction operation.

  The second vehicle control unit 1016 performs control to bring the vehicle 400 closer to the instructor when the response operation is detected by the response operation detection unit 1013.

  Next, the operation of the CPU 101 will be described with reference to FIG. FIG. 3 is a flowchart showing the operation of the CPU 101 used in the ECU 100 according to the first embodiment of the present invention.

(Operation)
The CPU 101 (instructor detection unit 1011) determines whether or not an instructor (human) is detected from the image captured by the camera 200 (S15). For example, the CPU 101 analyzes the image and determines (recognizes) an object that is similar to a human shape stored in advance in the memory 102 and that moves similar to a human action as a human.

  When the instructor (human) is detected (S15: YES), the CPU 101 (signal transmission unit 1012) has an intention to control the vehicle by, for example, lighting the vehicle light for a predetermined period. A signal (first signal) for confirming whether or not is transmitted (S20). On the other hand, in order to show the intention to control the vehicle, for example, the instructor raises both arms, for example. If no instructor (human) is detected (S15: NO), the CPU 101 repeats the process of S15.

  The CPU 101 (response operation detection unit 1013) determines whether or not a response operation (raising both arms) indicating the response of the instructor is detected from the image captured by the camera 200 (S25). Note that the information defining the response action is stored in advance in the memory 102, for example. For example, the CPU 101 compares the information indicating the action of the instructor obtained by analyzing (behavior analysis) the image captured by the camera 200 with the information defining the response action stored in the memory 102. If they are similar, it is determined that a response operation has been detected (recognized).

  When a response operation is detected (S25: YES), CPU 101 (second vehicle control unit 1016) performs control to bring vehicle 400 closer to the instructor (S30). For example, the CPU 101 advances the vehicle 400 to a position at a predetermined distance from the instructor and stops it. As a result, the proportion of the image of the instructor in the image captured by the camera 200 increases, and the instruction operation following the response operation is easily recognized in the image analysis.

  If no response operation is detected (S25: NO), the CPU 101 repeats the process of S25 for a predetermined time (S26: YES), and then returns to normal automatic operation (S27). In normal automatic driving, control for advancing the vehicle 400 while avoiding humans is performed.

  Subsequently, for example, the instructor leveles both arms as an instruction operation indicating an instruction to stop the vehicle 400.

  The CPU 101 (instruction action detection unit 1014) determines whether or not an instruction action has been detected from an image captured by the camera 200 (S35). Note that the information defining the instruction operation is stored in the memory 102 in advance, for example. For example, the CPU 101 compares the information indicating the operation of the instructor obtained by analyzing (behavior analysis) the image captured by the camera 200 with the information defining the instruction operation stored in the memory 102. When they are similar, it is determined that the instruction operation is detected (recognized).

  CPU101 (1st vehicle control part 1015) controls the vehicle 400 according to the instruction | indication which an instruction | indication action shows, when instruction | indication operation | movement is detected (S35: YES) (S40). Specifically, the CPU 101 performs control to stop the vehicle 400 in accordance with an instruction indicated by an operation (instruction operation) for leveling both arms. If no instruction operation is detected (S35: NO), the CPU 101 repeats the process of S35.

  As described above, according to the present embodiment, the vehicle can be controlled in accordance with an instruction from an instructor who does not have a terminal device outside the vehicle.

(Modification 1)
Next, Modification 1 of the first embodiment of the present invention will be described with reference to FIGS. 4A and 4B. 4A is a diagram illustrating the focal length when the angle of view of the camera 200 is wide, and FIG. 4B is a diagram illustrating the focal length when the angle of view of the camera 200 is narrow.

  In the first modification, the CPU 101 (second vehicle control unit 1016) performs control to bring the vehicle 400 closer to the instructor according to the angle of view of the camera 200 when a response operation is detected.

  For example, as shown in FIG. 4A, when the angle of view θ1 of the camera 200 is wide, the CPU 101 brings the vehicle 400 closer to the instructor, and the distance R to the instructor detected by the camera 200 corresponds to the angle of view θ1. When the distance d1 is reached, the vehicle 400 is stopped.

  On the other hand, when the angle of view θ2 of the camera 200 is narrow as shown in FIG. 4B, the CPU 101 brings the vehicle 400 close to the instructor, and the distance R detected by the camera 200 becomes the focal length d2 corresponding to the angle of view θ2. The vehicle 400 is stopped. Here, the relationship of θ2 <θ1 and d1 <d2 is established.

  Thereby, since the focus of the camera 200 is focused on the instructor, it is easy to detect the instruction operation of the instructor.

(Modification 2)
Next, Modification 2 of the first embodiment of the present invention will be described with reference to FIG. FIG. 5 is a flowchart showing the operation of the second modification of the first embodiment. In FIG. 5, the processing from S20 to S30 in FIG. 3 is omitted.

  In an emergency, it is assumed that a response action cannot be obtained from the instructor. Therefore, in this modification, the vehicle is controlled according to the instruction operation from the instructor without waiting for the response operation from the instructor. For example, when the instruction operation from the instructor indicates an instruction accompanying an emergency, the CPU 101 (first vehicle control unit 1015) performs control to stop the vehicle 400 in a safe place. As a result, the vehicle can be quickly controlled.

  Note that the CPU 101 (instruction operation detection unit 1014) may predict an instruction operation from an image captured by the camera 200. Here, the CPU 101 (second vehicle control unit 1016) performs control to bring the vehicle 400 closer to the instructor when the instruction operation is predicted. Thereby, the vehicle 400 can be brought close to the instructor without waiting for a response operation from the instructor.

(Second Embodiment)
Next, a second embodiment will be described with reference to FIG. FIG. 6 is a flowchart showing the operation of the second exemplary embodiment of the present invention. In FIG. 6, the process of S28 is added to the flowchart shown in FIG. Note that the system configuration of this embodiment is the same as that of the first embodiment.

  In S28, when a response operation is detected (S25: YES), the CPU 101 (signal transmission unit 1012) transmits a second signal different from the first signal. For example, the CPU 101 sends a signal (second signal) for notifying that the response operation of the instructor has been detected by blinking the vehicle light for a predetermined period (S20). The CPU 101 may sound a horn for a predetermined period or display “OK” on the display.

  According to the present embodiment, it is possible to notify the instructor that the response operation of the instructor has been detected. As a result, it is possible to give a sense of security to the instructor around the vehicle.

(Third embodiment)
Next, a third embodiment will be described with reference to FIGS. FIG. 7 is a flowchart showing the operation of the third exemplary embodiment of the present invention. In FIG. 7, the processes of S45 and S50 are added to the flowchart shown in FIG. Note that the system configuration of this embodiment is the same as that of the first embodiment.

  In this embodiment, there are at least two instructors. Hereinafter, in order to facilitate understanding, an example in which two instructors (guides) instruct a vehicle 400 during automatic driving in a parking lot will be described.

  As in the first embodiment, the CPU 101 executes the processing from S15 to S35 and detects the instruction operation of the first instructor (S35: YES). For example, the instruction operation of the first instructor indicates an instruction to move forward in the right direction, and includes an operation of pointing to the next instructor.

  CPU101 (1st vehicle control part 1015) controls vehicle 400 according to the instruction | indication which a 1st instructor's instruction | indication operation | movement shows, when the instruction | indication operation | movement of a 1st instructor is detected (S35: YES). (S40). Specifically, the CPU 101 performs control to advance the vehicle 400 in the right direction in accordance with an instruction indicated by the instruction operation of the first instructor.

  The CPU 101 (instruction operation detection unit 1014) determines whether or not the instruction operation of the second instructor in the direction pointed by the first instructor is detected from the image captured by the camera 200 (S45). .

  That is, as shown in FIG. 8, the CPU 101 (instruction action detection unit 1014) determines that the instruction operation of the first instructor includes the action of pointing the second instructor in the instruction action of the first instructor. Subsequently, the instruction operation of the second instructor is detected. Here, when the camera 200 has a telephoto zoom function, the CPU 101 may cause the second instructor to zoom in on the camera 200.

  CPU101 (1st vehicle control part 1015) controls the vehicle 400 according to the instruction | indication which 2nd instruction | indication operation | movement shows, when the instruction | indication operation | movement of a 2nd instructor is detected. Specifically, CPU 101 parks vehicle 400 in accordance with the instruction indicated by the instruction operation of the second instructor.

  That is, the CPU 101 (first vehicle control unit 1015) controls the vehicle 400 according to the instruction indicated by the instruction operation of the first instructor, and then performs the vehicle according to the instruction indicated by the instruction operation of the second instructor. 400 is controlled.

  According to this embodiment, an instruction can be taken over from the first instructor to the second instructor.

(Modification)
Next, a modification of the third embodiment of the present invention will be described with reference to FIG. FIG. 9 is a diagram illustrating a distance R1 from the camera 200 to the first instructor and a distance R2 from the camera 200 to the second instructor.

  In the third embodiment, control for bringing the vehicle 400 closer to the second instructor is not performed, but the CPU 101 (second vehicle control unit 1016) sets the vehicles 400 in order of increasing distance from the camera 200, respectively. Control may be performed to approach the instructor. For example, in the example of FIG. 9, since the distance R1 is smaller than the distance R1, the CPU 101 (second vehicle control unit 1016) brings the vehicle 400 close to the first instructor, and then moves the vehicle 400 to the second instructor. Control to bring it closer to.

  Thereby, priority can be given to two or more instructors.

  In addition, this invention is not limited to above-described embodiment, Various modifications are included. For example, the above-described embodiment has been described in detail for easy understanding of the present invention, and is not necessarily limited to the one having all the configurations described. Further, a part of the configuration of an embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of an embodiment. In addition, it is possible to add, delete, and replace other configurations for a part of the configuration of each embodiment.

  For example, the process of S28 (sending the second signal) of the second embodiment shown in FIG. 6 is added between the process of S25 and the process of S30 in the flowchart of the third embodiment shown in FIG. May be. Thereby, it is possible to notify the instructor that the response operation of the instructor has been detected. In addition to the processing of S28, the CPU 101 may output a sound “approaching” to the speaker or display that effect on the display. Accordingly, it is possible to give a notice to the instructor that the vehicle 400 is approaching the instructor and to give the instructor a sense of security.

  In addition, a process in which the signal transmission unit 1012 transmits the third signal may be added between the process of S35 and the process of S40 of the second embodiment illustrated in FIG. Thereby, it is possible to notify the instructor that the instruction operation of the instructor has been detected. Note that the first signal (first embodiment), the second signal (second embodiment), and the third signal are different from each other, but may be the same signal. Thereby, since control can be simplified, design / manufacturing cost can be reduced.

  In the first to third embodiments, in the process of S30 shown in FIGS. 3, 6, and 7, the CPU 101 performs control to bring the vehicle 400 closer to the instructor, but the camera 200 has a telephoto zoom function. The CPU 101 may cause the first instructor to zoom in on the camera 200 instead of the process of S30. Thereby, even if the vehicle 400 does not approach the instructor, the instruction operation can be easily detected.

  In the first to third embodiments, in the process of S20 shown in FIGS. 3, 6, and 7, for example, the light of the vehicle 400 is turned on as the first signal, but the traffic regulations of the country where the vehicle 400 is used. If this is recognized, the hazard may be turned on (flashing).

  In Modification 1 of the first embodiment shown in FIGS. 4A and 4B, the camera 200 is focused on the instructor by controlling the vehicle 400 close to the instructor according to the angle of view of the camera 200. The instruction operation detection unit 1014) may detect an instruction operation from an image (region) having the highest resolution among images captured by the camera 200. Thereby, the instruction operation can be easily detected.

  In the first to third embodiments, the instructor is a human, but the instructor may be a robot. In addition, when a sound sensor such as a microphone mounted on the vehicle 400 detects a predetermined sound such as “stop”, the CPU 101 (first vehicle control unit 1015) performs control for emergency stopping the vehicle 400. Also good. Thereby, safety is improved. Further, a signboard or the like on which a predetermined marker (for example, a rectangular frame) is drawn is placed behind the instructor, and the CPU 101 (instructor detecting unit 1011) starts from an image area (inside the rectangular frame) where the marker is detected. An instructor may be detected. This facilitates detection of the instructor. The instructor in the present embodiment is not necessarily limited to a human or a robot, and may include artificial intelligence and guidance signs (all known configurations such as lightning, analog method, and digital method).

  Further, as shown in FIG. 10, the CPU 101 (instruction operation detection unit 1014) may determine the authority of the instructor from an object 500 (reflecting material, object having a predetermined pattern) held by the instructor. Here, the correspondence relationship between the object and authority held by the instructor is stored in the memory 102 in advance. CPU101 (1st vehicle control part 1015) permits control of the vehicle 400 according to authority. Thereby, authority, such as a general person and a guidance person, can be changed, for example.

  Each of the above-described configurations, functions, and the like may be realized by hardware, for example, by designing a part or all of them with an integrated circuit. Each of the above-described configurations, functions, and the like may be realized by software by interpreting and executing a program that realizes each function by the processor. Information such as programs, tables, and files for realizing each function can be stored in a recording device such as a memory, a hard disk, or an SSD (Solid State Drive), or a recording medium such as an IC card, an SD card, or a DVD.

100 ... ECU
101 ... CPU
102 ... Memory 103 ... I / F
DESCRIPTION OF SYMBOLS 200 ... Camera 300 ... Light source, sound source, or display 400 ... Vehicle 500 ... Object 1011 ... Instructor detection part 1012 ... Signal sending part 1013 ... Response action detection part 1014 ... Instruction action detection part 1015 ... First vehicle control part 1016 ... Second vehicle control unit

Claims (10)

An instructor detection unit for detecting an instructor;
An instruction operation detector for detecting an instruction operation indicating an instruction of the instructor;
A first vehicle control unit configured to control a vehicle in response to the instruction indicated by the instruction operation when the instruction operation is detected;
A vehicle control device comprising: The vehicle control device according to claim 1,
A vehicle control device, further comprising: a signal sending unit that sends a first signal to the instructor when the instructor is detected. The vehicle control device according to claim 2,
A response operation detecting unit for detecting a response operation indicating a response of the instructor;
A second vehicle control unit that performs control to bring the vehicle closer to the instructor when the response operation is detected;
The vehicle control device further comprising: The vehicle control device according to claim 3,
The signal sending unit
A vehicle control device that sends a second signal different from the first signal when the response operation is detected. The vehicle control device according to claim 1,
A second vehicle control unit;
The instruction operation detection unit
Predicting the instruction action,
The second vehicle control unit
When the said instruction | indication operation | movement is estimated, control which brings the said vehicle close to the said instructor is performed. The vehicle control apparatus characterized by the above-mentioned. The vehicle control device according to claim 1,
The instructor includes at least a first instructor and a second instructor,
The instruction operation detection unit
When the instruction operation of the first instructor includes an operation of pointing the second instructor, the instruction operation of the second instructor is detected following the instruction operation of the first instructor. And
The first vehicle control unit
The vehicle is controlled in accordance with an instruction indicated by the instruction operation of the second instructor after the vehicle is controlled in accordance with an instruction indicated by the instruction operation of the first instructor. Control device. The vehicle control device according to claim 1,
The instruction operation detection unit
Determine the authority of the instructor from what the instructor holds,
The first vehicle control unit
A vehicle control device that permits control of the vehicle according to the authority. A vehicle control system including the vehicle control device according to claim 3,
A camera for capturing images;
Including at least one of a light source, a sound source, or a display;
The vehicle control device includes:
The instructor detection unit for detecting the instructor from an image captured by the camera;
A signal sending unit for sending the first signal to the instructor by controlling at least one of the light source, the sound source, or the display when the instructor is detected;
The response operation detecting unit for detecting the response operation from an image captured by the camera;
The second vehicle control unit that performs control to bring the vehicle closer to the instructor when the response operation is detected;
The instruction operation detection unit for detecting the instruction operation from an image captured by the camera;
A vehicle control system comprising: the first vehicle control unit that controls the vehicle according to the instruction indicated by the instruction action when the instruction action is detected. The vehicle control system according to claim 8,
The second vehicle control unit
A vehicle control system that performs control to bring the vehicle closer to the instructor according to an angle of view of the camera. The vehicle control system according to claim 8,
The instructor is 2 or more,
The second vehicle control unit
A vehicle control system that performs control such that the vehicle approaches each of the instructors in order of increasing distance from the camera.

Images