JP6978973B2 - Vehicle control device and vehicle control system - Google Patents

Description

The present invention relates to a vehicle control device and a vehicle control system.

A vehicle control system capable of reducing anxiety given to people around the vehicle is known (see, for example, Patent Document 1). In Patent Document 1, "the vehicle control system has at least one of a detection unit that detects the peripheral state of the vehicle and a speed control and a steering control of the vehicle based on the peripheral state of the vehicle detected by the detection unit. It was recognized by the automatic driving control unit that executes automatic driving automatically, the recognition unit that recognizes moving objects existing around the vehicle based on the peripheral state of the vehicle detected by the detection unit, and the recognition unit. It is provided with a transmitting unit that transmits information for calling attention to the terminal device in a direction other than the direction of the moving object. "

Japanese Unexamined Patent Publication No. 2017-207964

In the technology disclosed in Patent Document 1, people around the vehicle need to have a terminal device in order to receive information for calling attention. Also, with such technology, people around the vehicle cannot control the vehicle.

An object of the present invention is to provide a vehicle control device or the like capable of controlling a vehicle in response to an instruction of an instructor who does not have a terminal device outside the vehicle.

In order to achieve the above object, the vehicle control device of the example of the present invention includes an instructor detection unit for detecting an instructor, an instruction operation detection unit for detecting an instruction operation indicating the instruction of the instructor, and the instruction operation. When is detected, a first vehicle control unit that controls the vehicle in response to the instruction indicated by the instruction operation, a response operation detection unit that detects a response operation indicating the response of the instructor, and the response operation. When is detected, a second vehicle control unit that controls the vehicle to be brought closer to the instructor is provided.

According to the present invention, the vehicle can be controlled in response to an instruction from an instructor who does not have a terminal device outside the vehicle. Issues, configurations and effects other than those described above will be clarified by the following description of the embodiments.

It is a block diagram of the vehicle control system including the ECU as the vehicle control device by 1st Embodiment of this invention. It is a block diagram which shows the function of the CPU used in the ECU by 1st Embodiment of this invention. It is a flowchart which shows the operation of the CPU used for the ECU by 1st Embodiment of this invention. It is a figure for demonstrating the modification 1, and shows the focal length when the angle of view of a camera is wide. It is a figure for demonstrating the modification 1, and shows the focal length when the angle of view of a camera is narrow. It is a flowchart which shows the operation of the modification 2 of 1st Embodiment. It is a flowchart which shows the operation of the 2nd Embodiment of this invention. It is a flowchart which shows the operation of the 3rd Embodiment of this invention. It is a figure which shows the transfer of the instruction from the 1st instructor to the 2nd instructor. It is a figure for demonstrating the modification of the 3rd Embodiment of this invention. It is a figure for demonstrating the example which changes the authority according to the thing held by an instructor.

Hereinafter, the configuration and operation of the vehicle control system including the vehicle control device according to the first to third embodiments of the present invention will be described with reference to the drawings. In each figure, the same reference numerals indicate the same parts.

(First Embodiment)
First, the configuration of the vehicle control system will be described with reference to FIG. FIG. 1 is a configuration diagram of a vehicle control system 1 including an ECU 100 (Electronic Control Unit) as a vehicle control device according to the first embodiment of the present invention.

(System configuration)
The ECU 100 includes a CPU 101 (Central Processing Unit) as a processor, a memory 102 as a storage device, an I / F 103 as an input / output circuit, and the like. The ECU 100 is mounted on the vehicle.

The camera 200 captures an image and temporarily stores the captured image in the memory. The camera 200 is connected to the I / F 103 and controlled by the CPU 101. The camera 200 is, for example, a stereo camera.

The light source, the sound source, or the display 300 sends a signal (first signal) for confirming whether or not the instructor (human) in the vicinity of the vehicle intends to control the vehicle. The light source, sound source, or display 300 is connected to the I / F 103 and controlled by the CPU 101. The light source is, for example, a vehicle light, the sound source is, for example, a horn, and the display is, for example, an electric bulletin board provided on the front surface of a bus or the like. The light source, sound source, or display 300 is connected to the I / F 103 and controlled by the CPU 101.

The vehicle 400 includes a drive mechanism for driving the vehicle body, a braking mechanism for braking the vehicle body, a steering mechanism for steering the vehicle body, and the like.

Next, the function of the CPU 101 will be described with reference to FIG. FIG. 2 is a block diagram showing a function of the CPU 101 used in the ECU 100 according to the first embodiment of the present invention.

(function)
By executing a predetermined program stored in advance in the memory 102, the CPU 101 includes an instructor detection unit 1011, a signal transmission unit 1012, a response operation detection unit 1013, an instruction operation detection unit 1014, and a first vehicle control unit 1015. It functions as a second vehicle control unit 1016. It should be noted that these functions may be realized by a circuit such as FPGA (Field-Programmable Gate Array).

The instructor detection unit 1011 detects the instructor (human) from the image captured by the camera 200.

When the instructor is detected by the instructor detection unit 1011, the signal transmission unit 1012 confirms whether the instructor intends to control the vehicle by controlling at least one of the light source, the sound source, and the display. Signal (first signal) is sent. For example, the signal transmission unit 1012 turns on the light of the vehicle, sounds a horn, and displays a message "Do you want to control the vehicle?" On the electric bulletin board for a predetermined period.

When the instructor wants to control the vehicle, he / she performs a predetermined operation. The predetermined operation (response operation) is, for example, an operation of raising both arms, an operation of raising a flag, and the like.

The response motion detection unit 1013 detects a response motion indicating the response of the instructor from the image captured by the camera 200. The response operation then announces that the instruction operation indicating the instruction of the instructor will continue.

The instruction motion detection unit 1014 detects the instruction motion from the image captured by the camera 200. For example, when the instructor wants to stop the vehicle, both arms are horizontal, and when the vehicle is to be parked, the instructor points to the parking position and then puts one arm diagonally downward.

When the instruction operation is detected by the instruction operation detection unit 1014, the first vehicle control unit 1015 controls the vehicle 400 according to the instruction indicated by the instruction operation.

When the response motion is detected by the response motion detection unit 1013, the second vehicle control unit 1016 controls to bring the vehicle 400 closer to the instructor.

Next, the operation of the CPU 101 will be described with reference to FIG. FIG. 3 is a flowchart showing the operation of the CPU 101 used in the ECU 100 according to the first embodiment of the present invention.

(motion)
The CPU 101 (instructor detection unit 1011) determines whether or not the instructor (human) is detected from the image captured by the camera 200 (S15). For example, the CPU 101 analyzes an image and determines (recognizes) that an object that resembles a human shape stored in a memory 102 in advance and that performs a movement similar to a human behavior is a human.

When the instructor (human) is detected (S15: YES), the CPU 101 (signal transmission unit 1012) intends to control the vehicle by turning on the light of the vehicle for a predetermined period, for example. A signal (first signal) for confirming the above is transmitted (S20). On the other hand, the instructor raises, for example, both arms in order to show the intention to control the vehicle. If the instructor (human) is not detected (S15: NO), the CPU 101 repeats the process of S15.

The CPU 101 (response motion detection unit 1013) determines whether or not a response motion (raising both arms) indicating the response of the instructor is detected from the image captured by the camera 200 (S25). The information defining the response operation is stored in the memory 102 in advance, for example. The CPU 101 compares, for example, the information indicating the operation of the instructor obtained by analyzing (behavior analysis) the image captured by the camera 200 with the information defining the response operation stored in the memory 102. When they are similar, it is determined that the response operation is detected (recognized).

When the response operation is detected (S25: YES), the CPU 101 (second vehicle control unit 1016) controls to bring the vehicle 400 closer to the instructor (S30). For example, the CPU 101 advances the vehicle 400 to a position at a predetermined distance from the instructor and stops the vehicle 400. As a result, the proportion of the image of the instructor in the image captured by the camera 200 becomes large, and the instruction operation following the response operation is easily recognized in the image analysis.

When the response operation is not detected (S25: NO), the CPU 101 returns to the normal automatic operation after repeating the process of S25 for a predetermined time (S26: YES) (S27). In normal automatic driving, control is performed to move the vehicle 400 forward while avoiding humans.

Subsequently, the instructor makes both arms horizontal, for example, as an instruction operation indicating an instruction to stop the vehicle 400.

The CPU 101 (instruction motion detection unit 1014) determines whether or not the instruction motion is detected from the image captured by the camera 200 (S35). Information that defines the instruction operation is stored in the memory 102 in advance, for example. The CPU 101 compares, for example, the information indicating the operation of the instructor obtained by analyzing (behavior analysis) the image captured by the camera 200 with the information defining the instruction operation stored in the memory 102. When they are similar, it is determined that the instruction operation is detected (recognized).

When the instruction operation is detected (S35: YES), the CPU 101 (first vehicle control unit 1015) controls the vehicle 400 according to the instruction indicated by the instruction operation (S40). Specifically, the CPU 101 controls to stop the vehicle 400 according to the instruction indicated by the operation of leveling both arms (instruction operation). If the instruction operation is not detected (S35: NO), the CPU 101 repeats the process of S35.

As described above, according to the present embodiment, the vehicle can be controlled in response to an instruction from an instructor who does not have a terminal device outside the vehicle.

(Modification 1)
Next, a modification 1 of the first embodiment of the present invention will be described with reference to FIGS. 4A and 4B. FIG. 4A is a diagram showing a focal length when the angle of view of the camera 200 is wide, and FIG. 4B is a diagram showing a focal length when the angle of view of the camera 200 is narrow.

In the first modification, the CPU 101 (second vehicle control unit 1016) controls to bring the vehicle 400 closer to the instructor according to the angle of view of the camera 200 when the response operation is detected.

For example, when the angle of view θ1 of the camera 200 is wide as shown in FIG. 4A, the CPU 101 brings the vehicle 400 closer to the instructor, and the distance R to the instructor detected by the camera 200 is the focal length corresponding to the angle of view θ1. When the distance d1, the vehicle 400 is stopped.

On the other hand, when the angle of view θ2 of the camera 200 is narrow as shown in FIG. 4B, the CPU 101 brings the vehicle 400 closer to the instructor, and the distance R detected by the camera 200 becomes the focal length d2 corresponding to the angle of view θ2. At that time, the vehicle 400 is stopped. Here, the relationship of θ2 <θ1 and d1 <d2 is established.

As a result, the focus of the camera 200 is on the instructor, so that the instruction operation of the instructor can be easily detected.

(Modification 2)
Next, a modification 2 of the first embodiment of the present invention will be described with reference to FIG. FIG. 5 is a flowchart showing the operation of the second modification of the first embodiment. In FIG. 5, the processes from S20 to S30 in FIG. 3 are omitted.

In an emergency, it is assumed that the instructor will not be able to respond. Therefore, in this modification, the vehicle is controlled according to the instruction operation from the instructor without waiting for the response operation from the instructor. For example, when the instruction operation from the instructor indicates an instruction accompanying an emergency, the CPU 101 (first vehicle control unit 1015) controls to stop the vehicle 400 in a safe place. This allows the vehicle to be controlled quickly.

The CPU 101 (instruction motion detection unit 1014) may predict the instruction motion from the image captured by the camera 200. Here, the CPU 101 (second vehicle control unit 1016) controls to bring the vehicle 400 closer to the instructor when the instruction operation is predicted. As a result, the vehicle 400 can be brought closer to the instructor without waiting for the response operation from the instructor.

(Second embodiment)
Next, the second embodiment will be described with reference to FIG. FIG. 6 is a flowchart showing the operation of the second embodiment of the present invention. In FIG. 6, the process of S28 is added to the flowchart shown in FIG. The system configuration of this embodiment is the same as that of the first embodiment.

In S28, when the response operation is detected (S25: YES), the CPU 101 (signal transmission unit 1012) transmits a second signal different from the first signal. For example, the CPU 101 sends out a signal (second signal) for notifying that the response operation of the instructor has been detected by blinking the light of the vehicle for a predetermined period (S20). The CPU 101 may sound a horn or display "OK" on the display for a predetermined period.

According to the present embodiment, it is possible to notify the instructor that the response operation of the instructor has been detected. As a result, it is possible to give a sense of security to the instructor in the vicinity of the vehicle.

(Third embodiment)
Next, the third embodiment will be described with reference to FIGS. 7 and 8. FIG. 7 is a flowchart showing the operation of the third embodiment of the present invention. In FIG. 7, the processes of S45 and S50 are added to the flowchart shown in FIG. The system configuration of this embodiment is the same as that of the first embodiment.

In this embodiment, there are at least two instructors. In the following, in order to facilitate understanding, an example in which two instructors (guides) instruct the vehicle 400 during automatic driving in the parking lot will be described.

Similar to the first embodiment, the CPU 101 executes the processes from S15 to S35 and detects the instruction operation of the first instructor (S35: YES). For example, the instruction operation of the first instructor includes an operation of indicating an instruction to move forward to the right and pointing to the next instructor.

When the instruction operation of the first instructor is detected (S35: YES), the CPU 101 (first vehicle control unit 1015) controls the vehicle 400 in response to the instruction indicated by the instruction operation of the first instructor. (S40). Specifically, the CPU 101 controls to move the vehicle 400 forward to the right according to the instruction indicated by the instruction operation of the first instructor.

The CPU 101 (instruction motion detection unit 1014) determines whether or not the instruction motion of the second instructor in the direction pointed by the first instructor is detected from the image captured by the camera 200 (S45). ..

That is, as shown in FIG. 8, when the instruction operation of the first instructor includes the operation of pointing to the second instructor, the CPU 101 (instruction operation detection unit 1014) sets the instruction operation of the first instructor. Subsequently, the instruction operation of the second instructor is detected. Here, when the camera 200 has a telephoto zoom function, the CPU 101 may zoom in on the camera 200 to a second instructor.

When the instruction operation of the second instructor is detected, the CPU 101 (first vehicle control unit 1015) controls the vehicle 400 in response to the instruction indicated by the second instruction operation. Specifically, the CPU 101 parks the vehicle 400 according to the instruction indicated by the instruction operation of the second instructor.

That is, the CPU 101 (first vehicle control unit 1015) controls the vehicle 400 according to the instruction indicated by the instruction operation of the first instructor, and then the vehicle according to the instruction indicated by the instruction operation of the second instructor. Control 400.

According to the present embodiment, the instruction can be handed over from the first instructor to the second instructor.

(Modification example)
Next, a modified example of the third embodiment of the present invention will be described with reference to FIG. FIG. 9 is a diagram showing a distance R1 from the camera 200 to the first instructor and a distance R2 from the camera 200 to the second instructor.

In the third embodiment, the control to bring the vehicle 400 closer to the second instructor is not performed, but the CPU 101 (second vehicle control unit 1016) sets the vehicle 400 in the order of the closest distance from the camera 200. You may control to get closer to the instructor. For example, in the example of FIG. 9, since the distance R1 is smaller than the distance R1, the CPU 101 (second vehicle control unit 1016) brings the vehicle 400 closer to the first instructor, and then the vehicle 400 is the second instructor. Control to bring it closer to.

This makes it possible to prioritize two or more instructors.

The present invention is not limited to the above-described embodiment, and includes various modifications. For example, the above-described embodiment has been described in detail in order to explain the present invention in an easy-to-understand manner, and is not necessarily limited to the one including all the described configurations. Further, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Further, it is possible to add / delete / replace a part of the configuration of each embodiment with another configuration.

For example, the processing of S28 (transmitting the second signal) of the second embodiment shown in FIG. 6 is added between the processing of S25 and the processing of S30 in the flowchart of the third embodiment shown in FIG. 7. You may. As a result, it is possible to notify the instructor that the response operation of the instructor has been detected. In addition to the processing of S28, the CPU 101 may output a voice saying "approaching" to the speaker or display to that effect on the display. As a result, it is possible to give a notice that the vehicle 400 is approaching the instructor and give the instructor a sense of security.

Further, a process of transmitting a third signal by the signal transmission unit 1012 may be added between the processes of S35 and the process of S40 shown in FIG. As a result, it is possible to notify the instructor that the instruction operation of the instructor has been detected. The first signal (first embodiment), the second signal (second embodiment), and the above-mentioned third signal are different from each other, but may be the same signal. As a result, control can be simplified and design / manufacturing costs can be reduced.

In the first to third embodiments, in the process of S30 shown in FIGS. 3, 6 and 7, the CPU 101 controls to bring the vehicle 400 closer to the instructor, but the camera 200 has a telephoto zoom function. , The CPU 101 may zoom in on the camera 200 to the first instructor instead of the processing of S30. As a result, the instruction operation can be easily detected even if the vehicle 400 does not approach the instructor.

In the first to third embodiments, in the process of S20 shown in FIGS. 3, 6 and 7, for example, the light of the vehicle 400 is turned on as the first signal, but the traffic regulations of the country in which the vehicle 400 is used are used. If is recognized, the hazard may be turned on (blinking).

In the first modification of the first embodiment shown in FIGS. 4A and 4B, the camera 200 is focused on the instructor by controlling the vehicle 400 to be closer to the instructor according to the angle of view of the camera 200, but the CPU 101 ( The instruction motion detection unit 1014) may detect the instruction motion from the image (region) having the highest resolution among the images captured by the camera 200. Thereby, the instruction operation can be easily detected.

In the first to third embodiments, the instructor is a human being, but the instructor may be a robot. Further, when a sound sensor such as a microphone mounted on the vehicle 400 detects a predetermined voice such as "stop", the CPU 101 (first vehicle control unit 1015) controls the vehicle 400 to make an emergency stop. May be good. This improves safety. Further, a signboard or the like on which a predetermined marker (for example, a rectangular frame) is drawn is arranged behind the instructor, and the CPU 101 (instructor detection unit 1011) is from the area (inside the rectangular frame) of the image in which the marker is detected. The instructor may be detected. This facilitates the detection of the instructor. The instructor in this embodiment is not necessarily limited to humans and robots, and may include artificial intelligence and guide signs (any known configuration such as electric light, analog system, digital system, etc.).

Further, as shown in FIG. 10, the CPU 101 (instruction motion detection unit 1014) may determine the authority of the instructor from the object 500 (reflecting material, object having a predetermined pattern) held by the instructor. Here, the correspondence between the object held by the instructor and the authority is stored in the memory 102 in advance. The CPU 101 (first vehicle control unit 1015) permits control of the vehicle 400 according to the authority. Thereby, for example, the authority of a general person and a guide can be changed.

Each of the above configurations, functions, and the like may be realized by hardware, for example, by designing a part or all of them by an integrated circuit. Further, each of the above configurations, functions, and the like may be realized by software by the processor interpreting and executing a program that realizes each function. Information such as programs, tables, and files that realize each function can be placed in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.

100 ... ECU
101 ... CPU
102 ... Memory 103 ... I / F
200 ... Camera 300 ... Light source, sound source, or display 400 ... Vehicle 500 ... Object 1011 ... Instructor detection unit 1012 ... Signal transmission unit 1013 ... Response operation detection unit 1014 ... Instruction operation detection unit 1015 ... First vehicle control unit 1016 ... Second vehicle control unit

Claims (9)

The instructor detection unit that detects the instructor, and
An instruction operation detection unit that detects an instruction operation indicating the instruction of the instructor,
When the instruction operation is detected, a first vehicle control unit that controls the vehicle in response to the instruction indicated by the instruction operation, and a first vehicle control unit.
A response motion detection unit that detects a response motion indicating the response of the instructor,
When the response operation is detected, a second vehicle control unit that controls the vehicle to be brought closer to the instructor, and a second vehicle control unit.
A vehicle control device characterized by being provided with. The vehicle control device according to claim 1.
A vehicle control device further comprising a signal transmitting unit that transmits a first signal to the instructor when the instructor is detected. The vehicle control device according to claim 2.
The signal transmission unit is
A vehicle control device characterized in that when the response operation is detected, a second signal different from the first signal is transmitted. The vehicle control device according to claim 1.
The instruction operation detection unit,
Predicting the indicated operation,
The second vehicle control unit is
A vehicle control device for controlling the vehicle to be closer to the instructor when the instruction operation is predicted. The instructor detection unit that detects the instructor, and
An instruction operation detection unit that detects an instruction operation indicating the instruction of the instructor,
A vehicle control device including a first vehicle control unit that controls a vehicle in response to the instruction indicated by the instruction operation when the instruction operation is detected.
The instructor includes at least a first instructor and a second instructor.
The instruction operation detection unit is
When the instruction operation of the first instructor includes an operation of pointing to the second instructor, the instruction operation of the second instructor is detected following the instruction operation of the first instructor. death,
The first vehicle control unit is
A vehicle characterized in that after controlling the vehicle according to an instruction indicated by the instruction operation of the first instructor, the vehicle is controlled according to an instruction indicated by the instruction operation of the second instructor. Control device. The vehicle control device according to claim 1.
The instruction operation detection unit is
The authority of the instructor is determined from the object held by the instructor, and the authority of the instructor is determined.
The first vehicle control unit is
A vehicle control device characterized in that control of the vehicle is permitted according to the authority. A vehicle control system including a vehicle both controller,
A camera that captures images and
Includes at least one of a light source, sound source, or display.
The vehicle control device is
A finger示者detector that detect an image or et finger示者captured by the camera,
When the instructor is detected, a signal transmission unit that sends a first signal to the instructor by controlling at least one of the light source, the sound source, and the display.
A response operation detecting section that detect the response operation indicating a response from the captured image of the instructor in the camera,
If the response operation is detected, and the second vehicle control unit intends row control to bring the vehicles to the instructor,
Finger示動operation detecting section that detect an instruction operation from the captured image showing the indication of the instructor in the camera,
If the instruction operation is detected, the vehicle control system, characterized in that and a first vehicle control unit that controls the vehicles in accordance with the instruction the instruction operation is shown. The vehicle control system according to claim 7.
The second vehicle control unit is
A vehicle control system characterized in that control is performed to bring the vehicle closer to the instructor according to the angle of view of the camera. The vehicle control system according to claim 7.
The number of the instructors is 2 or more,
The second vehicle control unit is
A vehicle control system characterized in that control is performed to bring the vehicle closer to each instructor in order of increasing distance from the camera.

Images