JP7478285B2 - Vehicle, vehicle control method, and vehicle control program - Google Patents

Description

The present invention relates to a vehicle, a vehicle control method, and a vehicle control program, and in particular to a vehicle equipped with a vehicle control device that controls the traveling of the vehicle based on information on the current position of the vehicle, a vehicle control method, and a vehicle control program.

In recent years, vehicles equipped with ADAS (Advanced Driver Assistance System) have become known that enable automatic driving by detecting information about the external environment around the vehicle itself and controlling the driving of the vehicle on behalf of the driver to ensure safety and comfort for the driver.
In addition, when a driving disruption occurs due to a malfunction of the vehicle, etc., a remote driving technology is known that controls the driving of the vehicle through communication via a network in order to safely stop the vehicle or allow it to continue driving safely (for example, see Patent Document 1).

The vehicle control device described in Patent Document 1 allows the selection of a remote driving mode, an automatic driving mode, or a driving assistance mode, and the vehicle can be controlled in the selected operation mode. When the remote driving mode is selected, the operator remotely operates the vehicle using a remote driving device connected to the control device via a network.

JP 2020-164056 A JP 2020-32873 A

Meanwhile, in a vehicle remote control system such as that described in Patent Document 1, there is a demand for technology that can accurately measure the current position of a traveling vehicle, and this technology is being used to control autonomous driving and for navigation services that provide drivers or operators with directions to their destinations.
For example, in the automatic driving method described in Patent Document 2, GPS signals are received while the vehicle is traveling to obtain the vehicle's position (absolute position) in real time, and if the reliability of the vehicle's position accuracy decreases, the coordinates and azimuth angle based on the GPS (Satellite Positioning System) are aligned with the coordinates and azimuth angle based on an inertial measurement unit (IMU) to correct the absolute position.
Under these circumstances, there was a demand for technology that could more accurately measure the current location of moving vehicles.

The present invention has been made in consideration of the above problems, and an object of the present invention is to provide a vehicle equipped with a vehicle control device that can more accurately measure the current position of a traveling vehicle, a vehicle control method, and a vehicle control program.

The above-mentioned problem is solved by a vehicle of the present invention, which includes a vehicle control device that controls traveling of the vehicle , and an inertial measurement unit that measures information on acceleration and angular velocity of the vehicle , and the vehicle control device includes an external information acquisition unit that acquires, via a GNSS receiver mounted on the vehicle , GNSS information required for independent positioning and GNSS correction information required for relative positioning from an external reference station, the GNSS information and a corrected absolute position calculation unit that calculates a corrected absolute position of the vehicle based on the GNSS information and information on the acceleration and angular velocity of the vehicle measured by the inertial measurement unit, and the GNSS correction information and the corrected absolute position calculation unit that calculates a corrected absolute position of the vehicle based on the GNSS information and the information on the acceleration and angular velocity of the vehicle measured by the inertial measurement unit. The vehicle is equipped with a corrected relative position calculation unit that calculates a corrected relative position of the vehicle based on information on acceleration and angular velocity of the vehicle measured by an inertial measurement device , and a position identification unit that identifies position information of the vehicle, wherein when the GNSS information can be acquired in real time and the GNSS correction information can also be acquired in real time, the position identification unit identifies the position information of the vehicle using the corrected relative position, and when the GNSS correction information cannot be acquired in real time, the position identification unit identifies the position information of the vehicle using the corrected absolute position .

The above problem can also be solved by a vehicle control method using a vehicle equipped with a vehicle control device and an inertial measurement device , in which the vehicle control device, which controls the traveling of the vehicle , performs the following steps : acquiring GNSS information necessary for independent positioning and GNSS correction information necessary for relative positioning from an external reference station through a GNSS receiver mounted on the vehicle; calculating a corrected absolute position of the vehicle based on the GNSS information and information on acceleration and angular velocity of the vehicle measured by the inertial measurement device ; calculating a corrected relative position of the vehicle based on the GNSS correction information and information on acceleration and angular velocity of the vehicle measured by the inertial measurement device ; and identifying position information of the vehicle , in which, in the step of identifying the position information of the vehicle, when the GNSS information can be acquired in real time and the GNSS correction information can also be acquired in real time, the corrected relative position is used to identify the position information of the vehicle, and when the GNSS correction information cannot be acquired in real time, the corrected absolute position is used to identify the position information of the vehicle .
The above problem can also be solved by a vehicle control program that causes a computer, which is mounted on the vehicle and serves as a vehicle control device that controls the traveling of the vehicle , to execute the following processes: acquiring GNSS information necessary for individual positioning and GNSS correction information necessary for relative positioning from an external reference station through a GNSS receiver mounted on the vehicle; calculating a corrected absolute position of the vehicle based on the GNSS information and information on acceleration and angular velocity of the vehicle measured by an inertial measurement unit; calculating a corrected relative position of the vehicle based on the GNSS correction information and information on acceleration and angular velocity of the vehicle measured by the inertial measurement unit; and identifying position information of the vehicle , in which, in the process of identifying the position information of the vehicle, when the GNSS information can be acquired in real time and the GNSS correction information can also be acquired in real time, the corrected relative position is used to identify the position information of the vehicle, and when the GNSS correction information cannot be acquired in real time, the corrected absolute position is used to identify the position information of the vehicle .

The vehicle, vehicle control method, and vehicle control program of the present invention make it possible to more accurately measure the current position of a traveling vehicle.

1 is a diagram illustrating the overall configuration of a vehicle remote control system according to an embodiment of the present invention. FIG. 2 is a diagram illustrating a hardware configuration of the vehicle remote control system (excluding the control device). FIG. 2 is a diagram illustrating a hardware configuration of an operation device. FIG. 2 is a diagram illustrating functions of a vehicle control device and an operation device. 10A and 10B are diagrams for explaining processing by a position specifying unit, and are diagrams comparing the position accuracy of absolute positions, relative positions, and corrected relative positions. FIG. 2 is a process flow diagram showing a vehicle remote control method according to the present embodiment.

Hereinafter, an embodiment of the present invention will be described with reference to FIGS.
1, the vehicle remote operation system S of this embodiment is a system that realizes "automatic driving" in which the external environment of a traveling vehicle V is grasped, a driving route of the vehicle V is planned on behalf of the driver, and the vehicle V is driven by controlling the vehicle V based on the driving route, and "remote driving" in which an operator outside the vehicle V remotely controls (externally controls) the vehicle V to drive the vehicle, and is capable of performing a "mode switching process" that switches between the automatic driving mode and the remote driving mode. In addition, there is a manual driving mode (details of which will be described later) in which the driver gets into the vehicle V and drives the vehicle, and the mode switching process can switch between the manual driving mode and the automatic driving mode as well as between the manual driving mode and the remote driving mode.
The operator does not have to be a human being, but may be, for example, an AI (artificial intelligence).

<Hardware configuration of vehicle remote control system>
As shown in Figures 1 to 3, the vehicle remote control system S is equipped with a vehicle control device 1 that is mounted on a vehicle V and comprehensively controls the driving of the vehicle V, an on-board sensor 10 that detects the external environment around the vehicle V, an on-board locator 20 that receives GNSS signals from an artificial satellite SA and a reference station ST and measures the current position of the vehicle V, an on-board ECU 30 that controls the steering, acceleration/deceleration, etc. of the vehicle V, and an on-board communication device 40 that communicates with an operating device 50 installed outside the vehicle V and external devices.
The vehicle remote control system S is also connected to the vehicle control device 1 via a network (digital communication path) and includes an operation device 50 for operating the traveling of the vehicle V by communication via the network. Of course, the vehicle control device 1 and the operation device 50 may communicate directly with each other.

As shown in FIG. 2, the vehicle control device 1 is a computer connected to an on-vehicle sensor 10, an on-vehicle locator 20, an on-vehicle ECU 30, and an on-vehicle communication device 40 via an on-vehicle network (CAN).
Specifically, it is a computer equipped with a CPU as a data calculation and control processing device, ROM, RAM and HDD (SSD) as storage devices, and a communication interface for sending and receiving information data via an in-vehicle network.
In addition to a main program that performs the necessary functions of a computer, the memory device of the vehicle control device 1 also stores a vehicle control program and a vehicle remote operation program, and the functions of the vehicle control device 1 are performed by executing these programs by the CPU.
The in-vehicle ECU 30 (the integrated ECU 31) and the operation device 50 are also computers having similar hardware configurations.

In order to perform "autonomous driving," the vehicle control device 1 controls the driving of the vehicle V by controlling the on-board ECU 30 (overall ECU 31) based on information on the external environment obtained from the on-board sensor 10, information on the current location obtained from the on-board locator 20, and vehicle information obtained from the on-board ECU 30.
Furthermore, in order to execute "remote driving", the vehicle control device 1 wirelessly communicates with the operation device 50 via the in-vehicle communication device 40, and transmits information on the external environment, information on the current position, and vehicle information to the operation device 50. The operation device 50 receives this information, and displays content based on the information on the external environment and the information on the current position on the monitor 51 (navigation monitor 52), and can also notify the user to the operator.
More specifically, the vehicle control device 1 is newly installed on a vehicle V that is already equipped with an "autonomous driving function (on-vehicle sensor 10, on-vehicle locator 20, on-vehicle ECU 30)," thereby improving the performance of the existing "autonomous driving function" and adding a new "remote driving function."

The on-board sensor 10 detects the external environment surrounding the vehicle V, such as moving objects (other vehicles, pedestrians, etc.), various structures, road shapes, etc. around the vehicle V, and specifically, is mainly composed of multiple imaging devices 11, multiple radars 12, and multiple lidars 13.
The on-vehicle sensor 10 may further include detection sensors other than those described above.

The photographing device 11, also referred to as an imaging device, is a small photographing camera (wide-angle camera) that photographs (takes pictures) external images around the vehicle V. It creates external image data and transmits the external image data to the vehicle control device 1 in order to perform a "sensing function" for driving control of the vehicle V and a "monitoring function" for the driver (operator).
Multiple photographing devices 11 are mounted on the vehicle V, and include a first photographing device 11a, a second photographing device 11b, and a third photographing device 11c that are attached to the windshield of the vehicle V and photograph the front, right side, and left side of the vehicle V, a fourth photographing device 11d that is attached to the back bumper of the vehicle V and photographs the rear of the vehicle V, and a fifth photographing device 11e and a sixth photographing device 11f that are attached to the left and right mirrors of the vehicle V and photograph the right rear and left rear diagonal directions of the vehicle V, as main cameras.
In addition, the photographing device 11 is equipped with a seventh photographing device 11g, which is attached to the front bumper of the vehicle V as a sub-camera and photographs the area in front of the vehicle V, and an eighth photographing device 11h and a ninth photographing device 11i, which are attached around the left and right backlights of the vehicle V and photograph the area diagonally rear to the right and left of the vehicle V.
In this embodiment, a total of nine imaging devices 11 are attached to predetermined positions of the vehicle V, but the number and attachment positions of the imaging devices 11 can be changed depending on the type and shape of the vehicle V. The same applies to the radar 12 and the lidar 13.
As another example of the sub-camera, the seventh imaging device 11g may be attached to the upper part of the back window (rear window) of the vehicle V and may capture an image of the rear of the vehicle V from that position. In that case, it is preferable that the eighth imaging device 11h is attached to the front right A-pillar of the vehicle V and the ninth imaging device 11i is attached to the front left A-pillar.

The radar 12 is a millimeter wave radar that transmits radio waves while continuously changing the irradiation direction, detects a target object by receiving reflected waves from the target object (measures the position and speed of the target object), and performs three-dimensional spatial imaging. Compared to the imaging device 11 and the lidar 13, it can perform detection with high accuracy even in environmental conditions such as poor visibility at night or in bad weather.
The radar 12 acquires detection result data (detection signal) of the target object, and transmits the detection result data to the vehicle control device 1 .
Multiple radars 12 are mounted on the vehicle V, and include a first radar 12a and a second radar 12b mounted around the left and right front lights of the vehicle V, and a third radar 12c and a fourth radar 12d mounted around the left and right back lights of the vehicle V.
The radar 12 is not particularly limited to a millimeter wave radar, but may be a laser radar, an ultrasonic sensor, or the like.

The lidar 13 is a remote sensor that measures the distance to a target object by emitting a laser light and receiving the reflected light from the target object, and performs three-dimensional spatial imaging. Compared to the imaging device 11 and the radar 12, the lidar 13 can measure the distance to surrounding target objects in units of a few centimeters.
The LIDAR 13 acquires distance measurement data that measures the distance to the target object, and transmits the distance measurement data to the vehicle control device 1 .
Multiple riders 13 are mounted on the vehicle V, and include a first rider 13a and a second rider 13b attached around the left and right front lights of the vehicle V, a third rider 13c attached to the back bumper of the vehicle V, and a fourth rider 13d and a fifth rider 13e attached around the left and right back lights of the vehicle V.

The on-board locator 20 measures the current position of the vehicle V by utilizing a satellite positioning system using an artificial satellite SA and a reference station ST, and also measures the acceleration and angular velocity of the vehicle V to improve the accuracy of measuring the current position.
Specifically, the on-board locator 20 includes a GNSS receiver 21 that receives GNSS radio waves (GPS radio waves) from multiple artificial satellites SA, and an inertial measurement unit 22 that measures the acceleration and angular velocity of the vehicle V.

The GNSS receiver 21 is specifically an RTK-GNSS receiver that receives GNSS radio waves from multiple (specifically, four) artificial satellites SA, generates "GNSS information" necessary for single-unit positioning, and also receives "GNSS correction information" necessary for relative positioning from an external reference station ST.
The reference station ST is a fixed reference station set at a known point, receives GNSS radio waves from multiple artificial satellites SA, generates “GNSS correction information”, and transmits it to the GNSS receiver 21 .
“GNSS information” is distance information between multiple artificial satellites SA and the GNSS receiver 21 .
"GNSS correction information" is distance information in which the measurement error of the "GNSS information" is corrected by the reference station ST located at a known point receiving GNSS radio waves and communication between the reference station ST and the GNSS receiver 21.

The inertial measurement unit 22, also known as an IMU, is equipped with a three-axis gyro sensor (angular velocity sensor) and a three-axis acceleration sensor (accelerometer), measures the three-dimensional angular velocity and acceleration of the vehicle V, and transmits information on the acceleration and angular velocity of the vehicle V to the vehicle control device 1.
The vehicle control device 1 can measure the current position of the vehicle V with a smaller error range by combining GNSS information (GNSS correction information) received from the GNSS receiver 21 with the angular velocity and acceleration information of the vehicle V received from the inertial measurement device 22.

The on-board ECU 30 is, for example, an ECU for ADAS, and is equipped with an upper-level overall ECU 31 that is connected to the vehicle control device 1 and transmits and receives various data, and a steering ECU 32, an accelerator ECU 33, and a brake ECU 34 that are connected to the upper-level overall ECU 31 and control the steering, acceleration, deceleration, etc. of the vehicle V in a subdivided manner, forming a hierarchical structure.
The steering wheel ECU 32 is also called a driving support computer, and the accelerator ECU 33 and the brake ECU 34 are also called a power management control unit.
The number and functions of the individual ECUs connected to the integrated ECU 31 are not particularly limited to the above-mentioned three ECUs 32-34, and other ECUs may be provided at the same hierarchical level as these ECUs.

The steering wheel ECU 32 controls the electric power steering V1 of the vehicle V in response to instructions from the integrated ECU 31, and mainly controls the direction in which the vehicle V travels.
The electric power steering V1 includes a steering mechanism that steers the front wheels of the vehicle V. For example, in a manual driving mode, the front wheels of the vehicle V are steered by the driver's steering operation of the steering wheel V1a.
The steering wheel V1a is equipped with a torque sensor and an angle sensor, and a "mode switching process" for the driving mode can be performed based on the detection results of these sensors.

The accelerator ECU 33 controls the electric throttle V2 of the vehicle V in response to instructions from the integrated ECU 31, and mainly controls the acceleration and deceleration of the vehicle V.
The electric throttle V2 includes a drive mechanism that outputs a drive force to rotate the drive wheels of the vehicle V. For example, in a manual driving mode, the engine output is adjusted in response to the accelerator operation of the accelerator pedal V2a by the driver.

The brake ECU 34 controls the electromagnetic brake device V3 of the vehicle V in response to instructions from the integrated ECU 31, and mainly controls the deceleration and stopping of the vehicle V.
The electromagnetic brake device V3 is attached to each wheel of the vehicle V and has a mechanism for applying resistance to the rotation of the wheels to slow down or stop the vehicle V. For example, in the manual driving mode, the operation of the electromagnetic brake device V3 is adjusted in response to the braking operation of the brake pedal V3a by the driver.

The in-vehicle communication device 40 is a device that communicates information with an operation device 50 installed outside the vehicle V and an external server (not shown) via a network. For example, the in-vehicle communication device 40 transmits information on an external image acquired by the vehicle control device 1 as information necessary for "remote driving" and information on the current position to the operation device 50. The in-vehicle communication device 40 also receives driving operation information for the vehicle V from the operation device 50 that has accepted a user input by an operator, and transmits the information to the vehicle control device 1.
The in-vehicle communication device 40 communicates with an external server (not shown) and can receive, for example, the latest traffic information, weather information, and the like from the external server.

As shown in FIG. 3, the operation device 50 is a computer that is operated by an operator and performs “remote driving” of the vehicle V. The specific hardware configuration of the operation device 50 includes a plurality of monitors 51, a navigation monitor 52, a steering wheel 53, an accelerator pedal 54, a brake pedal 55, and a plurality of operation switches 56.
The operating device 50 may further include components such as a speaker, a microphone, and a shift lever.

The monitor 51 and the navigation monitor 52 are display units that output visual information for performing "remote driving", and the monitor 51 displays a composite video (composite image) that is a composite of external images of the vehicle V captured by multiple photographing devices 11a-11i based on specified layout information.
The predetermined layout information is, for example, a display mode of a layout that does not create blind spots for the operator and is easy for the operator to operate. In this case, the predetermined layout information may be stored in a predetermined storage unit of the vehicle V in association with a layout ID (layout identification information). In this case, after switching to the layout display using an operation switch or the like described later, a change operation of the layout information is further performed, and the changed layout ID is transmitted from the operation device 50 (operation switch 56) to the vehicle V. At this time, a composite image is generated in the vehicle V by combining an external image based on the layout information for the changed layout ID, and the composite image is displayed on the monitor 51, as will be described in detail later.

The handle 53 is an operating part that is operated by an operator and is used to adjust the steering angle (steering amount) of the vehicle V.
The accelerator pedal 54 and the brake pedal 55 are operating parts that are operated by an operator and are used to adjust the drive of the electric throttle V2 of the vehicle V and the operation of the electromagnetic brake device V3, respectively.
The multiple operation switches 56 are used to allow the user to input setting information for performing, for example, "remote driving." For example, by an operator appropriately operating the operation switches 56, the external image (composite image) of the vehicle V can be switched to a predetermined layout display, or the driving mode can be switched between an automatic driving mode and a remote driving mode.

<Vehicle remote control system functions>
As shown in FIG. 4 , from a functional standpoint, the vehicle control device 1 mainly comprises a memory unit 100 for storing various programs and various data, an external information acquisition unit 101, an absolute position calculation unit 102, a relative position calculation unit 103, a corrected position calculation unit 104, a reception determination unit 105, a position identification unit 106, an image processing unit 107, a communication unit 108, a vehicle control unit 109, a speed calculation unit 110, and a position information selection unit 111.
These are composed of a CPU, ROM, RAM, HDD, a communication interface, and various programs.

Regarding the operation device 50 from a functional perspective, its main components are a memory unit 500 that stores various programs and various data, a communication unit 501 that transmits and receives various data to and from the vehicle control device 1, a screen display unit 502 that displays external images and vehicle information of the vehicle V on the monitor 51 and also displays content based on information on the current position of the vehicle V (e.g., vehicle navigation) on the navigation monitor 52, an operation data creation unit 503 that accepts user operation input and creates operation data, and a user notification unit 504 that notifies the user to the operator.

The functions of the vehicle control device 1 will be described in detail below.
＜＜Main Functions＞＞
The external information acquisition unit 101 acquires “detection information of the external environment” around the vehicle V from the on-board sensor 10 , and also acquires “measurement information of the current position” of the vehicle V from the on-board locator 20 .
In detail, as "detection information of the external environment", external image data around the vehicle V is obtained from the imaging device 11, detection result data of target objects around the vehicle V is obtained from the radar 12, and distance measurement data measuring the distance between the vehicle V and the target object is obtained from the lidar 13.
In addition, as “measurement information of the current position”, GNSS information (GNSS correction information) is obtained from the GNSS receiver 21, and information on the angular velocity and acceleration of the vehicle V is obtained from the inertial measurement unit 22.
The external information acquisition unit 101 may further acquire "vehicle information" of the vehicle V from the in-vehicle ECU 30. Examples of the "vehicle information" include "steering angle information" obtained from the steering wheel ECU 32, "throttle opening information" obtained from the accelerator ECU 33, and "brake depression amount information" obtained from the brake ECU 34.

The absolute position calculation unit 102 acquires the above-mentioned "GNSS information" necessary for independent positioning, and calculates the "absolute position" of the vehicle V by independent positioning.
The "absolute position" of vehicle V is the three-dimensional position of vehicle V obtained by receiving GNSS radio waves from multiple satellites SA, measuring the distance between vehicle V and each of the satellites SA located at known points, and solving a three-dimensional equation that determines unknown points from each measured distance (corresponding to GNSS information).
As shown in FIG. 5, the position accuracy of the "absolute position" is about ±10 m.

The relative position calculation unit 103 acquires the above-mentioned “GNSS correction information” necessary for relative positioning, corrects the “absolute position” by relative positioning, and calculates the “relative position” of the vehicle V.
The "relative position" of vehicle V is the three-dimensional position of vehicle V that is determined by receiving GNSS radio waves at a reference station ST located at a known point, obtaining distances with smaller measurement error from the reference station ST (the distances between each artificial satellite SA and vehicle V), and calculating each measured distance (corresponding to GNSS correction information).
As shown in Fig. 5, the position accuracy of the "relative position" is about ±40 cm, which is higher than the position accuracy of the absolute position. In Fig. 5, the "GNSS correction information" is expressed as "GNSS information + RTK information".
The method for calculating the "relative position" includes an RTK positioning method (interferometric positioning method) and a DGPS positioning method (relative positioning method). Either method may be used.
In addition, the above-mentioned reference station ST is, in principle, the reference station installed at the position closest to the vehicle V among a plurality of reference stations ST located at known points.

The corrected position calculation unit 104 acquires the above-mentioned "angular velocity and acceleration information" of the vehicle V, and calculates a "corrected absolute position" by correcting the absolute position of the vehicle V based on the "GNSS information" and the "acceleration and angular velocity information".
The "corrected absolute position" of the vehicle V is the three-dimensional position of the vehicle V obtained by positioning the vehicle V by combining GNSS information with information on the angular velocity and acceleration of the vehicle V (also called IMU information).
The position accuracy of the "corrected absolute position" is higher than that of the absolute position.

In addition, the corrected position calculation unit 104 calculates a "corrected relative position" by correcting the relative position of the vehicle V based on the "GNSS correction information" and the "acceleration and angular velocity information."
As shown in FIG. 5, the position accuracy of the "corrected relative position" is about ±5 cm, which is higher than the position accuracy of the absolute position and the relative position.
In FIG. 5, "acceleration and angular velocity information" is expressed as "IMU information" and described as "GNSS information + RTK information + IMU information."

The reception determination unit 105 determines whether or not GNSS information can be received in real time, and if it determines that GNSS information can be received in real time, it subsequently determines whether or not GNSS correction information can be received in real time.
Specifically, the reception determination unit 105 assumes a case in which there is an obstacle around the vehicle V and therefore radio waves cannot be received from the artificial satellite SA, and in which data cannot be transmitted or received between the vehicle V and the reference station ST, and determines whether radio waves can be received from the artificial satellite SA and whether data can be transmitted or received between the vehicle V and the reference station ST.

The position identifying section 106 identifies the current position of the vehicle based on the result of the determination by the reception determining section 105 .
In detail, as shown in FIG. 6, when it is determined that GNSS information and GNSS information can be received in real time (S3 in FIG. 6), the position determination unit 106 determines the current position of the vehicle V using the “corrected relative position” with the highest position accuracy (S4 in FIG. 6).
In addition, if the position identification unit 106 determines that it can receive GNSS information in real time but cannot receive GNSS correction information in real time (S3 in Figure 6), it identifies the current position of the vehicle V using a "corrected absolute position" with high position accuracy (S5 in Figure 6).
Furthermore, when the position determination unit 106 determines that it is unable to receive GNSS information and GNSS correction information in real time (S2 in FIG. 6), it determines the current position of the vehicle V using an “estimated position” calculated based on the most recently received “GNSS information” and the “acceleration and angular velocity information” (S6 in FIG. 6).
The "estimated position" is a three-dimensional position of the vehicle V calculated based on the "GNSS information" received immediately before and the "acceleration and angular velocity information" from the past time when the "GNSS information" was received to the present time. The position accuracy of the "estimated position" is the same as that of the "absolute position".

The image processing unit 107 acquires external image data of the vehicle V from each of the multiple image capturing devices 11a-11i, and creates a composite image (composite image data) by combining the respective external images based on predetermined layout information.
By generating the above-mentioned composite image and transmitting the generated composite image data to the operation device 50, the amount of data to be transmitted can be reduced (the number of communication lines can be reduced) compared to the case of transmitting multiple external image data, thereby reducing the costs associated with data communication.

The communication unit 108 executes transmission and reception of data between the vehicle control device 1 and the operation device 50 by using the in-vehicle communication device 40 .
Specifically, the communication unit 108 transmits to the operation device 50 the “detection information of the external environment” obtained by the external information acquisition unit 101 and the “current location information” identified by the location identification unit 106 as information necessary for “remote driving” of the vehicle V.
The communication unit 108 may also transmit the “vehicle information” obtained by the external information acquisition unit 101 to the operation device 50 .
In addition, the communication unit 108 receives driving operation information of the vehicle V from the operation device 50 that accepts user input by the operator.

The vehicle control unit 109 controls the overall ECU 31 based on the "external environment detection information" obtained by the external information acquisition unit 101 and the "current location information" identified by the location identification unit 106, and performs "automatic driving" of the vehicle V.
In addition, the vehicle control unit 109 controls the overall ECU 31 based on the “driving operation information” of the vehicle V obtained from the operation device 50, and performs “remote driving” of the vehicle V.
In addition, when performing “automatic driving” of the vehicle V, the vehicle control unit 109 may obtain “vehicle information” of the vehicle V from the on-board ECU 30 and further combine the “vehicle information” to control the overall ECU 31.

<<Sub-functions>>
When determining the current position of the vehicle V, "speed information" of the vehicle V may be further obtained in order to further improve the position accuracy.
Specifically, the speed calculation unit 110 acquires the above-mentioned "angular velocity and acceleration information" of the vehicle V, and calculates the "speed" of the vehicle V by integrating the acceleration and angular velocity.
Then, the corrected position calculation unit 104 performs positioning by combining the "GNSS information", the "angular velocity and acceleration information", and the new "velocity information", thereby making it possible to calculate a "corrected absolute position" with higher positional accuracy. Alternatively, it is possible to calculate a "corrected relative position".
The speed calculation unit 110 may process the "GNSS information (GNSS correction information)" and the "acceleration and angular velocity information" using a Kalman filter to calculate the "speed" of the vehicle V. In this way, the "speed" can be calculated with higher accuracy.
In addition, when identifying the current position of the vehicle V, "steering angle information" of the vehicle V may be acquired and further combined for calculation. For example, by newly mounting a steering angle sensor on the vehicle V, it is possible to acquire "steering angle information" through the steering angle sensor.
In addition, when obtaining "speed information" of the vehicle V, a wheel speed sensor may be newly installed on the vehicle V and the "speed information" may be obtained through the wheel speed sensor.

In addition, when performing "automatic driving" or "remote driving" of vehicle V in an environment where there are no obstacles around vehicle V, it may be possible to prioritize reducing the amount of data communication and the costs associated with data communication over the positional accuracy of vehicle V's current position.
Specifically, the position information selection unit 111 selects whether to use the "relative position" or the "corrected relative position" based on the detection information of the external environment obtained by the external information acquisition unit 101 before S7 in FIG. 6 .
More specifically, when the external environment detection information indicates that there are no moving objects or buildings (structures) around the vehicle V (for example, when traveling through wilderness or grassland), it is advisable to select the "relative position" in order to reduce the amount of data communication of the vehicle control device 1. In other words, it is advisable not to acquire information on the angular velocity and acceleration of the vehicle V (measurement of the angular velocity and acceleration may be temporarily suspended).
Alternatively, when the detection information of the external environment indicates that there are moving objects or buildings around the vehicle V (for example, when driving in an urban area), the position accuracy of the current position of the vehicle V should be prioritized and the "corrected relative position" should be selected.
In this way, the method of identifying the current position of the vehicle V can be appropriately selected in accordance with the external environment surrounding the vehicle V.
The selection by the position information selection unit 111 may be performed according to a selection condition set in advance or a selection condition set by the user.

The above configuration makes it possible to realize a vehicle remote control system that can flexibly measure the position information of the vehicle V in response to the external environment around the vehicle V (radio wave conditions and data communication conditions) and identify the current position with higher position accuracy.

<Method of remotely controlling the vehicle>
Next, the process of the vehicle remote control program (vehicle remote control method) executed by the vehicle remote control system S will be described with reference to FIG.
The above-mentioned program in this embodiment is a program for realizing the above-mentioned external information acquisition unit 101, absolute position calculation unit 102, relative position calculation unit 103, corrected position calculation unit 104, reception judgment unit 105, position identification unit 106, video processing unit 107, communication unit 108, and vehicle control unit 109 as functional components of a vehicle control device 1 equipped with a memory unit 100, and the CPU of the vehicle control device 1 executes this vehicle maintenance support program.
The above program is executed upon receiving an operational instruction from a user (specifically, a driver or an operator).

The vehicle remote control flow shown in FIG. 6 begins with step S<b>1 in which the external information acquisition unit 101 starts acquiring “GNSS information” through the GNSS receiver 21 .
In practice, the external information acquisition unit 101 starts acquiring “GNSS information” and “GNSS correction information” through the GNSS receiver 21 , and also starts acquiring “angular velocity and acceleration information” of the vehicle V through the inertial measurement unit 22 .
In addition, the external information acquisition unit 101 also begins acquiring “detection information of the external environment” around the vehicle V from the on-board sensor 10.

Next, in step S2, the reception determination unit 105 determines whether or not the GNSS information can be received in real time.
If it is determined that the "GNSS information" can be received (step S2: Yes), the process proceeds to step S3, where the reception determination unit 105 subsequently determines whether or not the "GNSS correction information" can be received in real time.
If the reception determination unit 105 determines that the "GNSS correction information" can also be received (step S3: Yes), the process proceeds to step S4.

In step S4, the absolute position calculation unit 102, the relative position calculation unit 103, and the corrected position calculation unit 104 calculate the "corrected relative position (relative position)" of the vehicle V.
Specifically, first, the absolute position calculation unit 102 acquires "GNSS information" and calculates the "absolute position" of the vehicle V by single-unit positioning. Then, the relative position calculation unit 103 acquires "GNSS correction information" and calculates the "relative position" of the vehicle V by relative positioning. Then, the corrected position calculation unit 104 acquires "angular velocity and acceleration information" and calculates a "corrected relative position" by correcting the relative position of the vehicle V. Then, the process proceeds to step S7.
The "corrected relative position" is the position information with the highest positional accuracy.

In the above step S3, if the reception determination unit 105 determines that the “GNSS correction information” cannot be received either (step S3: No), the process proceeds to step S5.
In step S5, the absolute position calculation unit 102 and the corrected position calculation unit 104 calculate the “absolute corrected position (absolute position)” of the vehicle V.
Specifically, first, the absolute position calculation unit 102 acquires "GNSS information" and calculates the "absolute position" of the vehicle V by single point positioning. Then, the corrected position calculation unit 104 acquires "angular velocity and acceleration information" and calculates a "corrected absolute position" by correcting the absolute position of the vehicle V.
Then, proceed to step S7.

In the above step S2, if the reception determination unit 105 determines that the "GNSS information" cannot be received (step S2: No), the process proceeds to step S6.
In step S6, the position identification unit 106 calculates the "estimated position" of the vehicle V.
Specifically, the position identification unit 106 calculates an "estimated position" based on the most recently received "GNSS information" and "acceleration and angular velocity information" (corresponding to an estimated position calculation unit).
Then, proceed to step S7.

Next, in step S7, the position identifying unit 106 identifies the current position of the vehicle V using the calculated position information.
If the process proceeds from step S4, the position identifying unit 106 identifies the current position of the vehicle V using the "corrected relative position" that has the highest positional accuracy.
If the process proceeds from step S5, the position identifying unit 106 identifies the current position of the vehicle V using the "corrected absolute position."
If the process proceeds from step S6 above, the position identifying unit 106 identifies the current position of the vehicle V using the "estimated position".

Next, in step S8, the communication unit 108 transmits to the operation device 50 the “current location information” identified by the location identification unit 106 and the “external environment detection information” obtained by the external information acquisition unit 101 as information necessary for “remote driving” of the vehicle V.
The operation device 50 receives this information, displays on the monitor 51 and the navigation monitor 52 contents based on the current position information and the detected information of the external environment, and notifies the operator of the user as necessary.

When the process goes through steps S1 to S8 and finally receives a command to stop remote operation from the user (step S9: Yes), the process of FIG. 6 ends.
On the other hand, if the instruction to stop the remote operation has not been received (step S9: No), the process returns to step S2.
The above-described configuration of the vehicle remote control program makes it possible to measure the current position of the vehicle V more accurately in response to the external environment surrounding the vehicle V.

<Other embodiments>
In the above embodiment, as shown in FIG. 2, the vehicle remote control system S includes a vehicle control device 1 and an on-board ECU 30, and newly adds a "remote driving function (vehicle control device 1)" to a vehicle V equipped with an "automatic driving function (on-board ECU 30)". However, this can be modified without any particular limitation.
For example, the vehicle control device 1 may also have the function of the on-vehicle ECU 30. That is, the vehicle remote control system S may be mainly composed of the vehicle control device 1 (including the function of the on-vehicle ECU 30), the on-vehicle sensor 10, the on-vehicle locator 20, the on-vehicle communication device 40, and the operation device 50 (the on-vehicle ECU 30 may be excluded from the configuration).

In the above embodiment, as shown in FIG. 2, the vehicle remote control system S is equipped with a vehicle control device 1 and an on-board locator 20, but without being particularly limited thereto, the vehicle control device 1 may also have an on-board locator 20.
In that case, it is preferable that the in-vehicle locator 20 has an absolute position calculation unit 102 that calculates the absolute position of the vehicle V, a relative position calculation unit 103 that calculates the relative position of the vehicle V, and a corrected position calculation unit 104 that calculates the corrected relative position of the vehicle V. Then, it is preferable that the vehicle control device 1 acquires information on the absolute position, the relative position, and the corrected relative position of the vehicle V, respectively.

In the above embodiment, as shown in Figures 2 and 4, the corrected position calculation unit 104 acquires information on the acceleration and angular velocity of the vehicle V from the inertial measurement device 22 and calculates the corrected relative position of the vehicle V, but this can be changed without any particular limitation.
For example, when the vehicle control device 1 specifies the current position of the vehicle V, it is also possible not to acquire (use) information on the acceleration and angular velocity of the vehicle V. In that case, the current position of the vehicle V is specified using the “absolute position” and “relative position” of the vehicle V.
Also, for example, when the vehicle control device 1 determines the current position of the vehicle V, it may not acquire (use) information on the acceleration and angular velocity of the vehicle V, but may acquire (use) information on the speed of the vehicle V.

In the above embodiment, as shown in FIG. 2, the photographing device 11 detects information about the external environment around the vehicle V as part of the on-board sensor 10, but is not particularly limited thereto, and the photographing device 11 may simply acquire external images around the vehicle V.
In other words, the radar 12 and the lidar 13 may function as an on-board sensor 10 to detect information about the external environment around the vehicle V.

In the above embodiment, a vehicle remote operation program is stored in a recording medium readable by the vehicle control device 1, and the vehicle control device 1 reads and executes the program to execute processing. Here, the recording medium readable by the vehicle control device 1 refers to a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, etc.
In addition, a dedicated web application may be started using a terminal (mobile terminal) serving as the vehicle control device 1, and the vehicle remote control program may be executed on a web browser.

In the above embodiment, the vehicle remote control system and the vehicle remote control method according to the present invention have been mainly described.
However, the above embodiment is merely an example for facilitating understanding of the present invention, and does not limit the present invention. The present invention can be modified and improved without departing from the spirit of the present invention, and the present invention naturally includes equivalents thereof.

S Vehicle remote control system V Vehicle V1 Electric power steering V1a Steering wheel V2 Electric throttle V2a Accelerator pedal V3 Electromagnetic brake device V3a Brake pedal 1 Vehicle control device 10 Vehicle-mounted sensor 11 Imaging device 11a-11i First imaging device-ninth imaging device 12 Radar (millimeter wave radar)
12a-12d 1st radar-4th radar 13 LIDAR 13a-13e 1st LIDAR-5th LIDAR 20 Vehicle-mounted locator 21 GNSS receiver (RTK-GNSS receiver)
22 Inertial Measurement Unit (IMU)
30 In-vehicle ECU
31 Integrated ECU
32 Steering ECU
33 Accelerator ECU
34 Brake ECU
40 In-vehicle communication device 50 Operation device 51 Display monitor 52 Display navigation monitor 53 Steering wheel 54 Accelerator pedal 55 Brake pedal 56 Operation switch 100 Memory unit 101 External information acquisition unit 102 Absolute position calculation unit 103 Relative position calculation unit 104 Corrected position calculation unit 105 Reception determination unit 106 Position identification unit 107 Image processing unit 108 Communication unit (first communication unit)
109 Vehicle control unit 110 Speed calculation unit 111 Position information selection unit 500 Storage unit 501 Communication unit (second communication unit)
502 Screen display unit 503 Operation data creation unit 504 User notification unit SA Artificial satellite ST Reference station

Claims (4)

A vehicle control device that controls the traveling of the host vehicle ;
an inertial measurement device that measures information on the acceleration and angular velocity of the host vehicle ;
The vehicle control device includes:
an external information acquisition unit that acquires GNSS information required for individual positioning and GNSS correction information required for relative positioning from an external reference station through a GNSS receiver mounted on the vehicle;
a corrected absolute position calculation unit that calculates a corrected absolute position of the host vehicle based on the GNSS information and information on the acceleration and angular velocity of the host vehicle measured by the inertial measurement device ;
a corrected relative position calculation unit that calculates a corrected relative position of the host vehicle based on the GNSS correction information and information on the acceleration and angular velocity of the host vehicle measured by the inertial measurement device ;
A position identification unit that identifies position information of the vehicle ,
The position identification unit is
When the GNSS information can be acquired in real time,
When the GNSS correction information can also be acquired in real time, the corrected relative position is used to identify the position information of the vehicle;
When the GNSS correction information cannot be acquired in real time, the vehicle identifies position information of the vehicle using the corrected absolute position. The vehicle control device acquires speed information of the host vehicle,
The corrected absolute position calculation unit further calculates the corrected absolute position by combining speed information of the host vehicle,
The vehicle according to claim 1 , wherein the corrected relative position calculation unit calculates the corrected relative position by further combining with speed information of the host vehicle. A vehicle control method using a vehicle equipped with a vehicle control device and an inertial measurement unit ,
The vehicle control device that controls the traveling of the host vehicle,
acquiring GNSS information required for individual positioning and GNSS correction information required for relative positioning from an external reference station through a GNSS receiver mounted on the vehicle ;
Calculating a corrected absolute position of the host vehicle based on the GNSS information and information on acceleration and angular velocity of the host vehicle measured by the inertial measurement device ;
calculating a corrected relative position of the host vehicle based on the GNSS correction information and information on the acceleration and angular velocity of the host vehicle measured by the inertial measurement unit ;
A step of identifying position information of the vehicle ;
In the step of identifying the position information of the vehicle,
When the GNSS information can be acquired in real time,
When the GNSS correction information can also be acquired in real time, the corrected relative position is used to identify the position information of the vehicle;
A vehicle control method , comprising: determining position information of the host vehicle using the corrected absolute position when the GNSS correction information cannot be obtained in real time. A computer as a vehicle control device mounted on a host vehicle and controlling the running of the host vehicle,
A process of acquiring GNSS information required for independent positioning and GNSS correction information required for relative positioning from an external reference station through a GNSS receiver mounted on the vehicle ;
A process of calculating a corrected absolute position of the host vehicle based on the GNSS information and information on acceleration and angular velocity of the host vehicle measured by an inertial measurement unit ;
A process of calculating a corrected relative position of the host vehicle based on the GNSS correction information and information on the acceleration and angular velocity of the host vehicle measured by the inertial measurement unit ;
and executing a process of identifying the position information of the vehicle ;
In the process of identifying the position information of the vehicle,
When the GNSS information can be acquired in real time,
When the GNSS correction information can also be acquired in real time, the corrected relative position is used to identify the position information of the vehicle;
A vehicle control program that, when the GNSS correction information cannot be obtained in real time, identifies position information of the host vehicle using the corrected absolute position .

Images