CN114355943A - Remote driving system based on 5G mobile communication technology and high-definition video transmission - Google Patents
Remote driving system based on 5G mobile communication technology and high-definition video transmission Download PDFInfo
- Publication number
- CN114355943A CN114355943A CN202210005325.3A CN202210005325A CN114355943A CN 114355943 A CN114355943 A CN 114355943A CN 202210005325 A CN202210005325 A CN 202210005325A CN 114355943 A CN114355943 A CN 114355943A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- communication
- background control
- control end
- signal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 45
- 238000005516 engineering process Methods 0.000 title claims abstract description 19
- 238000010295 mobile communication Methods 0.000 title claims abstract description 17
- 238000004891 communication Methods 0.000 claims abstract description 138
- 238000012544 monitoring process Methods 0.000 claims abstract description 49
- 238000012545 processing Methods 0.000 claims abstract description 14
- 230000006855 networking Effects 0.000 claims abstract description 7
- 238000012795 verification Methods 0.000 claims description 28
- 238000011156 evaluation Methods 0.000 claims description 23
- 230000002159 abnormal effect Effects 0.000 claims description 22
- 238000000034 method Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 7
- 238000005259 measurement Methods 0.000 claims description 5
- 230000004927 fusion Effects 0.000 claims description 4
- 230000007246 mechanism Effects 0.000 claims description 4
- 230000001360 synchronised effect Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
- G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0276—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle
Abstract
The invention discloses a remote driving system based on a 5G mobile communication technology and high-definition video transmission, which relates to the technical field of remote driving and comprises a hardware part and a software part; the hardware part comprises a driving control operation end and a remote vehicle execution end; the software part comprises communication, image processing and communication control between an operation end and an execution end; after the background control end and the vehicle execution end are respectively electrified, a remote networking connection is established between the background control end and the vehicle execution end through a 5G communication module, a special link is established on the basis of a public network for encrypted communication, and the functions of encrypted data transmission and image transmission based on the public network are realized; after the background control end receives the video information, automatically adjusting the monitoring information displayed on a background screen according to the gear state and the steering size of the vehicle so as to monitor the state of the remote vehicle execution end in real time; the background control end is also used for verifying the communication state of the monitoring sensor in real time, so that the control of the vehicle due to the influence of poor communication state is avoided, and the driving safety is improved.
Description
Technical Field
The invention relates to the technical field of remote driving, in particular to a remote driving system based on a 5G mobile communication technology and high-definition video transmission.
Background
In recent years, the automatic driving technology is rapidly developed, a traditional vehicle is transformed, core sensors such as a high-definition camera, a laser radar and a high-precision positioning device are added, data are collected in real time through the sensors, a high-precision map is matched, automatic driving in a simple environment is achieved, the automatic driving is limited by the power consumption and the volume of the vehicle, and automatic driving control based on a single independent vehicle is in bottleneck;
on the other hand, with the large-scale popularization and landing of 5G communication, network communication with high speed, low time delay and large connection becomes practical, further development of 5G vehicle networking, industrial internet and the like is supported, for an automatic driving scene, the 5G technology provides high-reliability low-time delay communication, decision can be completed at the cloud end, and is fed back to a vehicle rapidly, and meanwhile, network communication support is provided for vehicle-vehicle interconnection, so that cooperative driving becomes possible; under the circumstance, how to effectively utilize the 5G network, form remote driving capability at the cloud end, solve the problem of network delay, improve the interconnection communication among vehicles and realize higher-level automatic driving of the vehicles becomes a problem which needs to be solved urgently; therefore, a remote driving system based on a 5G mobile communication technology and high-definition video transmission is provided.
Disclosure of Invention
The present invention is directed to solving at least one of the problems of the prior art. Therefore, the remote driving system based on the 5G mobile communication technology and high-definition video transmission can break through the limitation of regions, realize remote driving in any region covering a 5G network and ensure low-delay reliable communication; meanwhile, the high-speed characteristic of 5G enables the user experience rate to reach 1Gbps, and low-delay high-definition video transmission can be realized.
To achieve the above object, an embodiment according to a first aspect of the present invention provides a remote driving system based on 5G mobile communication technology and high definition video transmission, including a hardware portion and a software portion;
the hardware part comprises a driving control operation end and a remote vehicle execution end; the software part comprises communication, image processing and communication control between an operation end and an execution end; the driving control operation end is a background control end controlled by an operator, namely a cabin; the remote vehicle execution end is a vehicle;
after the background control end and the vehicle execution end are respectively electrified, a remote networking connection is established between the background control end and the vehicle execution end through a 5G communication module, a special link is established on the basis of a public network, encryption communication is carried out, and functions of encrypted data transmission and image transmission based on the public network are realized;
after the background control end receives the video information, the fusion of the vehicle state and the video monitoring is realized through the processing of the industrial personal computer, and the monitoring information displayed on a background screen is automatically adjusted according to the vehicle gear state and the steering size, which is specifically represented as follows: when the vehicle is in straight line driving with the front gear engaged, a front view and an image inside the vehicle are called; when the vehicle is turned around by engaging a forward gear, forward-looking images and four-way around-looking images are called; when the vehicle is in reverse gear, calling back-view and four-way around-view images;
the background control end is also used for verifying the communication state of the monitoring sensor in real time, sending a first verification signal to the 5G communication module according to the verification period corresponding to the monitoring sensor, calculating to obtain a communication coefficient TX of the monitoring sensor, and then comparing the communication coefficient TX with a communication threshold value to obtain an evaluation signal; and in the driving process of the vehicle execution end, evaluating the communication deviation value TP according to the evaluation signal, and judging whether the communication state of the monitoring sensor is abnormal or not.
Further, a dedicated link is established on the basis of the public network to perform encrypted communication, and the specific steps are as follows: on the special transmission link, the vehicle execution end packs the data stream and the video stream into a data packet, the gateway equipment encrypts the data packet and converts the target address of the data packet, and after the data packet is transmitted to the background control end gateway equipment by virtue of a public network, the data packet is unpacked and decrypted into data and video information which are uploaded to the background control end by the vehicle execution end.
Further, the background control end is composed of a driving seat, a monitor, a gear position and gear position sensor, an accelerator pedal and pedal position sensor, a brake pedal and pedal position sensor, a steering wheel and angle sensor, a vehicle control unit VCU, a 5G communication module, a video decoding module and an industrial personal computer; the vehicle execution end comprises a whole vehicle, a VCU (vehicle control unit), an electronic throttle system, an electronic brake system, a steering auxiliary system, an angle sensor, a GPS (global position system) and IMU (inertial measurement unit) combined navigation system, a 5G communication module, a video coding module and an industrial personal computer.
Further, the working steps of the background control end comprise:
establishing TCP and UDP connection with a remote vehicle execution end to realize data transmission and image transmission functions between the TCP and the UDP;
after connection is established, the gear position, the accelerator pedal position, the brake pedal position and the steering angle position of a steering wheel are collected through a monitoring sensor, further converted into specific control quantity and uploaded to an industrial personal computer, and the industrial personal computer combines data transmission messages on the basis and sends the data transmission messages to a vehicle execution end in real time through a 5G communication module; the specific control quantity comprises a gear value, a throttle value, a brake value and a steering value;
after the connection is established, the data transmission message which is composed of the vehicle position information, the speed and the course angle and is recovered by the 5G communication module is processed by the industrial personal computer and then is drawn on a monitor picture of the background control end; and after the UDP data stream recovered by the 5G communication module is decoded by the video decoding module, the decoded UDP data stream is displayed on a background control end monitor picture.
Further, the monitoring sensors comprise a monitor, a gear position sensor, an accelerator pedal position sensor, a brake pedal position sensor and an angle sensor.
Furthermore, the background control end and the vehicle execution end are connected in a long way by using a TCP, and whether the background control end and the vehicle execution end are on line is detected by using a heartbeat mechanism.
Further, the working steps of the remote vehicle execution end comprise:
after the vehicle execution end is electrified, the connection with the background control end is established through the 5G communication module;
the 5G communication module receives an instruction issued by the background control end, the instruction is processed and analyzed by the industrial personal computer and issued to the VCU of the whole vehicle controller, and the gear, the accelerator, the brake and the steering of the vehicle are controlled by controlling and driving the direct current motor, the brake motor and the steering motor;
after the connection is established, the current state information of the vehicle is collected in real time and combined into a TCP message through a combined navigation system which is installed on the vehicle and combines with an IMU (global navigation satellite system) and the TCP message is sent to a background control end through a 5G communication module; the current state information comprises vehicle position information, speed and course angle;
the 4-path cameras arranged in the front, the rear, the left and the right of the vehicle are used for collecting surrounding video information, the surrounding video information is transmitted back to the background control end by means of the 5G communication module, and the video information is displayed on a monitor picture of the background control end.
Further, the specific verification step of the background control end is as follows:
the background control end sends a first verification signal to the monitoring sensor through the 5G communication module according to a verification period corresponding to the monitoring sensor, wherein the first verification signal carries a first signal quality threshold;
after the monitoring sensor receives the first verification signal, the monitoring sensor determines the current signal quality of the first verification signal and sends a second synchronization signal by taking the current signal quality of the first verification signal as a reference; the second synchronous signal is transmitted to the background control end through the 5G communication module;
after the background control end receives the second synchronous signal, the background control end determines the signal quality of the second synchronous signal, and compares the signal quality of the second synchronous signal with the first signal quality threshold to obtain a corresponding quality difference value Z1; calculating the time difference between the moment when the background control end sends the first verification signal and the moment when the background control end monitors the second synchronization signal again to obtain a response time XT;
calculating a communication coefficient TX by using a formula TX/(Z1 × a1+ XT × a2), wherein a1 and a2 are coefficient factors; comparing the communication coefficient TX with a communication threshold value to obtain an evaluation signal;
in the driving process of a vehicle execution end, the ratio of the times of the good signal, the normal signal and the range signal which are compared with the evaluation signal is counted, and the signals are sequentially marked as Zb1, Zb2 and Zb 3; calculating a communication bias value TP by using a formula TP (Zb2 × b2+ Zb3 × b3)/(Zb1 × b1+ u), wherein b1, b2 and b3 are coefficient factors, b3 > b2 > b1 and u is a compensation factor; if TP is larger than the bias value threshold value, judging that the communication problem of the current monitoring sensor is serious, and generating a communication abnormal signal;
the background control end is used for sending a communication abnormal signal to a mobile phone terminal of an operator, prompting the operator that the communication state between the current monitoring sensor and the background control end is not good, and suggesting timely processing; after detecting the communication abnormal signal, the vehicle execution section enters an active safety mode, emergently brakes, and keeps the position state of the vehicle execution section before the communication abnormal, namely, the vehicle execution section does not execute the corresponding command by the instruction of the abnormal communication signal; and after the vehicle execution end judges that the communication signal is normal, continuing the communication between the vehicle execution end and the vehicle execution end.
Further, the communication coefficient TX is compared with a communication threshold to obtain an evaluation signal, and the specific steps are as follows:
the communication threshold comprises Y1 and Y2, wherein Y1 and Y2 are preset values, and Y1 is less than Y2; if TX is more than or equal to Y2, the evaluation signal is a good signal; if Y1 is less than or equal to TX < Y2, the evaluation signal is a general signal;
if 0 < TX < Y1, the evaluation signal is a very poor signal.
Compared with the prior art, the invention has the beneficial effects that:
1. in the invention, after the background control end and the vehicle execution end are respectively electrified, the background control end and the vehicle execution end establish remote networking connection through the 5G communication module, and establish a special link on the basis of a public network to carry out encryption communication, thereby realizing the functions of encrypted data transmission and image transmission based on the public network; the remote control is realized based on the 5G communication technology, the influence of the distance on the remote control can be ignored, and the remote control spanning provinces, cities and regions can be realized as long as the remote terminal vehicle is in the region covering the mobile communication network; after the background control end receives the video information, the fusion of the vehicle state and the video monitoring is realized through the processing of a software part, and the monitoring information displayed on a background screen is automatically adjusted according to the vehicle gear state and the steering size, so that the background control end can monitor the state of the remote vehicle execution end in real time after the vehicle execution end is started, the data transmission delay can be controlled within 100ms, the image transmission delay is controlled within 250ms, and the remote control is more timely and reliable;
2. the background control end is also used for verifying the communication state of the monitoring sensor in real time, calculating to obtain a communication coefficient TX by combining the corresponding quality difference Z1 and the response duration XT, and comparing the communication coefficient TX with a communication threshold value to obtain an evaluation signal; in the driving process of a vehicle execution end, counting the ratio of times of good signals, general signals and range signals which are respectively compared with evaluation signals, calculating to obtain a communication deviation value TP, if TP is larger than a deviation value threshold value, judging that the communication problem of a current monitoring sensor is serious, generating a communication abnormal signal, prompting an operator that the communication state between the current monitoring sensor and a background control end is not good, suggesting timely processing, entering an active safety mode after a vehicle execution section detects the communication abnormal signal, emergently braking, keeping the position state of the vehicle before the communication is abnormal, and not executing a corresponding command by using an instruction of the abnormal communication signal; after the vehicle execution end judges that the communication signal is normal, the communication between the vehicle execution end and the vehicle execution end is continued; the control to the vehicle is avoided being influenced, the driving safety is improved, and the automatic driving function is combined, so that the automatic driving auxiliary device can be used as an auxiliary means of automatic driving.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a communication diagram of the present invention;
FIG. 2 is a diagram illustrating a hardware connection of a background control end according to the present invention;
FIG. 3 is a schematic diagram of the hardware connection of the vehicle actuator according to the present invention;
fig. 4 is a schematic flow diagram of video monitoring data according to the present invention.
Detailed Description
The technical solutions of the present invention will be described clearly and completely with reference to the following embodiments, and it should be understood that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1 to 4, a remote driving system based on 5G mobile communication technology and high definition video transmission comprises a hardware part and a software part;
the hardware part mainly comprises a driving control operation end and a remote vehicle execution end; the software part comprises communication, image processing and communication control between an operation end and an execution end;
the driving control operation end is a background control end controlled by an operator, namely a cabin; the remote vehicle execution end is a vehicle; the background control end consists of a driving seat, a monitor, a gear position and gear position sensor, an accelerator pedal and pedal position sensor, a brake pedal and pedal position sensor, a steering wheel and angle sensor, a vehicle control unit VCU, a 5G communication module, a video decoding module and an industrial personal computer; the vehicle execution end consists of a whole vehicle, a VCU (vehicle control unit), an electronic accelerator system, an electronic brake system, a steering auxiliary system, an angle sensor, a GPS (global positioning system) and IMU (inertial measurement unit) combined navigation, a 5G communication module, a video coding module and an industrial personal computer;
after the background control end and the vehicle execution end are respectively electrified, the background control end and the vehicle execution end establish remote networking connection through a 5G communication module, and then the functions of data transmission and image transmission can be realized; when a backstage control end operator operates a stop lever, an accelerator, a brake and a steering wheel of a cabin, a gear position sensor, an accelerator pedal position sensor, a brake pedal position sensor and an angle sensor sense operation signals and convert the operation signals into specific control quantities, the specific control quantities are uploaded to an industrial personal computer in real time, the industrial personal computer integrates the received control quantities into control messages through a software part and sends the control messages to a remote vehicle execution end through a 5G communication module, the industrial personal computer on the vehicle execution end analyzes command contents and sends the command contents to a vehicle control unit VCU, and the vehicle control unit VCU is used for respectively controlling and driving a direct current motor, a brake motor and a steering motor according to the received command contents, so that gear, accelerator, brake and steering control of a vehicle are realized;
after the vehicle execution end is electrified, the position information and the state of the vehicle are sent to the upper computer software of the industrial personal computer in real time through a combined navigation system formed by combining GNSS (global navigation satellite system) and IMU (inertial measurement unit) installed on the vehicle and are returned to the background control end through the 5G communication module, the background control end draws the vehicle information on a background end screen after processing the received information by a software part, so that the background control end can monitor the state of the remote vehicle execution end in real time after the vehicle execution end is started; wherein the background control end is an operator;
it should be further noted that the background control end and the vehicle execution end are connected with each other by using a long TCP, and whether the background control end and the vehicle execution end are online is detected by using a heartbeat mechanism, so that the connection between the background control end and the vehicle execution end is ensured to exist all the time when an accident occurs; the operation saves more TCP establishing and closing operations, reduces the influence of network congestion and ensures the connection reliability;
realizing remote networking between a background control end and a vehicle execution end, and establishing a special link on the basis of a public network to carry out encrypted communication; on the special transmission link, the vehicle execution end packs the data stream and the video stream into a data packet, the gateway equipment encrypts the data packet and converts a target address of the data packet, the data packet is unpacked and decrypted into data and video information which are uploaded to the background control end by the vehicle execution end after the data packet is transmitted to the background control end gateway equipment by means of a public network, and therefore functions of encrypted data transmission and image transmission based on the public network are achieved.
After the background control end receives the video information, the fusion of the vehicle state and the video monitoring is realized through the processing of the software part, the monitoring information displayed on a background screen is automatically adjusted according to the vehicle gear state and the steering size, and the specific expression is as follows: when the vehicle is in straight line driving with the front gear engaged, a front view and an image inside the vehicle are called; when the vehicle is turned around by engaging a forward gear, forward-looking images and four-way around-looking images are called; when the vehicle is in reverse gear, calling back-view and four-way around-view images;
in this embodiment, the background control end is further configured to verify a communication state of the monitoring sensor in real time, the monitoring sensor includes a monitor, a gear position sensor, an accelerator pedal position sensor, a brake pedal position sensor, and an angle sensor, and the specific verification steps include:
the background control end sends a first verification signal to the 5G communication module according to a verification period corresponding to the monitoring sensor, wherein the first verification signal carries a first signal quality threshold; after the 5G communication module receives the first verification signal, the first verification signal is immediately sent to the monitoring sensor;
after the monitoring sensor receives the first verification signal, the monitoring sensor determines the current signal quality of the first verification signal and sends a second synchronization signal by taking the current signal quality of the first verification signal as a reference; the second synchronous signal is transmitted to the background control end through the 5G communication module;
after receiving the second synchronization signal, the background control end determines the signal quality of the second synchronization signal, and compares the signal quality of the second synchronization signal with the first signal quality threshold to obtain a corresponding quality difference value Z1, where it should be understood by those skilled in the art that any metric known in the art can be used to characterize the signal quality, such as RSRQ, RSRP, RSSI, etc.; the quality difference can reflect the attenuation of the signal in the transmission process;
calculating the time difference between the moment when the background control end sends the first verification signal and the moment when the background control end monitors the second synchronization signal again to obtain a response time XT; calculating a communication coefficient TX by using a formula TX/(Z1 × a1+ XT × a2), wherein a1 and a2 are coefficient factors;
comparing the communication coefficient TX with a communication threshold value to obtain an evaluation signal, wherein the communication threshold value comprises Y1 and Y2, Y1 and Y2 are preset values, and Y1 is less than Y2; the method specifically comprises the following steps:
if TX is more than or equal to Y2, the evaluation signal is a good signal;
if Y1 is less than or equal to TX < Y2, the evaluation signal is a general signal;
if 0 < TX < Y1, the evaluation signal is a range signal;
in the driving process of a vehicle execution end, the ratio of the times of the good signal, the normal signal and the range signal which are compared with the evaluation signal is counted, and the signals are sequentially marked as Zb1, Zb2 and Zb 3;
calculating a communication bias value TP by using a formula TP (Zb2 × b2+ Zb3 × b3)/(Zb1 × b1+ u), wherein b1, b2 and b3 are coefficient factors, b3 > b2 > b1 and u is a compensation factor;
comparing the communication offset value TP with an offset threshold value, if TP is larger than the offset threshold value, judging that the communication problem of the current monitoring sensor is serious, and generating a communication abnormal signal; the background control end is used for sending a communication abnormal signal to a mobile phone terminal of an operator, prompting the operator that the communication state between the current monitoring sensor and the background control end is not good, suggesting timely processing, avoiding influencing the control of the vehicle and improving the driving safety;
after detecting the communication abnormal signal, the vehicle execution section enters an active safety mode, emergently brakes, and keeps the position state of the vehicle execution section before the communication abnormal, namely, the vehicle execution section does not execute the corresponding command by the instruction of the abnormal communication signal; and after the vehicle execution end judges that the communication signal is normal, continuing the communication between the vehicle execution end and the vehicle execution end.
The above formulas are all calculated by removing dimensions and taking numerical values thereof, the formula is a formula which is obtained by acquiring a large amount of data and performing software simulation to obtain the closest real situation, and the preset parameters and the preset threshold value in the formula are set by the technical personnel in the field according to the actual situation or obtained by simulating a large amount of data.
The working principle of the invention is as follows:
when the remote driving system works, when a backstage control end operator operates a stop lever, an accelerator, a brake and a steering wheel of a cabin, a gear position sensor, an accelerator pedal position sensor, a brake pedal position sensor and an angle sensor sense operation signals and convert the operation signals into specific control quantities to be uploaded to an industrial personal computer in real time, and a VCU (vehicle control unit) is used for respectively controlling a driving direct current motor, a brake motor and a steering motor according to received command contents to realize gear, accelerator, brake and steering control of a vehicle; after the vehicle execution end is electrified, the position information and the state of the vehicle are sent to the upper computer software of the industrial personal computer in real time through a combined navigation system formed by combining a GNSS (global navigation satellite system) and an IMU (inertial measurement unit) which are installed on the vehicle, and the vehicle information is drawn on a background end screen; meanwhile, a heartbeat mechanism is utilized to detect whether the background control end and the vehicle execution end are on line or not, so that the connection between the background control end and the vehicle execution end is ensured to exist all the time when an accident happens;
the background control end is also used for verifying the communication state of the monitoring sensor in real time, and the background control end sends a first verification signal to the monitoring sensor through the 5G communication module according to the verification period corresponding to the monitoring sensor; after the monitoring sensor receives the first verification signal, the monitoring sensor determines the current signal quality of the first verification signal, sends a second synchronization signal by taking the current signal quality of the first verification signal as a reference, and transmits the second synchronization signal to the background control end through the 5G communication module; calculating to obtain a communication coefficient TX by combining the corresponding quality difference Z1 and the response duration XT, and comparing the communication coefficient TX with a communication threshold value to obtain an evaluation signal; in the driving process of a vehicle execution end, the ratio of times of good signals, general signals and range signals which are compared with the evaluation signals is counted, a communication deviation value TP is obtained through calculation, if TP is larger than a deviation value threshold value, the communication problem of the current monitoring sensor is judged to be serious, a communication abnormal signal is generated, an operator is prompted that the communication state between the current monitoring sensor and a background control end is not good, timely processing is recommended, the control of the vehicle is prevented from being influenced, and the driving safety is improved.
In the description herein, references to the description of "one embodiment," "an example," "a specific example" or the like are intended to mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The preferred embodiments of the invention disclosed above are intended to be illustrative only. The preferred embodiments are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the invention and the practical application, to thereby enable others skilled in the art to best utilize the invention. The invention is limited only by the claims and their full scope and equivalents.
Claims (8)
1. A remote driving system based on 5G mobile communication technology and high-definition video transmission is characterized by comprising a hardware part and a software part;
the hardware part comprises a driving control operation end and a remote vehicle execution end; the software part comprises communication, image processing and communication control between an operation end and an execution end; the driving control operation end is a background control end controlled by an operator, namely a cabin; the remote vehicle execution end is a vehicle;
after the background control end and the vehicle execution end are respectively electrified, a remote networking connection is established between the background control end and the vehicle execution end through a 5G communication module, a special link is established on the basis of a public network, encryption communication is carried out, and functions of encrypted data transmission and image transmission based on the public network are realized;
after the background control end receives the video information, the fusion of the vehicle state and the video monitoring is realized through the processing of the industrial personal computer, and the monitoring information displayed on a background screen is automatically adjusted according to the vehicle gear state and the steering size, which is specifically represented as follows: when the vehicle is in straight line driving with the front gear engaged, a front view and an image inside the vehicle are called; when the vehicle is turned around by engaging a forward gear, forward-looking images and four-way around-looking images are called; when the vehicle is in reverse gear, calling back-view and four-way around-view images;
the background control end is also used for verifying the communication state of the monitoring sensor in real time, sending a first verification signal to the 5G communication module according to the verification period corresponding to the monitoring sensor, calculating to obtain a communication coefficient TX of the monitoring sensor, and then comparing the communication coefficient TX with a communication threshold value to obtain an evaluation signal; and in the driving process of the vehicle execution end, evaluating the communication deviation value TP according to the evaluation signal, and judging whether the communication state of the monitoring sensor is abnormal or not.
2. The remote driving system based on 5G mobile communication technology and high definition video transmission as claimed in claim 1, wherein a dedicated link is established on the basis of public network to perform encrypted communication, and the specific steps are as follows: on the special transmission link, the vehicle execution end packs the data stream and the video stream into a data packet, the gateway equipment encrypts the data packet and converts the target address of the data packet, and after the data packet is transmitted to the background control end gateway equipment by virtue of a public network, the data packet is unpacked and decrypted into data and video information which are uploaded to the background control end by the vehicle execution end.
3. The remote driving system based on the 5G mobile communication technology and the high-definition video transmission is characterized in that the background control end consists of a driving seat, a monitor, a gear position and gear position sensor, an accelerator pedal and pedal position sensor, a brake pedal and pedal position sensor, a steering wheel and angle sensor, a vehicle control unit VCU, a 5G communication module, a video decoding module and an industrial personal computer; the vehicle execution end comprises a whole vehicle, a VCU (vehicle control unit), an electronic throttle system, an electronic brake system, a steering auxiliary system, an angle sensor, a GPS (global position system) and IMU (inertial measurement unit) combined navigation system, a 5G communication module, a video coding module and an industrial personal computer.
4. The remote driving system based on 5G mobile communication technology and high definition video transmission as claimed in claim 3, wherein the working step of the background control end comprises:
establishing TCP and UDP connection with a remote vehicle execution end to realize data transmission and image transmission functions between the TCP and the UDP;
after connection is established, the gear position, the accelerator pedal position, the brake pedal position and the steering angle position of a steering wheel are collected through a monitoring sensor, further converted into specific control quantity and uploaded to an industrial personal computer, and the industrial personal computer combines data transmission messages on the basis and sends the data transmission messages to a vehicle execution end in real time through a 5G communication module; the specific control quantity comprises a gear value, a throttle value, a brake value and a steering value;
after the connection is established, the data transmission message which is composed of the vehicle position information, the speed and the course angle and is recovered by the 5G communication module is processed by the industrial personal computer and then is drawn on a monitor picture of the background control end; and after the UDP data stream recovered by the 5G communication module is decoded by the video decoding module, the decoded UDP data stream is displayed on a background control end monitor picture.
5. The remote driving system based on 5G mobile communication technology and high definition video transmission as claimed in claim 4, wherein the monitoring sensors comprise a monitor, a gear position sensor, an accelerator pedal position sensor, a brake pedal position sensor and an angle sensor.
6. The remote driving system based on the 5G mobile communication technology and the high-definition video transmission as claimed in claim 1, wherein a long TCP connection is used between the background control end and the vehicle execution end, and a heartbeat mechanism is utilized to detect whether the background control end and the vehicle execution end are online.
7. The remote driving system based on 5G mobile communication technology and high-definition video transmission as claimed in claim 1, wherein the working step of the remote vehicle execution end comprises:
after the vehicle execution end is electrified, the connection with the background control end is established through the 5G communication module;
the 5G communication module receives an instruction issued by the background control end, the instruction is processed and analyzed by the industrial personal computer and issued to the VCU of the whole vehicle controller, and the gear, the accelerator, the brake and the steering of the vehicle are controlled by controlling and driving the direct current motor, the brake motor and the steering motor;
after the connection is established, the current state information of the vehicle is collected in real time and combined into a TCP message through a combined navigation system which is installed on the vehicle and combines with an IMU (global navigation satellite system) and the TCP message is sent to a background control end through a 5G communication module; the current state information comprises vehicle position information, speed and course angle;
the 4-path cameras arranged in the front, the rear, the left and the right of the vehicle are used for collecting surrounding video information, the surrounding video information is transmitted back to the background control end by means of the 5G communication module, and the video information is displayed on a monitor picture of the background control end.
8. The remote driving system based on the 5G mobile communication technology and the high-definition video transmission as claimed in claim 1, wherein the specific verification step of the background control end is as follows:
comparing the communication coefficient TX with communication thresholds comprising Y1, Y2, wherein Y1, Y2 are both preset values and Y1 < Y2; if TX is more than or equal to Y2, the evaluation signal is a good signal; if Y1 is less than or equal to TX < Y2, the evaluation signal is a general signal; if 0 < TX < Y1, the evaluation signal is a range signal; in the driving process of a vehicle execution end, the ratio of the times of the good signal, the normal signal and the range signal which are compared with the evaluation signal is counted, and the signals are sequentially marked as Zb1, Zb2 and Zb 3;
calculating a communication bias value TP by using a formula TP (Zb2 × b2+ Zb3 × b3)/(Zb1 × b1+ u), wherein b1, b2 and b3 are coefficient factors, b3 > b2 > b1 and u is a compensation factor; if TP is larger than the bias value threshold value, judging that the communication problem of the current monitoring sensor is serious, and generating a communication abnormal signal;
the background control end is used for sending a communication abnormal signal to a mobile phone terminal of an operator, prompting the operator that the communication state between the current monitoring sensor and the background control end is not good, and suggesting timely processing;
after detecting the communication abnormal signal, the vehicle execution section enters an active safety mode, emergently brakes, and keeps the position state of the vehicle execution section before the communication abnormal, namely, the vehicle execution section does not execute the corresponding command by the instruction of the abnormal communication signal; and after the vehicle execution end judges that the communication signal is normal, continuing the communication between the vehicle execution end and the vehicle execution end.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210005325.3A CN114355943A (en) | 2022-01-05 | 2022-01-05 | Remote driving system based on 5G mobile communication technology and high-definition video transmission |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210005325.3A CN114355943A (en) | 2022-01-05 | 2022-01-05 | Remote driving system based on 5G mobile communication technology and high-definition video transmission |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114355943A true CN114355943A (en) | 2022-04-15 |
Family
ID=81106873
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210005325.3A Pending CN114355943A (en) | 2022-01-05 | 2022-01-05 | Remote driving system based on 5G mobile communication technology and high-definition video transmission |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114355943A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115277788A (en) * | 2022-08-23 | 2022-11-01 | 石家庄开发区天远科技有限公司 | Engineering truck remote control system and method |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7499545B1 (en) * | 2001-02-05 | 2009-03-03 | Ati Technologies, Inc. | Method and system for dual link communications encryption |
| CN102346473A (en) * | 2011-09-05 | 2012-02-08 | 北京航空航天大学 | Remote vehicle driving control device |
| CN110377000A (en) * | 2018-04-13 | 2019-10-25 | 北京智行者科技有限公司 | A kind of long-range dispensing control system of automatic driving vehicle and method |
| CN112286166A (en) * | 2020-10-12 | 2021-01-29 | 上海交通大学 | Vehicle remote driving control system and method based on 5G network |
| CN112492094A (en) * | 2020-12-01 | 2021-03-12 | 宁波波导易联电子有限公司 | Remote driving mobile terminal, remote driving system and remote driving method |
| CN112863005A (en) * | 2021-02-02 | 2021-05-28 | 江苏奥力威传感高科股份有限公司 | Electric automobile intelligent monitoring system and method based on Internet of vehicles |
| CN113030885A (en) * | 2021-03-02 | 2021-06-25 | 安特微智能通讯（深圳）有限公司 | Millimeter wave radar detection system |
-
2022
- 2022-01-05 CN CN202210005325.3A patent/CN114355943A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7499545B1 (en) * | 2001-02-05 | 2009-03-03 | Ati Technologies, Inc. | Method and system for dual link communications encryption |
| CN102346473A (en) * | 2011-09-05 | 2012-02-08 | 北京航空航天大学 | Remote vehicle driving control device |
| CN110377000A (en) * | 2018-04-13 | 2019-10-25 | 北京智行者科技有限公司 | A kind of long-range dispensing control system of automatic driving vehicle and method |
| CN112286166A (en) * | 2020-10-12 | 2021-01-29 | 上海交通大学 | Vehicle remote driving control system and method based on 5G network |
| CN112492094A (en) * | 2020-12-01 | 2021-03-12 | 宁波波导易联电子有限公司 | Remote driving mobile terminal, remote driving system and remote driving method |
| CN112863005A (en) * | 2021-02-02 | 2021-05-28 | 江苏奥力威传感高科股份有限公司 | Electric automobile intelligent monitoring system and method based on Internet of vehicles |
| CN113030885A (en) * | 2021-03-02 | 2021-06-25 | 安特微智能通讯（深圳）有限公司 | Millimeter wave radar detection system |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115277788A (en) * | 2022-08-23 | 2022-11-01 | 石家庄开发区天远科技有限公司 | Engineering truck remote control system and method |
| CN115277788B (en) * | 2022-08-23 | 2024-04-26 | 石家庄开发区天远科技有限公司 | Engineering vehicle remote control system and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3548352B1 (en) | Applications for using mass estimations for vehicles | |
| JP7270685B2 (en) | Remote control method, remote control device, remote control system, electronic device, storage medium, cockpit, cloud server, self-driving vehicle, and program | |
| CN109017757B (en) | Automobile remote designated driving method and system | |
| US11242066B2 (en) | Vehicle control apparatus and vehicle control system and vehicle control method thereof | |
| CN107871399B (en) | Automatic vehicle driving system and method | |
| CN111866809A (en) | Unmanned mine card remote control driving system and method | |
| KR20230002176A (en) | Roadside sensing system, traffic control method, traffic control systems, and computer programs | |
| CN110868331A (en) | Information processing method, information processing apparatus, and information processing system | |
| CN114355943A (en) | Remote driving system based on 5G mobile communication technology and high-definition video transmission | |
| KR102478809B1 (en) | Method for invoking a teleoperated driving session, apparatus for performing the steps of the method, vehicle and computer program | |
| CN110891160A (en) | Image transmission device, image transmission method, and recording medium | |
| US20070115109A1 (en) | Security system and method for mass transit vehicles | |
| US20220229446A1 (en) | Applications for using mass estimations for vehicles | |
| WO2024022003A1 (en) | Vehicle avoidance method and apparatus, terminal device and storage medium | |
| CN110460677A (en) | A kind of excavator and excavator tele-control system | |
| CN113022445A (en) | Vehicle environment information monitoring method and system | |
| Yu et al. | Remote driving control with real-time video streaming over wireless networks: Design and evaluation | |
| CN113873199A (en) | Vehicle-mounted camera sharing system and method based on direct connection communication and cellular network communication | |
| EP3702864B1 (en) | Accounting for latency in teleoperated remote driving | |
| CN208384420U (en) | A kind of unmanned emergent treatment system in expressway | |
| CN110843797A (en) | Vehicle, vehicle equipment and screen display prompting method for vehicle running state | |
| CN114339059A (en) | Method and device for adjusting remote driving picture | |
| KR101728903B1 (en) | Apparatus and method for steering control of vehicles for following target parking path | |
| CN117037517A (en) | Vehicle control method, device, electronic equipment and storage medium | |
| CN220639810U (en) | Parallel driving transportation system suitable for wisdom harbour |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220415 |
|
| RJ01 | Rejection of invention patent application after publication |