CN113260938A - Remote access system and method for unmanned vehicle, electronic device, and storage medium - Google Patents
Remote access system and method for unmanned vehicle, electronic device, and storage medium Download PDFInfo
- Publication number
- CN113260938A CN113260938A CN202080007604.2A CN202080007604A CN113260938A CN 113260938 A CN113260938 A CN 113260938A CN 202080007604 A CN202080007604 A CN 202080007604A CN 113260938 A CN113260938 A CN 113260938A
- Authority
- CN
- China
- Prior art keywords
- remote control
- control instruction
- state information
- middleware
- remote
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 35
- 238000004590 computer program Methods 0.000 claims description 9
- 238000010586 diagram Methods 0.000 description 10
- 238000011161 development Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 4
- 238000012423 maintenance Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 230000001360 synchronised effect Effects 0.000 description 2
- ZVKAMDSUUSMZES-NZQWGLPYSA-N OS II Natural products CC(=O)N[C@H]1[C@H](OC[C@@H](O)[C@@H](O)[C@@H](O)CO)O[C@H](CO)[C@H](O[C@H]2O[C@H](CO)[C@@H](O)[C@H](O)[C@H]2O)[C@@H]1O[C@@H]3O[C@H](CO)[C@@H](O)[C@H](O)[C@H]3O ZVKAMDSUUSMZES-NZQWGLPYSA-N 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 230000010399 physical interaction Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
- G05D1/0214—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory in accordance with safety or protection criteria, e.g. avoiding hazardous areas
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B19/00—Programme-control systems
- G05B19/02—Programme-control systems electric
- G05B19/04—Programme control other than numerical control, i.e. in sequence controllers or logic controllers
- G05B19/042—Programme control other than numerical control, i.e. in sequence controllers or logic controllers using digital processors
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0276—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
- H04L67/125—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks involving control of end-device applications over a network
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- General Physics & Mathematics (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- Aviation & Aerospace Engineering (AREA)
- General Health & Medical Sciences (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Medical Informatics (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Telephonic Communication Services (AREA)
- Selective Calling Equipment (AREA)
Abstract
A remote takeover system (200) and method, electronic device, storage medium for an unmanned vehicle, comprising: the system comprises a network middleware (210) for receiving a remote control instruction sent by a remote control end and sending the remote control instruction to a remote calling module (230) through a universal network interface; the remote calling module (230) is used for receiving a remote control instruction sent by the network middleware (210) and calling the vehicle driving module (250) according to the remote control instruction; a vehicle driving module (250) for invoking a system middleware (270) according to the remote control instruction; the system middleware (270) is used for calling an interface of the embedded operating system (290) according to the remote control instruction; and the embedded operating system (290) is used for sending the remote control instruction to the hardware equipment through an interface of the embedded operating system (290) so that the hardware equipment executes the remote control instruction to remotely take over the unmanned automobile. The real-time performance and the stability of the remote take-over system (200) of the unmanned automobile are improved, the cost and the power consumption are reduced, and the efficiency is improved.
Description
Technical Field
The present disclosure relates to the field of unmanned technologies, and in particular, to a remote access system and method for an unmanned vehicle, an electronic device, and a storage medium.
Background
With the continuous development of science and technology, the unmanned technology gradually comes into the visual field of people. The unmanned technology is that the vehicle-mounted sensor is used for sensing the surrounding environment of the vehicle, and the steering and the speed of the vehicle are controlled according to the road, the vehicle position and the obstacle information obtained by sensing, so that the vehicle can safely and reliably run on the road. However, when the unmanned vehicle encounters some emergency or system problems, it is very important to take over the unmanned vehicle remotely in time, and the safety of the unmanned vehicle and the safety of related personnel can be ensured. However, the conventional remote takeover method for the unmanned vehicle is poor in instantaneity and stability, and cannot be used for remotely taking over the unmanned vehicle in time.
Disclosure of Invention
Therefore, it is necessary to provide a remote takeover system, a method, an electronic device, and a storage medium for an unmanned vehicle, which can improve the real-time performance and stability of the remote takeover system of the unmanned vehicle, reduce the cost and power consumption, and further improve the efficiency.
A remote takeover system for an unmanned vehicle, comprising:
the network middleware is used for receiving a remote control instruction sent by a remote control end and sending the remote control instruction to the remote calling module through a universal network interface;
the remote calling module is used for receiving a remote control instruction sent by the network middleware and calling the vehicle driving module according to the remote control instruction;
the vehicle driving module is used for calling system middleware according to the remote control instruction;
the system middleware is used for calling an interface of the embedded operating system according to the remote control instruction;
and the embedded operating system is used for sending the remote control instruction to hardware equipment through an interface of the embedded operating system so that the hardware equipment executes the remote control instruction to remotely take over the unmanned automobile.
In one embodiment, the method further comprises the following steps:
and the state reporting module is used for receiving the automobile state information sent by the automobile driving module and the system state information sent by the system middleware, and reporting the automobile state information and the system state information to the remote control terminal through the network middleware so that the remote control terminal sends a remote control instruction according to the automobile state information and the system state information.
In one embodiment, the network middleware is further configured to provide the universal network interface based on the embedded operating system and a network protocol stack, so that the state reporting module reports the vehicle state information and the system state information to the remote control terminal through the universal network interface.
In one embodiment, the embedded operating system is further configured to obtain an execution result of the control instruction from the hardware device, call an interface of the system middleware, and feed back the execution result to a vehicle driving module;
the vehicle driving module is further used for receiving the execution result and feeding the execution result back to the network middleware through the remote calling module;
the network middleware is further configured to receive the execution result and feed back the execution result to the remote control end.
In one embodiment, the remote control end comprises a control center and a monitoring center, wherein the control center is used for sending a remote control instruction according to the automobile state information and the system state information and receiving an execution result of the remote control instruction fed back by the network middleware;
the monitoring center is used for receiving the automobile state information and the system state information reported by the network middleware and sending the automobile state information and the system state information to the control center.
In one embodiment, the network protocol stack comprises an embedded TCP/IP network protocol stack.
In one embodiment, the embedded operating system comprises embedded Linux, FreeRTOS, RT-Thread.
A remote takeover method of an unmanned automobile is applied to a remote takeover system of the unmanned automobile, the system comprises a network middleware, a remote calling module, a vehicle driving module, a system middleware and an embedded operating system, and the method comprises the following steps:
receiving a remote control instruction sent by a remote control end through a network middleware, and sending the remote control instruction to a remote calling module through a universal network interface;
receiving a remote control instruction sent by the network middleware through a remote calling module, and calling a vehicle driving module according to the remote control instruction;
calling system middleware through a vehicle driving module according to the remote control instruction;
calling an interface of the embedded operating system through the system middleware according to the remote control instruction;
and sending the remote control instruction to hardware equipment through an interface of the embedded operating system so that the hardware equipment executes the remote control instruction to remotely take over the unmanned automobile.
In one embodiment, the system further includes a status reporting module, and the method further includes:
and a state reporting module is used for receiving automobile state information sent by the automobile driving module and system state information sent by the system middleware, and reporting the automobile state information and the system state information to the remote control end through the network middleware, so that the remote control end sends a remote control instruction according to the automobile state information and the system state information.
In one embodiment, the method further comprises:
and providing the universal network interface through a network middleware based on the embedded operating system and a network protocol stack, so that the state reporting module reports the automobile state information and the system state information to the remote control terminal through the universal network interface.
In one embodiment, the method further comprises:
acquiring an execution result of the control instruction from the hardware equipment through the embedded operating system, calling an interface of the system middleware, and feeding the execution result back to a vehicle driving module;
receiving the execution result through the vehicle driving module, and feeding back the execution result to the network middleware through the remote calling module;
and receiving the execution result through the network middleware, and feeding back the execution result to the remote control end.
In one embodiment, the remote control end includes a control center and a monitoring center, and the method further includes:
sending a remote control instruction and receiving an execution result of the remote control instruction fed back by the network middleware through the control center according to the automobile state information and the system state information;
and receiving the automobile state information and the system state information reported by the network middleware through the monitoring center, and sending the automobile state information and the system state information to the control center.
An electronic device comprising a memory and a processor, the memory having stored therein a computer program which, when executed by the processor, causes the processor to carry out the steps of the above method.
A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method as above.
The system comprises a network middleware, a remote control module and a remote calling module, wherein the network middleware is used for receiving a remote control instruction sent by a remote control end and sending the remote control instruction to the remote calling module through a universal network interface; the remote calling module is used for receiving a remote control instruction sent by the network middleware and calling the vehicle driving module according to the remote control instruction; the vehicle driving module is used for calling the system middleware according to the remote control instruction; the system middleware is used for calling an interface of the embedded operating system according to the remote control instruction; and the embedded operating system is used for sending the remote control instruction to the hardware equipment through an interface of the embedded operating system so that the hardware equipment executes the remote control instruction to remotely take over the unmanned automobile. The remote takeover system based on the embedded operating system has the characteristics of low development cost, low maintenance cost, high response speed, high stability and low power consumption of the embedded operating system. The system can adapt to the diversity of bottom layer hardware and bottom layer network equipment by setting the network middleware, can adapt to objectified and abstracted vehicle driving modules by setting the remote calling module, and can package the embedded operating system to provide a general interface for upper layer access by setting the system middleware. Therefore, the real-time performance and the stability of the remote pipe connecting system of the unmanned automobile are improved, the cost and the power consumption are reduced, and the efficiency is improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic diagram of an application environment of a remote takeover system of an unmanned vehicle according to an embodiment;
FIG. 2 is an architecture diagram of a remote takeover system for an unmanned vehicle, in accordance with one embodiment;
FIG. 3 is an architecture diagram of a remote takeover system for an unmanned vehicle in accordance with another embodiment;
FIG. 4 is an architecture diagram of a remote takeover system for an unmanned vehicle in accordance with one particular embodiment;
FIG. 5 is a flow diagram of a remote takeover method for an unmanned vehicle in accordance with one embodiment;
FIG. 6 is a flow diagram of a remote takeover method for an unmanned vehicle in accordance with one embodiment;
fig. 7 is a schematic diagram of an internal structure of an electronic device in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
It will be understood that, as used herein, the terms "first," "second," and the like may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish one element from another.
Fig. 1 is a schematic diagram of an application environment of a remote takeover system of an unmanned vehicle. As shown in fig. 1, the application environment includes a car 120 and a server 140, and the car 120 and the server 140 are connected via a network. The remote take-over system of the unmanned automobile runs on the vehicle-mounted control equipment or the vehicle-mounted computer 122 of the automobile 120, so that a remote control instruction sent by a remote control end can be received through the network middleware, and the remote control instruction is sent to the remote calling module through a universal network interface; receiving a remote control instruction sent by the network middleware through a remote calling module, and calling a vehicle driving module according to the remote control instruction; calling system middleware through a vehicle driving module according to a remote control instruction; calling an interface of the embedded operating system through the system middleware according to the remote control instruction; and sending the remote control instruction to the hardware equipment through an interface of the embedded operating system so that the hardware equipment executes the remote control instruction to remotely take over the unmanned automobile. The vehicle 120 may be an unmanned vehicle or a manned vehicle, and the vehicle may be a warehouse forklift, an excavator, or the like. The remote takeover system is suitable for the situation that an unmanned automobile needs to start remote takeover under certain special situations, and can also be the situation that a manned automobile needs to start remote takeover under certain special situations.
Fig. 2 is an architecture diagram of a remote takeover system of an unmanned aerial vehicle in an embodiment, as shown in fig. 2, a remote takeover system 200 of an unmanned aerial vehicle includes a network middleware 210, a remote invocation module 230, a vehicle driving module 250, a system middleware 270, and an embedded operating system 290.
The network middleware 210 is configured to receive a remote control instruction sent by a remote control end, and send the remote control instruction to the remote call module through a general network interface. The universal network middleware based on the embedded operating system and the vehicle hardware equipment control has the characteristics of strong universality, small size, high real-time property, high throughput and low power consumption, and can be operated on real-time embedded operating systems such as embedded Linux, FreeRTOS, RT-Thread and the like. The remote control instruction may be a remote control instruction directly calculated by the server, or may be a remote control instruction sent by a security guard operating the unmanned vehicle, and the remote control instruction is sent to the unmanned vehicle by the server.
And the remote calling module 230 is configured to receive a remote control instruction sent by the network middleware and call the vehicle driving module according to the remote control instruction. The remote calling module based on the vehicle driving module and the network middleware has the characteristics of standardization and protocol, is the topmost layer of an embedded software system architecture, and the cloud end realizes remote take-over through an interface of the remote calling module.
And the vehicle driving module 250 is used for calling the system middleware according to the remote control instruction. The vehicle driving module realized based on the system middleware has the characteristics of packaging property, low coupling and modularization.
And the system middleware 270 is used for calling an interface of the embedded operating system according to the remote control instruction. The universal system middleware based on the real-time embedded operating system and the vehicle hardware equipment control has the characteristics of strong universality, small size, tailorability, high real-time performance and low power consumption, and can be operated on real-time embedded operating systems such as embedded Linux, FreeRTOS, RT-Thread and the like.
And the embedded operating system 290 is configured to send the remote control instruction to the hardware device through an interface of the embedded operating system, so that the hardware device executes the remote control instruction to remotely take over the unmanned vehicle. The real-time operating system running on the embedded hardware equipment has the characteristics of low power consumption, real-time performance and high customization, and comprises but is not limited to real-time embedded operating systems such as embedded Linux, FreeRTOS, RT-Thread and the like.
Specifically, the remote control terminal may be a server, and when the unmanned vehicle initiates a request for remote takeover to the remote control terminal, the remote control terminal starts to take over the unmanned vehicle remotely. The remote control instruction comprises a control instruction customized according to the current automobile state information of the unmanned automobile, such as a brake instruction, a parking instruction, a steering wheel control instruction, an accelerator instruction and the like. This is, of course, not a limitation of the present application. When the remote control end remotely takes over the unmanned automobile, firstly, the remote control end sends a remote control instruction to the network middleware; then, receiving a remote control instruction sent by a remote control end through a network middleware, and sending the remote control instruction to a remote calling module through a universal network interface; secondly, receiving a remote control instruction sent by the network middleware through a remote calling module, and calling a vehicle driving module according to the remote control instruction; calling system middleware through a vehicle driving module according to a remote control instruction; thirdly, calling an interface of the embedded operating system through the system middleware according to the remote control instruction; and finally, sending the remote control instruction to hardware equipment of the unmanned automobile through an interface of the embedded operating system so that the hardware equipment executes the remote control instruction to take over the unmanned automobile remotely. The remote control system can ensure that the unmanned automobile can be remotely taken over by the remote control end in time under special conditions so as to ensure that the automobile safely runs until the automobile is safely stopped.
In the embodiment of the application, the remote takeover system based on the embedded operating system has the characteristics of low development cost, low maintenance cost, high response speed, high stability and low power consumption of the embedded operating system. The system can adapt to the diversity of bottom layer hardware and bottom layer network equipment by setting the network middleware, can adapt to objectified and abstracted vehicle driving modules by setting the remote calling module, and can package the embedded operating system to provide a general interface for upper layer access by setting the system middleware. Therefore, the real-time performance and the stability of the remote pipe connecting system of the unmanned automobile are improved, the cost and the power consumption are reduced, and the efficiency is improved.
In one embodiment, as shown in fig. 3, a remote takeover system for an unmanned vehicle is provided, which further includes a state reporting module 220, configured to receive vehicle state information sent by a vehicle driving module and system state information sent by a system middleware, and report the vehicle state information and the system state information to a remote control end through the network middleware, so that the remote control end sends a remote control instruction according to the vehicle state information and the system state information.
Specifically, when the unmanned vehicle initiates a request for remote takeover to the remote control end, and the remote control end starts to take over the unmanned vehicle remotely, a remote control instruction needs to be generated according to vehicle state information and system state information, and the remote control instruction is sent to the network middleware. The vehicle driving module can acquire vehicle state information from the vehicle hardware equipment, wherein the vehicle state information comprises a driving state of a vehicle driver, a vehicle hardware state and the like, and therefore the vehicle state information is sent to the state reporting module. The system middleware can acquire the system state information of the automobile, so that the system state information is sent to the state reporting module.
And the state reporting module reports the received automobile state information and system state information to the remote control end through the network middleware, so that the remote control end sends a remote control instruction according to the automobile state information and the system state information.
In the embodiment of the application, the state reporting module is additionally arranged in the remote takeover system of the unmanned automobile, automobile state information and system state information can be timely acquired from the automobile driving module and the system middleware through the state reporting module, so that the automobile state information and the system state information are timely reported to the remote control end through the state reporting module, and finally the remote control end can timely follow the real-time automobile state information and the real-time system state information to send out a correct remote control instruction.
In one embodiment, the network middleware is further configured to provide a general network interface based on the embedded operating system and the network protocol stack, so that the state reporting module reports the vehicle state information and the system state information to the remote control terminal through the general network interface.
In particular, middleware is a common service that sits between a platform (hardware and operating system) and an application, with standard programming interfaces and protocols. These services may have multiple implementations that conform to the interface and protocol specifications for different operating systems and hardware platforms.
In the embodiment of the application, because the network middleware provides a universal network interface based on the embedded operating system and the network protocol stack, the state reporting module can report the automobile state information and the system state information to the remote control terminal through the universal network interface. Here, the network middleware shields the complexity of the underlying operating system, so that a program developer faces a simple and uniform development environment, the complexity of program design is reduced, attention is focused on business, and repeated work for transplanting programs on different system software is not needed, thereby greatly reducing technical burden. Therefore, the middleware brings to the application system, not only is development simple and convenient and development period shortened, but also reduces workload of maintenance, operation and management of the system and reduces investment of total cost of a computer.
In one embodiment, the embedded operating system is further configured to obtain an execution result of the control instruction from the hardware device, call an interface of the system middleware, and feed back the execution result to the vehicle driving module;
the vehicle driving module is also used for receiving the execution result and feeding the execution result back to the network middleware through the remote calling module;
and the network middleware is also used for receiving the execution result and feeding back the execution result to the remote control end.
Referring to fig. 4, after the control instruction is issued at the remote control end corresponding to the issue of the control instruction by the remote control end and the hardware device executes the control instruction, it is necessary to feed back an execution result of whether the execution is successful or not to the remote control end. Firstly, the hardware equipment sends an execution result to the embedded operating system, the embedded operating system obtains the execution result of the control instruction from the hardware equipment, an interface of the system middleware is called, and the execution result is fed back to the vehicle driving module. Then, the vehicle driving module receives the execution result and feeds the execution result back to the network middleware through the remote calling module. And finally, the network middleware receives the execution result and feeds the execution result back to the remote control end.
In the embodiment of the application, after the remote control instruction is issued to the embedded operating system step by step through the remote control end, after the embedded operating system controls the hardware device to execute the corresponding control instruction, the execution result of the control instruction needs to be acquired in time, and the execution result is fed back to the remote control end along the original path. Therefore, the remote control end can not only send out the control instruction in time, but also can acquire the execution condition of the sent control instruction in time and obtain the feedback in time so as to make a subsequent control instruction according to the execution condition.
In one embodiment, the remote control end comprises a control center and a monitoring center, wherein the control center is used for sending a remote control instruction according to automobile state information and system state information and receiving an execution result of the remote control instruction fed back by the network middleware;
the monitoring center is used for receiving the automobile state information and the system state information reported by the network middleware and sending the automobile state information and the system state information to the control center.
In the embodiment of the present application, as shown in fig. 4, the remote control end includes a control center and a monitoring center, where the monitoring center receives the vehicle state information acquired by the state reporting module from the vehicle driving module and the system state information acquired from the system middleware, so that the monitoring center can monitor the vehicle state and the system state in real time to find out an abnormal condition in time. And then, the monitoring center sends the acquired automobile state information and the system state information to the control center. And after receiving the automobile state information and the system state information, the control center analyzes the automobile state information and the system state information and calculates a control instruction by combining the current driving road section and road conditions. And the control instruction is sent to the network middleware and is transmitted to the hardware equipment through the network middleware, the remote calling module, the vehicle driving module, the system middleware and the embedded operating system, so that the hardware equipment executes the control instruction, the remote take-over of the vehicle hardware equipment through the remote control end is realized, and the safe driving of the vehicle is ensured until the vehicle is parked safely.
In one embodiment, the network protocol stack includes an embedded TCP/IP network protocol stack.
A Protocol stack (also called Protocol stack) is a specific software implementation of a computer network Protocol suite. One protocol in a suite of protocols is typically designed for only one purpose, which may make the design easier. Since each protocol module usually has to communicate with two other protocol modules above and below, they can usually be imagined as layers in a protocol stack. The lowest level of protocols always describes physical interactions with the hardware. Each advanced level adds more features. The user application is only handling the top-most protocol.
In the embodiment of the application, the network protocol stack in the remote takeover system of the unmanned automobile adopts an embedded TCP/IP network protocol stack, the embedded TCP/IP network protocol stack can be operated as a lightweight TCP/IP network protocol stack, a very small memory is used, some unnecessary functions are discarded, and main functions are reserved. Common embedded TCP/IP protocol stacks include the following μ C/TCP-IP, LwIP, UIP, which, of course, is not limited in this application.
In one embodiment, the embedded operating system comprises embedded Linux, FreeRTOS, RT-Thread.
Specifically, an Embedded Operating System (EOS) refers to an Operating System for an Embedded System. An embedded operating system is a kind of system software with wide application, and generally includes hardware-related underlying driver software, a system kernel, a device driver interface, a communication protocol, a graphical interface, a standardized browser, and the like. The embedded operating system is responsible for the allocation of all software and hardware resources of the embedded system, task scheduling, control and coordination of concurrent activities. It must be characterized by the system in which it is located, and it is possible to achieve the functions required by the system by loading and unloading certain modules. Currently, operating systems widely used in the embedded field are: embedded real-time operating system μ C/OS-II, Embedded Linux, Windows Embedded, VxWorks, FreeRTOS, RT-Thread, etc., and Android, iOS, etc. applied to smart phones and tablet computers.
In the embodiment of the application, the embedded Operating System has the advantages of small System kernel, strong specialization, simplified System, high real-time performance and the like, so that the remote takeover System of the unmanned vehicle is realized based on the embedded Operating System with low power consumption, and compared with the traditional architecture based on an ROS (Robot Operating System) System or a Linux System, the architecture can integrate highly customized hardware equipment, reduce System resource redundancy, improve efficiency, reduce cost and reduce power consumption.
In one embodiment, as shown in fig. 5, a remote takeover method for an unmanned vehicle is provided, and is applied to a remote takeover system for an unmanned vehicle, where the system includes a network middleware, a remote invocation module, a vehicle driving module, a system middleware, and an embedded operating system, and includes:
In the embodiment of the application, when the remote control end remotely takes over the unmanned automobile, firstly, the remote control end sends a remote control instruction to the network middleware; then, receiving a remote control instruction sent by a remote control end through a network middleware, and sending the remote control instruction to a remote calling module through a universal network interface; secondly, receiving a remote control instruction sent by the network middleware through a remote calling module, and calling a vehicle driving module according to the remote control instruction; calling system middleware through a vehicle driving module according to a remote control instruction; thirdly, calling an interface of the embedded operating system through the system middleware according to the remote control instruction; and finally, sending the remote control instruction to hardware equipment of the unmanned automobile through an interface of the embedded operating system so that the hardware equipment executes the remote control instruction to take over the unmanned automobile remotely. The remote control system can ensure that the unmanned automobile can be remotely taken over by the remote control end in time under special conditions so as to ensure that the automobile safely runs until the automobile is safely stopped.
The method adopts a remote takeover system based on the embedded operating system, and has the characteristics of low development cost, low maintenance cost, high response speed, high stability and low power consumption of the embedded operating system. The system can adapt to the diversity of bottom layer hardware and bottom layer network equipment by setting the network middleware, can adapt to objectified and abstracted vehicle driving modules by setting the remote calling module, and can package the embedded operating system to provide a general interface for upper layer access by setting the system middleware. Therefore, the real-time performance and the stability of the remote take-over system of the unmanned automobile are improved.
In one embodiment, as shown in fig. 6, the system further includes a status reporting module, and the method for remotely taking over an unmanned vehicle further includes:
and step 520, receiving the automobile state information sent by the automobile driving module and the system state information sent by the system middleware through the state reporting module, and reporting the automobile state information and the system state information to the remote control end through the network middleware, so that the remote control end sends a remote control instruction according to the automobile state information and the system state information.
In one embodiment, there is provided a remote takeover method of an unmanned vehicle further comprising:
and a universal network interface is provided by the network middleware based on the embedded operating system and the network protocol stack, so that the state reporting module reports the automobile state information and the system state information to the remote control terminal through the universal network interface.
In one embodiment, there is provided a remote takeover method of an unmanned vehicle further comprising:
acquiring an execution result of the control instruction from the hardware equipment through the embedded operating system, calling an interface of a system middleware, and feeding the execution result back to the vehicle driving module;
receiving an execution result through a vehicle driving module, and feeding the execution result back to the network middleware through a remote calling module;
and receiving the execution result through the network middleware, and feeding the execution result back to the remote control end.
In one embodiment, the remote control end comprises a control center and a monitoring center, and the method for remotely taking over the unmanned automobile further comprises the following steps:
sending a remote control instruction and receiving an execution result of the remote control instruction fed back by the network middleware through the control center according to the automobile state information and the system state information;
and the monitoring center receives the automobile state information and the system state information reported by the network middleware and sends the automobile state information and the system state information to the control center.
Fig. 7 is a schematic diagram of an internal structure of an electronic device in one embodiment. As shown in fig. 7, the electronic device includes a processor and a memory connected by a system bus. Wherein, the processor is used for providing calculation and control capability and supporting the operation of the whole electronic equipment. The memory may include a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The computer program is executable by a processor for implementing a method for remote takeover of an unmanned vehicle as provided in various embodiments below. The internal memory provides a cached execution environment for the operating system computer programs in the non-volatile storage medium. The electronic device may be a vehicle-mounted control device or a vehicle-mounted computer of the unmanned vehicle, which is not limited in this application.
The embodiment of the application also provides a computer readable storage medium. One or more non-transitory computer-readable storage media containing computer-executable instructions that, when executed by one or more processors, cause the processors to perform the steps of a remote takeover method for an unmanned vehicle.
A computer program product containing instructions which, when run on a computer, cause the computer to perform a remote takeover method for an unmanned vehicle.
Any reference to memory, storage, database, or other medium used by embodiments of the present application may include non-volatile and/or volatile memory. Suitable non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM), which acts as external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), synchronous Link (Synchlink) DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and bus dynamic RAM (RDRAM).
The above examples only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (14)
1. A remote takeover system for an unmanned vehicle, comprising:
the network middleware is used for receiving a remote control instruction sent by a remote control end and sending the remote control instruction to the remote calling module through a universal network interface;
the remote calling module is used for receiving a remote control instruction sent by the network middleware and calling the vehicle driving module according to the remote control instruction;
the vehicle driving module is used for calling system middleware according to the remote control instruction;
the system middleware is used for calling an interface of the embedded operating system according to the remote control instruction;
and the embedded operating system is used for sending the remote control instruction to hardware equipment through an interface of the embedded operating system so that the hardware equipment executes the remote control instruction to remotely take over the unmanned automobile.
2. The system of claim 1, further comprising:
and the state reporting module is used for receiving the automobile state information sent by the automobile driving module and the system state information sent by the system middleware, and reporting the automobile state information and the system state information to the remote control terminal through the network middleware so that the remote control terminal sends a remote control instruction according to the automobile state information and the system state information.
3. The system of claim 2, wherein the network middleware is further configured to provide the universal network interface based on the embedded operating system and a network protocol stack, so that the status reporting module reports the vehicle status information and the system status information to the remote control end through the universal network interface.
4. The system according to any one of claims 1 to 3, wherein the embedded operating system is further configured to obtain an execution result of the control instruction from the hardware device, call an interface of the system middleware, and feed back the execution result to a vehicle driving module;
the vehicle driving module is further used for receiving the execution result and feeding the execution result back to the network middleware through the remote calling module;
the network middleware is further configured to receive the execution result and feed back the execution result to the remote control end.
5. The system according to any one of claims 1 to 4, wherein the remote control end comprises a control center and a monitoring center, and the control center is used for sending a remote control command according to the automobile state information and the system state information and receiving an execution result of the remote control command fed back by the network middleware;
the monitoring center is used for receiving the automobile state information and the system state information reported by the network middleware and sending the automobile state information and the system state information to the control center.
6. The system according to any of claims 1 to 5, wherein said network protocol stack comprises an embedded TCP/IP network protocol stack.
7. The system according to any one of claims 1 to 6, wherein the embedded operating system comprises embedded Linux, FreeRTOS, RT-Thread.
8. A remote takeover method of an unmanned automobile is applied to a remote takeover system of the unmanned automobile, the system comprises a network middleware, a remote calling module, a vehicle driving module, a system middleware and an embedded operating system, and is characterized by comprising the following steps:
receiving a remote control instruction sent by a remote control end through a network middleware, and sending the remote control instruction to a remote calling module through a universal network interface;
receiving a remote control instruction sent by the network middleware through a remote calling module, and calling a vehicle driving module according to the remote control instruction;
calling system middleware through a vehicle driving module according to the remote control instruction;
calling an interface of the embedded operating system through the system middleware according to the remote control instruction;
and sending the remote control instruction to hardware equipment through an interface of the embedded operating system so that the hardware equipment executes the remote control instruction to remotely take over the unmanned automobile.
9. The method of claim 8, wherein the system further comprises a status reporting module, and wherein the method further comprises:
and a state reporting module is used for receiving automobile state information sent by the automobile driving module and system state information sent by the system middleware, and reporting the automobile state information and the system state information to the remote control end through the network middleware, so that the remote control end sends a remote control instruction according to the automobile state information and the system state information.
10. The method of claim 9, further comprising:
and providing the universal network interface through a network middleware based on the embedded operating system and a network protocol stack, so that the state reporting module reports the automobile state information and the system state information to the remote control terminal through the universal network interface.
11. The method according to any one of claims 8 to 10, further comprising:
acquiring an execution result of the control instruction from the hardware equipment through the embedded operating system, calling an interface of the system middleware, and feeding the execution result back to a vehicle driving module;
receiving the execution result through the vehicle driving module, and feeding back the execution result to the network middleware through the remote calling module;
and receiving the execution result through the network middleware, and feeding back the execution result to the remote control end.
12. The method according to any one of claims 8 to 11, wherein the remote control terminal comprises a control center and a monitoring center, the method further comprising:
sending a remote control instruction and receiving an execution result of the remote control instruction fed back by the network middleware through the control center according to the automobile state information and the system state information;
and receiving the automobile state information and the system state information reported by the network middleware through the monitoring center, and sending the automobile state information and the system state information to the control center.
13. An electronic device comprising a memory and a processor, the memory having stored thereon a computer program, wherein the computer program, when executed by the processor, causes the processor to perform the steps of the method of remote takeover of an unmanned vehicle according to any of claims 8 to 12.
14. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method for remote take-over of an unmanned vehicle according to any of claims 8 to 12.
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2020/074942 WO2021159346A1 (en) | 2020-02-12 | 2020-02-12 | Remote takeover system and method for driverless vehicle, electronic device, and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113260938A true CN113260938A (en) | 2021-08-13 |
| CN113260938B CN113260938B (en) | 2022-10-28 |
Family
ID=77181259
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202080007604.2A Active CN113260938B (en) | 2020-02-12 | 2020-02-12 | Remote access system and method for unmanned vehicle, electronic device, and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113260938B (en) |
| WO (1) | WO2021159346A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113844465A (en) * | 2021-12-02 | 2021-12-28 | 上海车右智能科技有限公司 | Automatic driving method and system |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114387802B (en) * | 2021-12-27 | 2023-07-04 | 江苏徐工工程机械研究院有限公司 | Remote take-over system and method for unmanned mining truck of surface mine |
| CN115209108A (en) * | 2022-07-13 | 2022-10-18 | 安徽工业大学 | Transmission system based on 5G embedded video |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104424176A (en) * | 2013-08-20 | 2015-03-18 | 上海能感物联网有限公司 | Method for controlling automobile driving in short range or long distance by foreign language natural language text |
| CN105022315A (en) * | 2015-04-16 | 2015-11-04 | 广东好帮手电子科技股份有限公司 | Vehicle remote control system based on Internet of Vehicles, control method and vehicle |
| WO2016089067A1 (en) * | 2014-12-03 | 2016-06-09 | 황호정 | Unmanned vehicle remote control system and method |
| CN108241352A (en) * | 2016-12-25 | 2018-07-03 | 青岛祥智电子技术有限公司 | A kind of long-range control method of unmanned motor vehicle |
| CN110032176A (en) * | 2019-05-16 | 2019-07-19 | 广州文远知行科技有限公司 | Remote take-over method, device, equipment and storage medium for unmanned vehicle |
| CN110083109A (en) * | 2019-05-20 | 2019-08-02 | 中国汽车工程研究院股份有限公司 | A kind of unmanned vehicle tele-control system and its method |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4507884B2 (en) * | 2005-01-11 | 2010-07-21 | トヨタ自動車株式会社 | Vehicle with remote control system and remote control device |
| CN106354130B (en) * | 2016-10-31 | 2021-01-22 | 北京新能源汽车股份有限公司 | Unmanned automobile and control method and remote monitoring system thereof |
-
2020
- 2020-02-12 WO PCT/CN2020/074942 patent/WO2021159346A1/en active Application Filing
- 2020-02-12 CN CN202080007604.2A patent/CN113260938B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104424176A (en) * | 2013-08-20 | 2015-03-18 | 上海能感物联网有限公司 | Method for controlling automobile driving in short range or long distance by foreign language natural language text |
| WO2016089067A1 (en) * | 2014-12-03 | 2016-06-09 | 황호정 | Unmanned vehicle remote control system and method |
| CN105022315A (en) * | 2015-04-16 | 2015-11-04 | 广东好帮手电子科技股份有限公司 | Vehicle remote control system based on Internet of Vehicles, control method and vehicle |
| CN108241352A (en) * | 2016-12-25 | 2018-07-03 | 青岛祥智电子技术有限公司 | A kind of long-range control method of unmanned motor vehicle |
| CN110032176A (en) * | 2019-05-16 | 2019-07-19 | 广州文远知行科技有限公司 | Remote take-over method, device, equipment and storage medium for unmanned vehicle |
| CN110083109A (en) * | 2019-05-20 | 2019-08-02 | 中国汽车工程研究院股份有限公司 | A kind of unmanned vehicle tele-control system and its method |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113844465A (en) * | 2021-12-02 | 2021-12-28 | 上海车右智能科技有限公司 | Automatic driving method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113260938B (en) | 2022-10-28 |
| WO2021159346A1 (en) | 2021-08-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113260938B (en) | Remote access system and method for unmanned vehicle, electronic device, and storage medium | |
| US10994709B2 (en) | System for driverless operation of utility vehicles | |
| US20170349139A1 (en) | Anti-theft system for driverless vehicle | |
| US20170269600A1 (en) | Method and device for monitoring a system of a vehicle which provides an at least partially automated driving function | |
| US11975727B2 (en) | Autonomous vehicle control system | |
| CN113366399B (en) | Vehicle control method and device based on remote take-over and computer equipment | |
| CN113359759A (en) | Parking control method and system based on automatic driving, vehicle and storage medium | |
| EP2430625B1 (en) | Method for using the computing unit of an emergency call unit, and emergency call unit | |
| CN110949406A (en) | Intelligent driving system and method | |
| CN114228791A (en) | Method and device for upgrading remote RM after remote restart of full-automatic operation train | |
| CN113306534B (en) | Unmanned hub redundancy brake control system and method | |
| CN112905313B (en) | Task processing method and device and electronic equipment | |
| CN110588669A (en) | Method and control unit for performing a minimum risk maneuver in a malfunctioning vehicle | |
| CN111505977B (en) | Function auxiliary debugging method, device, system and medium | |
| JP2019175017A (en) | Communication device and communication method | |
| EP3072022A1 (en) | System and method to make possible autonomous operation and/or external control of a motor vehicle | |
| CN110688418A (en) | Intelligent prison patrol method and device, computer equipment and storage medium | |
| EP4097996A1 (en) | Method for exchanging data between a trailer and a road user, trailer communication module and trailer | |
| US20150221143A1 (en) | Method for managing breakdown of construction equipment for providing current workable information | |
| CN115320639A (en) | L3-level automatic driving system state degradation processing method and system | |
| EP3072020A1 (en) | System and method to make possible autonomous operation and/or external control of a motor vehicle | |
| CN116097192A (en) | Method and device for controlling chassis of unmanned vehicle and computer equipment | |
| CN114545892B (en) | Method for monitoring abnormality of execution mechanism, monitoring device and computer readable storage medium | |
| CN118833185A (en) | Mining vehicle remote control method, whole vehicle controller and remote control system | |
| CN114545892A (en) | Execution mechanism abnormity monitoring method, monitoring equipment and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |