CN114839967A - Remote driving assisting method and device, vehicle and storage medium - Google Patents
Remote driving assisting method and device, vehicle and storage medium Download PDFInfo
- Publication number
- CN114839967A CN114839967A CN202210319167.9A CN202210319167A CN114839967A CN 114839967 A CN114839967 A CN 114839967A CN 202210319167 A CN202210319167 A CN 202210319167A CN 114839967 A CN114839967 A CN 114839967A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- teaching
- obstacle
- mounted terminal
- indication information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 79
- 238000001514 detection method Methods 0.000 claims abstract description 81
- 230000008569 process Effects 0.000 claims abstract description 29
- 238000004590 computer program Methods 0.000 claims description 19
- 238000012163 sequencing technique Methods 0.000 claims description 6
- 230000007547 defect Effects 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 10
- 230000000007 visual effect Effects 0.000 description 7
- 230000006870 function Effects 0.000 description 5
- 230000008447 perception Effects 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000004888 barrier function Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
- G05D1/0238—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using obstacle or wall sensors
- G05D1/024—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using obstacle or wall sensors in combination with a laser
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
- G05D1/0214—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory in accordance with safety or protection criteria, e.g. avoiding hazardous areas
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
- G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
- G05D1/0253—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0257—Control of position or course in two dimensions specially adapted to land vehicles using a radar
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0276—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Aviation & Aerospace Engineering (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Electromagnetism (AREA)
- Optics & Photonics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Traffic Control Systems (AREA)
Abstract
The embodiment of the invention provides a remote assistant driving method and device, a vehicle and a storage medium. The remote assistant driving method includes: in the automatic driving process of the vehicle, when an obstacle is detected, obstacle detection information sent by the vehicle-mounted terminal is received; receiving a teaching instruction aiming at the obstacle detection information, and generating indication information according to the teaching instruction; and sending the indication information to the vehicle-mounted terminal, wherein the vehicle-mounted terminal is used for determining a running path according to the indication information and controlling the vehicle to run based on the running path. According to the embodiment of the invention, the vehicle can be remotely assisted to pass through the teaching instruction of the cloud server under the condition that the vehicle can not pass through the obstacle during automatic driving, so that the defects of sensing, decision and planning and the like of the vehicle-mounted terminal are overcome, and the passing capacity of the automatic driving vehicle is effectively improved.
Description
Technical Field
The invention relates to the technical field of unmanned driving, in particular to a remote assistant driving method, a remote assistant driving device, a vehicle and a storage medium.
Background
As the progress of vehicle intelligence advances, the related art has now enabled automatic driving for driving control of a vehicle. And in the automatic driving process, the vehicle is controlled to run mainly according to the information of the high-precision map and the real-time environment information monitored by the vehicle sensor by the vehicle-mounted terminal. However, when the vehicle encounters a situation such as a front obstacle blocking in the automatic driving process, the vehicle-mounted terminal may not be able to make an automatic driving decision due to the fact that the decision is made only by the environment information detected by the vehicle-mounted terminal and the sensing, decision and planning of the vehicle-mounted terminal are insufficient. The vehicle may stop in place without continuing to travel, resulting in a reduction in the ability to autonomously drive.
Disclosure of Invention
In view of the above, embodiments of the present invention are proposed in order to provide a remote assisted driving method, a corresponding remote assisted driving apparatus, a vehicle and a storage medium that overcome or at least partially solve the above-mentioned problems.
The embodiment of the invention provides a remote auxiliary driving method which is applied to a cloud server, wherein the cloud server is connected with a vehicle-mounted terminal, and the method comprises the following steps:
in the automatic driving process of the vehicle, when an obstacle is detected, obstacle detection information sent by the vehicle-mounted terminal is received;
receiving a teaching instruction for the obstacle detection information;
generating indication information according to the teaching instruction;
and sending the indication information to the vehicle-mounted terminal, wherein the vehicle-mounted terminal is used for determining a running path according to the indication information and controlling the vehicle to run based on the running path.
Optionally, the obstacle detection information includes an obstacle image, the teaching instruction includes a teaching pose point, and the step of receiving the teaching instruction for the obstacle detection information includes:
acquiring global map data;
determining an obstacle icon according to the obstacle image;
displaying the obstacle icon on a visualized global map data;
teaching pose points on the visualized global map data is received.
Optionally, the obstacle detection information further includes vehicle reachable space information, and the step of generating instruction information according to the teaching instruction includes:
determining an reachable position range corresponding to the vehicle reachable space information;
judging whether the teaching pose point is located in the reachable position range;
and when the teaching pose point is positioned in the reachable position range, generating a teaching path based on the teaching pose point to serve as indication information.
Optionally, the teaching pose point corresponds to a teaching time, and the step of generating a teaching path based on the teaching pose point includes:
sequencing the teaching time to determine a teaching sequence;
and connecting the teaching pose points according to the teaching sequence to generate a teaching path.
Optionally, the obstacle detection information includes obstacle edge information; the teach instruction includes an obstacle envelope range value, and the step of generating the instruction information according to the teach instruction includes:
and updating the obstacle edge information by adopting the obstacle envelope range value to generate indication information.
The embodiment of the invention also provides a remote auxiliary driving method, which is applied to a vehicle-mounted terminal, wherein the vehicle-mounted terminal is connected with a cloud server, and the method comprises the following steps:
in the automatic driving process of the vehicle, when an obstacle is detected, obstacle detection information is generated;
the obstacle detection information is sent to the cloud server, and the cloud server is used for receiving the obstacle detection information sent by the vehicle-mounted terminal; receiving a teaching instruction for the obstacle detection information; generating indication information according to the teaching instruction; sending the indication information to the vehicle-mounted terminal;
receiving the indication information sent by the cloud server;
and determining a running path according to the indication information, and controlling the vehicle to run based on the running path.
The embodiment of the invention also provides a remote auxiliary driving device, which is applied to a cloud server, wherein the cloud server is connected with a vehicle-mounted terminal, and the device comprises:
the first receiving module is used for receiving obstacle detection information sent by the vehicle-mounted terminal when an obstacle is detected in the automatic driving process of the vehicle;
a second receiving module for receiving a teaching instruction for the obstacle detection information,
the generation module is used for generating indication information according to the teaching instruction;
the first sending module is used for sending the indication information to the vehicle-mounted terminal, and the vehicle-mounted terminal is used for determining a running path according to the indication information and controlling the vehicle to run based on the running path.
The embodiment of the invention also provides a remote auxiliary driving device, which is applied to a vehicle-mounted terminal, wherein the vehicle-mounted terminal is connected with a cloud server, and the device comprises:
the detection module is used for generating obstacle detection information when an obstacle is detected in the automatic driving process of the vehicle;
the second sending module is used for sending the obstacle detection information to the cloud server, and the cloud server is used for receiving the obstacle detection information sent by the vehicle-mounted terminal; receiving a teaching instruction for the obstacle detection information; generating indication information according to the teaching instruction; sending the indication information to the vehicle-mounted terminal;
the third receiving module is used for receiving the indication information sent by the cloud server;
and the control module is used for determining a running path according to the indication information and controlling the vehicle to run based on the running path.
Embodiments of the present invention further provide a vehicle, which includes a processor, a memory, and a computer program stored on the memory and capable of running on the processor, and when the computer program is executed by the processor, the steps of the remote assistant driving method are implemented.
An embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program implements the steps of the remote assisted driving method described above.
The embodiment of the invention has the following advantages:
in the embodiment of the invention, in the automatic driving process of the vehicle, when an obstacle is detected, obstacle detection information sent by the vehicle-mounted terminal is received; receiving a teaching instruction for the obstacle detection information; generating instruction information according to the teaching instruction; and sending the indication information to the vehicle-mounted terminal, wherein the vehicle-mounted terminal is used for determining a running path according to the indication information and controlling the vehicle to run based on the running path. When the automatic driving meets the obstacle and cannot pass, a teaching instruction of an operation decision performed on obstacle detection information corresponding to the obstacle is sent out through the perception and decision-making capability of a remote driver by means of the cloud server, a feasible path of the vehicle is indicated, the vehicle-mounted terminal passes through the obstacle based on the indication information, a driving assistance effect is provided for a remote driver, the defects in the aspects of perception, decision-making, planning and the like of the vehicle-mounted terminal are overcome, and the passing capability of the automatic driving vehicle is effectively improved.
Drawings
FIG. 1 is a flow chart illustrating the steps of an embodiment of a remote assisted driving method of the present invention;
FIG. 2 is a flow chart of steps in another remote assisted driving method embodiment of the present invention;
FIG. 3 is a graphical illustration of global map data for an example remote assisted driving method of the present invention;
FIG. 4 is a flow chart illustrating steps of yet another embodiment of a remote assisted driving method of the present invention;
FIG. 5 is a block diagram of an embodiment of a remote assistant driving device according to the present invention;
fig. 6 is a block diagram showing another embodiment of the remote driving assistance apparatus according to the present invention.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
In the related art, for the driving control of an autonomous vehicle, the environment around the vehicle is mainly detected and recognized by an on-vehicle sensor such as a camera, a laser radar, or the like, and a path is planned according to the current position of the vehicle and a high-precision map. And travel according to the real-time environment information according to the established route. However, in the practical application process, since a certain safe lateral margin is reserved in the driving control, when the vehicle passes through some narrow channels or encounters an obstacle on a road, the vehicle may stop in place and not continue to run due to detection errors of the measured margin or a small detection margin before the vehicle stops in an automatic driving condition, and the automatic driving has poor traffic capacity, so that the whole process of unmanned driving cannot be realized.
Referring to fig. 1, a flowchart illustrating steps of an embodiment of a remote assistant driving method according to the present invention is shown, and the method is applied to a cloud server, and the cloud server is connected to a vehicle-mounted terminal. The connection network between the cloud server and the vehicle-mounted terminal may be a mobile communication network or an internet of things, and the embodiment of the present invention is not particularly limited.
The cloud server can be connected with terminal equipment used by a remote driver, and the remote driver can make manual aid decision on data received by the cloud server through the terminal equipment.
The remote assistant driving method may specifically include the steps of:
the cloud server can be connected with the vehicle-mounted terminal in real time. In the automatic driving process of the vehicle based on the automatic driving working condition, when the vehicle-mounted terminal detects that the driving path has an obstacle, the related information generated by the obstacle detection can be used as obstacle detection information; the vehicle can not continue to travel due to the fact that the obstacle is clamped, the vehicle-mounted terminal can send obstacle detection information to the cloud server, and the cloud server receives the obstacle detection information. It should be noted that, the driving process may refer to a driving path along which the vehicle moves, such as a path from a location to a target location; it is also possible to have a parking path to park the vehicle, such as a path to automatically park the target parking space from the present location. In addition, the automatic driving function of the vehicle can be started in a remote starting mode, and a related instruction can be triggered in the vehicle.
after the cloud server receives the obstacle detection information, the remote driver can remotely observe the current obstacle condition of the vehicle on the cloud server. The remote driver can make teaching instructions for manual decision aiming at the obstacle detection information; and the cloud server receives the teaching instruction.
103, generating instruction information according to the teaching instruction;
and the cloud server generates indicating information for indicating the vehicle-mounted terminal to make an automatic driving decision according to the visual angle instruction of the decision made by the remote driver for the obstacle detection information. The indication information is used for assisting the vehicle-mounted terminal to make an automatic decision.
In an embodiment of the present invention, the obstacle detection information includes obstacle edge information; the teach instruction includes an obstacle envelope range value, and the step of generating instruction information according to the teach instruction includes:
and a substep S1031, updating the obstacle edge information by using the obstacle envelope range value to generate indication information.
In practical application, when a vehicle encounters an obstacle in an automatic driving process, the vehicle-mounted terminal is difficult to judge the edge of the obstacle and the obstacle cannot be clamped, so that the detected edge information of the obstacle is wrong. Therefore, the teaching instruction issued by the remote driver may include the obstacle envelope range value. And updating the obstacle edge information by adopting the obstacle envelope range value so as to correct the actual edge of the obstacle to generate indication information and indicate the vehicle-mounted terminal to carry out path planning based on the updated obstacle edge.
And 104, sending the indication information to the vehicle-mounted terminal, wherein the vehicle-mounted terminal is used for determining a running path according to the indication information and controlling the vehicle to run based on the running path.
And after the cloud server generates the indication information, the indication information is sent to the vehicle-mounted terminal. The vehicle-mounted terminal can determine a driving path passing through the obstacle according to the indication information; and controlling the vehicle to run through the obstacle according to the running path.
In the automatic driving process of the vehicle, when an obstacle is detected, the embodiment of the invention receives obstacle detection information sent by the vehicle-mounted terminal; receiving a teaching instruction for the obstacle detection information; generating indication information according to the teaching instruction; and sending the indication information to the vehicle-mounted terminal, wherein the vehicle-mounted terminal is used for determining a running path according to the indication information and controlling the vehicle to run based on the running path. When the automatic driving meets the obstacle and cannot pass, a teaching instruction of an operation decision performed on obstacle detection information corresponding to the obstacle is sent out through the perception and decision-making capability of a remote driver by means of the cloud server, a feasible path of the vehicle is indicated, the vehicle-mounted terminal passes through the obstacle based on the indication information, a driving assistance effect is provided for a remote driver, the defects in the aspects of perception, decision-making, planning and the like of the vehicle-mounted terminal are overcome, and the passing capability of the automatic driving vehicle is effectively improved.
Referring to fig. 2, a flowchart illustrating steps of another embodiment of the remote assistant driving method according to the present invention is shown, and the method is applied to a cloud server, and the cloud server is connected to a vehicle-mounted terminal. The remote assistant driving method may specifically include the steps of:
in the automatic driving process of the vehicle, when the vehicle-mounted terminal detects an obstacle and cannot drive according to an automatic driving planned path due to the fact that the movement intention (such as the fact that the vehicle is in a jammed state in front) or the specific edge of the obstacle cannot be judged, the vehicle-mounted terminal takes an obstacle image shot aiming at the obstacle and the detected real-time vehicle reachable space information of the current position of the vehicle as obstacle detection information and sends the obstacle detection information to the cloud server. The cloud server receives the obstacle detection information. The obstacle image can be obtained by shooting with a vision sensor installed on the vehicle, for example, an image containing an obstacle shot by a camera of a vehicle-mounted all-round system is used as the obstacle image; a third party camera, such as a tachograph, mounted on the vehicle may also be used.
It should be noted that the vehicle reachable space information is a position where the vehicle can reach without obstacle at the current position. The vehicle reachable spatial information is composed of a plurality of reachable spatial coordinate points. The reachable spatial coordinate point may be regarded as coordinates of a position that is finally reachable by walking along several straight lines from the current position of the vehicle toward other positions.
after the cloud server receives the obstacle image and the vehicle reachable space information, the current position of the vehicle can be determined. And acquiring global map data according to the current position. The global map data may be global map data within a preset distance range with a current position as a center, so as to avoid a phenomenon that decision efficiency is reduced due to obtaining of too large global map data. The size of the preset distance range may be determined according to a planned path range of the automatic driving of the vehicle, such as a circular range with a radius of 10 meters, and other size ranges may be set by a person skilled in the art, which is not specifically limited in the embodiment of the present invention.
and determining the occupied range of the obstacle on the global map according to the obstacle image, thereby determining the corresponding obstacle icon. The obstacle icon can be an icon showing the specific outline of the obstacle, or a simple icon with a corresponding size can be determined only for the edge width of the obstacle. If a rectangular icon is used as the simple icon, the edge width of the obstacle is used as the length of the rectangular icon.
after the global map data is obtained, visualization processing can be performed on the global map data, the global map data is displayed in a graphic form, and the obstacle icon is displayed on the visual global map data, so that a remote driver can visually observe the obstacle. Referring to fig. 3, a style of a current road is shown on the global map 100, an obstacle icon 200 is shown on the road, and the obstacle icon 200 visually shows an edge of an obstacle.
the remote driver can click on the visual global map data, and the cloud server receives a plurality of teaching pose points clicked by the click operation. The teaching pose points are vehicle pose control points of a remote driver passing through the obstacle. The number of the teaching pose points is at least two, so that the teaching pose points can form a path. For example, referring to fig. 3, teaching pose points are 5.
in practical application, the specific position range which can be reached by the vehicle at present can be determined according to the reachable space information of the vehicle, and the position range is taken as the reachable position range. Specifically, the edge of the reachable position range may be determined according to the reachable space coordinate point in the vehicle reachable space information, and the position range surrounded by the edge may be the reachable position range.
after the reachable position range is determined, whether the teaching pose point is in the reachable position range or not can be judged, whether the teaching pose point for the remote driver to make a decision is realized by the vehicle or not is determined, and the teaching pose point is verified.
The manner of judgment may be a difference method. The specific position coordinates of the teaching pose points are subtracted from the reachable position range, and when the difference value meets the requirement of the edge of the reachable position range, the teaching pose points are determined to be in the reachable position range; otherwise, determining that the teaching pose point is not in the reachable position range. The teaching pose points can be judged one by one. Thereby determining the state of each teaching pose point.
208, when the teaching pose point is located in the reachable position range, generating a teaching path based on the teaching pose point to serve as indication information;
when the teaching pose point is located in the reachable position range, the teaching pose point can be determined to be a position point which can be reached by the vehicle, and a teaching path is generated based on the teaching pose point located in the reachable position range and used as indication information to indicate the vehicle-mounted terminal to run.
Further, when the teaching pose point is outside the reachable position range, the teaching pose point may be deleted.
In an embodiment of the present invention, the teaching pose point corresponds to a teaching time, and the step of generating a teaching path based on the teaching pose point includes:
substep S2081, sequencing the teaching time and determining a teaching sequence;
in practical application, the time for a remote driver to click on a teaching pose point on visual global map data can be used as the teaching time corresponding to the teaching pose point. And sequencing the teaching time corresponding to all the teaching pose points according to the time sequence, and determining the teaching sequence.
And a substep S2082 of connecting the teaching pose points according to the teaching sequence and generating a teaching path.
After the teaching sequence is determined, sequentially connecting the teaching pose point from the first teaching pose point to the next teaching pose point according to the teaching sequence until the last teaching pose point, and generating a teaching path. The teaching path should be a smooth transition multi-line segment, so that there is no abrupt point in the path.
And 209, sending the indication information to the vehicle-mounted terminal, wherein the vehicle-mounted terminal is used for determining a running path according to the indication information and controlling the vehicle to run based on the running path.
In practical application, the cloud server sends the teaching path to the vehicle-mounted terminal, and the vehicle-mounted terminal controls the vehicle to travel according to the travel path according to the teaching path as the travel path of the vehicle.
In the embodiment of the invention, when the automatic driving is unable to pass through obstacles, teaching pose points are made on visual global map data by means of the perception and decision-making capability of a remote driver of the cloud server, and the cloud server can generate a teaching path for indicating the vehicle-mounted terminal according to the teaching pose points; the vehicle-mounted terminal passes through the barrier based on the teaching path, and when the vehicle-mounted terminal cannot make a decision, the decision can be made according to the teaching pose point of the remote driver, so that the vehicle-mounted terminal can continuously control the vehicle to pass through the barrier, and the traffic capacity of the automatically-driven vehicle is effectively improved; so that the vehicle can be driven completely unmanned.
Referring to fig. 4, a flowchart illustrating steps of another embodiment of a remote assistant driving method according to the present invention is shown, and the method is applied to a vehicle-mounted terminal, where the vehicle-mounted terminal is connected to a cloud server.
The remote assistant driving method specifically comprises the following steps:
the vehicle-mounted terminal determines whether an obstacle is detected or not according to signals of the sensor in real time in the automatic driving process of the vehicle, and when the obstacle is detected, obstacle detection information can be generated. The obstacle detection information may include an image of an obstacle, edge information, and reachable position space information of the vehicle, among others.
the vehicle-mounted terminal sends the obstacle detection information to the cloud server, the cloud server can receive and display the obstacle detection information, and a remote driver makes corresponding teaching instructions for the obstacle detection information to assist the vehicle-mounted terminal in making decisions; and the cloud server generates corresponding indication information according to the teaching instruction and sends the indication information to the vehicle-mounted terminal.
after the cloud server sends the indication information, the indication information can be received; and analyzes the indication information.
And step 404, determining a running path according to the indication information, and controlling the vehicle to run based on the running path.
In practical application, the driving path passing through the obstacle can be determined according to the indication information. The vehicle-mounted terminal can detect the current pose of the vehicle, control the current pose of the vehicle to enter the pose corresponding to the starting point of the driving path, control the vehicle to drive according to the driving path and pass through the barrier.
In order to facilitate the further understanding of the present invention for the skilled person, the present invention is described below by way of specific scenarios by way of example.
A user remotely calls a vehicle in an application program to trigger the vehicle to carry out an automatic driving working condition; and the vehicle-mounted terminal plans a path according to the high-precision map under the automatic driving working condition and controls the vehicle to run according to the path.
When the vehicle stops on a road when encountering a vehicle (obstacle) in front and stops to drive, and the vehicle-mounted terminal cannot judge the movement intention of the vehicle in front, the vehicle cannot be controlled to drive according to a planned path, so that the vehicle is not stuck in front. At this time, the vehicle-mounted terminal can shoot an image (obstacle image) of the vehicle in front according to the panoramic all-around system on the vehicle, determine real-time vehicle reachable space information according to the vehicle surrounding environment condition of the panoramic all-around system, and send the image of the vehicle in front and the vehicle reachable space information to the cloud server.
And the cloud server acquires global map data after receiving the image of the front vehicle and the vehicle reachable space information sent by the vehicle-mounted terminal, and visualizes the global map data. The width of the vehicle is determined from the images of the vehicles in front and corresponding obstacle icons are generated. And the obstacle icon is displayed on the visualized global map data 100, as shown in fig. 3, on which the obstacle icon 200 is displayed.
The remote driver clicks on the visualized global map data 100, showing 5 teaching pose points (1-5). And when the teaching pose points are determined to be in the vehicle reachable space range, determining vehicle poses corresponding to the 5 teaching pose points, and connecting the 5 teaching pose points to determine an indication path. And issuing the indication path to the vehicle-mounted terminal.
The vehicle-mounted terminal receives the indication path, converts the indication path into a running path for controlling the vehicle to run, detects the current pose 300 of the vehicle, controls the vehicle to enter a first indication pose point from the current pose, and then controls the vehicle to run forwards according to the running path.
It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the illustrated order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments of the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.
Referring to fig. 5, a block diagram of a remote assistant driving device according to an embodiment of the present invention is shown, where the remote assistant driving device is applied to a cloud server, and the cloud server is connected to a vehicle-mounted terminal, and the remote assistant driving device specifically includes the following modules:
a first receiving module 501, configured to receive obstacle detection information sent by the vehicle-mounted terminal when an obstacle is detected in an automatic driving process of a vehicle;
a second receiving module 502 for receiving teaching instructions for the obstacle detection information,
a generating module 503, configured to generate instruction information according to the teaching instruction;
a first sending module 504, configured to send the indication information to the vehicle-mounted terminal, where the vehicle-mounted terminal is configured to determine a driving path according to the indication information, and control the vehicle to drive based on the driving path.
In an embodiment of the present invention, the obstacle detection information includes an obstacle image, the teaching instruction includes a teaching pose point, and the second receiving module 502 includes:
the acquisition submodule is used for acquiring global map data;
the first determining submodule is used for determining an obstacle icon according to the obstacle image;
a display sub-module for displaying the obstacle icon on the visualized global map data;
and the receiving submodule is used for receiving teaching pose points on the visual global map data.
In an embodiment of the present invention, the obstacle detection information further includes vehicle reachable space information, and the generating module 503 includes:
the second determining submodule is used for determining the reachable position range corresponding to the vehicle reachable space information;
the judgment submodule is used for judging whether the teaching pose point is positioned in the reachable position range;
and the generation sub-module is used for generating a teaching path based on the teaching pose point as indication information when the teaching pose point is positioned in the reachable position range.
In an embodiment of the present invention, the teaching pose corresponds to a teaching time, and the generating sub-module includes:
the sequencing unit is used for sequencing the teaching time and determining a teaching sequence;
and the connection module is used for connecting the teaching pose points according to the teaching sequence to generate a teaching path.
In an embodiment of the present invention, the obstacle detection information includes obstacle edge information; the teach instruction includes an obstacle envelope range value, the
And the updating module is used for updating the obstacle edge information by adopting the obstacle envelope range value so as to generate indication information.
Referring to fig. 6, a block diagram of another embodiment of the remote assistant driving device according to the present invention is shown, where the remote assistant driving device is applied to a vehicle-mounted terminal, and the vehicle-mounted terminal is connected to a cloud server, and specifically includes the following modules:
the detection module 601 is used for generating obstacle detection information when an obstacle is detected in the automatic driving process of the vehicle;
a second sending module 602, configured to send the obstacle detection information to the cloud server, where the cloud server is configured to receive the obstacle detection information sent by the vehicle-mounted terminal; receiving a teaching instruction for the obstacle detection information; generating indication information according to the teaching instruction; sending the indication information to the vehicle-mounted terminal;
a third receiving module 603, configured to receive the indication information sent by the cloud server;
and the control module 604 is configured to determine a running path according to the indication information, and control the vehicle to run based on the running path.
In an embodiment of the present invention, the obstacle detection information includes an obstacle image, the teaching instruction includes teaching pose points, and the cloud server is configured to obtain global map data; determining an obstacle icon according to the obstacle image; displaying the obstacle icon on a visualized global map data; teaching pose points on the visualized global map data is received.
In an embodiment of the present invention, the obstacle detection information further includes vehicle reachable space information, and the cloud server is configured to determine a reachable position range corresponding to the vehicle reachable space information; judging whether the teaching pose point is located in the reachable position range; and when the teaching pose point is positioned in the reachable position range, generating a teaching path based on the teaching pose point to serve as indication information.
In an embodiment of the present invention, the teaching pose points correspond to teaching times, and the cloud server is further configured to sort the teaching times and determine a teaching sequence; and connecting the teaching pose points according to the teaching sequence to generate a teaching path.
In an embodiment of the present invention, the obstacle detection information includes obstacle edge information; the teaching instruction comprises an obstacle envelope range value, and the cloud server is used for updating the obstacle edge information by adopting the obstacle envelope range value so as to generate indication information.
For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.
An embodiment of the present invention further provides an electronic device, including:
a processor and a storage medium storing a computer program executable by the processor, the computer program being executable by the processor to perform a method according to any one of the embodiments of the invention when the electronic device is run. The specific implementation manner and technical effects are similar to those of the method embodiment, and are not described herein again.
Embodiments of the present invention also provide a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the method according to any one of the embodiments of the present invention. The specific implementation manner and technical effects are similar to those of the method embodiment, and are not described herein again.
The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the embodiments of the invention.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or terminal that comprises the element.
The above detailed description of the remote assistant driving method, the remote assistant driving device, the vehicle and the storage medium provided by the present invention, and the detailed examples are applied herein to explain the principle and the implementation of the present invention, and the above descriptions of the embodiments are only used to help understanding the method and the core idea of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (10)
1. The remote driving assistance method is applied to a cloud server, the cloud server is connected with a vehicle-mounted terminal, and the method comprises the following steps:
in the automatic driving process of the vehicle, when an obstacle is detected, obstacle detection information sent by the vehicle-mounted terminal is received;
receiving a teaching instruction for the obstacle detection information;
generating instruction information according to the teaching instruction;
and sending the indication information to the vehicle-mounted terminal, wherein the vehicle-mounted terminal is used for determining a running path according to the indication information and controlling the vehicle to run based on the running path.
2. The method according to claim 1, wherein the obstacle detection information includes an obstacle image, the teaching instruction includes a teaching pose point, and the step of receiving the teaching instruction for the obstacle detection information includes:
acquiring global map data;
determining an obstacle icon according to the obstacle image;
displaying the obstacle icon on a visualized global map data;
teaching pose points on the visualized global map data is received.
3. The method of claim 2, wherein the obstacle detection information further includes vehicle reachable space information, and the step of generating the indication information according to the teaching instruction includes:
determining an reachable position range corresponding to the vehicle reachable space information;
judging whether the teaching pose point is located in the reachable position range;
and when the teaching pose point is positioned in the reachable position range, generating a teaching path based on the teaching pose point to serve as indication information.
4. The method according to claim 3, wherein the teaching pose point has a teaching time, and the step of generating a teaching path based on the teaching pose point comprises:
sequencing the teaching time to determine a teaching sequence;
and connecting the teaching pose points according to the teaching sequence to generate a teaching path.
5. The method of claim 1, wherein the obstacle detection information includes obstacle edge information; the teach instruction includes an obstacle envelope range value, and the step of generating the instruction information according to the teach instruction includes:
and updating the obstacle edge information by adopting the obstacle envelope range value to generate indication information.
6. A remote driving assistance method is applied to a vehicle-mounted terminal, the vehicle-mounted terminal is connected with a cloud server, and the method comprises the following steps:
in the automatic driving process of the vehicle, when an obstacle is detected, obstacle detection information is generated;
the obstacle detection information is sent to the cloud server, and the cloud server is used for receiving the obstacle detection information sent by the vehicle-mounted terminal; receiving a teaching instruction for the obstacle detection information; generating indication information according to the teaching instruction; sending the indication information to the vehicle-mounted terminal;
receiving the indication information sent by the cloud server;
and determining a running path according to the indication information, and controlling the vehicle to run based on the running path.
7. The utility model provides a long-range driver assistance device, its characterized in that is applied to high in the clouds server, high in the clouds server is connected with vehicle-mounted terminal, the device includes:
the first receiving module is used for receiving obstacle detection information sent by the vehicle-mounted terminal when an obstacle is detected in the automatic driving process of the vehicle;
a second receiving module for receiving a teaching instruction for the obstacle detection information,
the generation module is used for generating indication information according to the teaching instruction;
the first sending module is used for sending the indication information to the vehicle-mounted terminal, and the vehicle-mounted terminal is used for determining a running path according to the indication information and controlling the vehicle to run based on the running path.
8. The utility model provides a long-range driver assistance device, its characterized in that is applied to vehicle mounted terminal, vehicle mounted terminal is connected with high in the clouds server, the device includes:
the detection module is used for generating obstacle detection information when an obstacle is detected in the automatic driving process of the vehicle;
the second sending module is used for sending the obstacle detection information to the cloud server, and the cloud server is used for receiving the obstacle detection information sent by the vehicle-mounted terminal; receiving a teaching instruction for the obstacle detection information; generating indication information according to the teaching instruction; sending the indication information to the vehicle-mounted terminal;
the third receiving module is used for receiving the indication information sent by the cloud server;
and the control module is used for determining a running path according to the indication information and controlling the vehicle to run based on the running path.
9. A vehicle, characterized in that the vehicle comprises a processor, a memory and a computer program stored on the memory and executable on the processor, which computer program, when executed by the processor, carries out the steps of the remote assisted driving method of any one of claims 1 to 5, or of claim 6.
10. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when being executed by a processor, carries out the steps of the remote driving assistance method according to one of claims 1 to 5, or according to claim 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210319167.9A CN114839967A (en) | 2022-03-29 | 2022-03-29 | Remote driving assisting method and device, vehicle and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210319167.9A CN114839967A (en) | 2022-03-29 | 2022-03-29 | Remote driving assisting method and device, vehicle and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114839967A true CN114839967A (en) | 2022-08-02 |
Family
ID=82564674
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210319167.9A Pending CN114839967A (en) | 2022-03-29 | 2022-03-29 | Remote driving assisting method and device, vehicle and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114839967A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109753069A (en) * | 2019-01-16 | 2019-05-14 | 北京百度网讯科技有限公司 | Control method for vehicle and device |
| CN110667783A (en) * | 2019-08-30 | 2020-01-10 | 安徽科微智能科技有限公司 | Unmanned boat auxiliary driving system and method thereof |
| CN110871790A (en) * | 2018-08-28 | 2020-03-10 | 现代自动车株式会社 | Apparatus for controlling parking of vehicle, system having the same, and method thereof |
| CN111045425A (en) * | 2019-12-05 | 2020-04-21 | 中国北方车辆研究所 | Auxiliary teleoperation driving method for ground unmanned vehicle |
| CN112124326A (en) * | 2020-09-25 | 2020-12-25 | 斑马网络技术有限公司 | Automatic driving method, device, electronic equipment and storage medium |
| CN113777975A (en) * | 2021-08-18 | 2021-12-10 | 浙江越影科技有限公司 | Remote auxiliary system and method for automatically driving vehicle |
-
2022
- 2022-03-29 CN CN202210319167.9A patent/CN114839967A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110871790A (en) * | 2018-08-28 | 2020-03-10 | 现代自动车株式会社 | Apparatus for controlling parking of vehicle, system having the same, and method thereof |
| CN109753069A (en) * | 2019-01-16 | 2019-05-14 | 北京百度网讯科技有限公司 | Control method for vehicle and device |
| CN110667783A (en) * | 2019-08-30 | 2020-01-10 | 安徽科微智能科技有限公司 | Unmanned boat auxiliary driving system and method thereof |
| CN111045425A (en) * | 2019-12-05 | 2020-04-21 | 中国北方车辆研究所 | Auxiliary teleoperation driving method for ground unmanned vehicle |
| CN112124326A (en) * | 2020-09-25 | 2020-12-25 | 斑马网络技术有限公司 | Automatic driving method, device, electronic equipment and storage medium |
| CN113777975A (en) * | 2021-08-18 | 2021-12-10 | 浙江越影科技有限公司 | Remote auxiliary system and method for automatically driving vehicle |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110775052B (en) | Automatic parking method based on fusion of vision and ultrasonic perception | |
| US10983529B2 (en) | Method and system for providing data for a first and second trajectory | |
| JP6283484B2 (en) | Display of information related to detected radar signals | |
| CN110415550B (en) | Automatic parking method based on vision | |
| CN112518739A (en) | Intelligent self-navigation method for reconnaissance of tracked chassis robot | |
| CN112394725B (en) | Prediction and reaction field of view based planning for autopilot | |
| CN112537294B (en) | Automatic parking control method and electronic equipment | |
| CN210139859U (en) | Automobile collision early warning system, navigation and automobile | |
| CN115235500B (en) | Lane line constraint-based pose correction method and device and all-condition static environment modeling method and device | |
| CN112445204B (en) | Object movement navigation method and device in construction site and computer equipment | |
| CN113706704B (en) | Method and equipment for planning route based on high-precision map and automatic driving vehicle | |
| JP2022172444A (en) | Method and assist device for assisting traveling operation of motor vehicle, and motor vehicle | |
| CN109883439A (en) | A kind of automobile navigation method, device, electronic equipment and storage medium | |
| CN113469045A (en) | Unmanned card-collecting visual positioning method and system, electronic equipment and storage medium | |
| JPWO2018070022A1 (en) | Self-position estimation method and self-position estimation apparatus | |
| CN113158779B (en) | Walking method, walking device and computer storage medium | |
| US12071159B2 (en) | Aerial view generation for vehicle control | |
| US20240034354A1 (en) | Vehicle Placement on Aerial Views for Vehicle Control | |
| CN116088533B (en) | Information determination method, remote terminal, device, mower and storage medium | |
| EP4250041A1 (en) | Method for determining information, remote terminal, and mower | |
| CN114839967A (en) | Remote driving assisting method and device, vehicle and storage medium | |
| CN110928277B (en) | Obstacle prompting method, device and equipment for intelligent road side unit | |
| CN112224197B (en) | Method and device for detecting parking space during reversing, vehicle and storage medium | |
| CN114663503A (en) | Three-dimensional position prediction from images | |
| JP2023031115A (en) | Remote driver support method, support system, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |