CN114839915A - Method and system for remotely controlling vehicle - Google Patents
Method and system for remotely controlling vehicle Download PDFInfo
- Publication number
- CN114839915A CN114839915A CN202210630304.0A CN202210630304A CN114839915A CN 114839915 A CN114839915 A CN 114839915A CN 202210630304 A CN202210630304 A CN 202210630304A CN 114839915 A CN114839915 A CN 114839915A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- remote
- driver
- environment information
- delta
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B19/00—Programme-control systems
- G05B19/02—Programme-control systems electric
- G05B19/04—Programme control other than numerical control, i.e. in sequence controllers or logic controllers
- G05B19/042—Programme control other than numerical control, i.e. in sequence controllers or logic controllers using digital processors
- G05B19/0423—Input/output
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/20—Pc systems
- G05B2219/23—Pc programming
- G05B2219/23051—Remote control, enter program remote, detachable programmer
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Automation & Control Theory (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
The invention discloses a method and a system for remotely controlling a vehicle, belonging to the technical field of vehicles, wherein detected vehicle surrounding environment information is stored in the vehicle, vehicle cabin environment is stored in the vehicle in advance, and the vehicle surrounding environment information and the vehicle cabin environment information are sent to a remote signal receiving device; projecting the vehicle surrounding environment information and the vehicle cabin environment information to the periphery of a remote driver by using a remote signal receiving device projection technology so that the remote driver can carry out remote control according to a virtual driving environment and a cabin environment; after system authentication and remote control permission, remote driving operation is carried out according to the surrounding environment information of the vehicle and the environment information of the vehicle cabin, the driving intention of a remote driver is monitored, and a control instruction is sent to the vehicle by the detected driving intention. According to the invention, through the virtual driving environment, a remote control person can perform complex operations such as gear shifting, braking, an accelerator, a steering lamp and the like in a relatively real environment.
Description
Technical Field
The invention belongs to the technical field of vehicles, and particularly relates to a method and a system for remotely controlling a vehicle.
Background
Vehicle remote control mainly relies on mobile device to carry out short distance simple operation, if advance, retreat, and the visual field of driving and the operation mode can't normally be simulated to most vehicles, causes remote control personnel can't control vehicle operation accurately completely, takes place the accident.
At present, the vehicle is simply moved forward and backward and the steering lamp is operated mainly through Bluetooth or cellular signals of a mobile phone, and a more real driving state cannot be simulated.
Disclosure of Invention
Aiming at the defects or the improvement requirements of the prior art, the invention provides a method and a system for remotely controlling a vehicle, which enable remote control personnel to perform complex operations such as gear shifting, braking, an accelerator, a steering lamp and the like in a relatively real environment through a virtual driving environment.
To achieve the above object, according to one aspect of the present invention, there is provided a remote control vehicle method including:
storing the detected vehicle surrounding environment information in the vehicle, storing the vehicle cabin environment in the vehicle in advance, and sending the vehicle surrounding environment information and the vehicle cabin environment information to a remote signal receiving device;
after the position of the remote signal receiving device is fixed, the remote signal receiving device projection technology is used for projecting the vehicle surrounding environment information and the vehicle cabin environment information to the periphery of a remote driver so that the remote driver can carry out remote control according to the virtual driving environment and the cabin environment;
after system authentication and remote control permission, remote driving operation is carried out according to the surrounding environment information of the vehicle and the environment information of the vehicle cabin, the driving intention of a remote driver is monitored, and a control instruction is sent to the vehicle by the detected driving intention so as to control the vehicle.
In some alternative embodiments, monitoring the driving intent of the remote driver includes:
and monitoring the position change of the remote driver holding the steering wheel to determine the left steering intention and the right steering intention of the remote driver.
In some alternative embodiments, monitoring a change in a position of a hand-held steering wheel of the remote driver to determine a left-steering intent of the remote driver, comprises:
detecting that a right-hand feature is on the right side at the time t0, predicting frame position center coordinates to be (x1, y1), detecting that the right-hand feature is above a steering wheel at the time t0 +. DELTA.t, predicting frame position center coordinates to be (x2, y2), and calculating hand movement speeds to be (x2-x 1)/. DELTA.t) and Vy ((y2-y 1)/. DELTA.t), wherein the prediction frame is used for tracking hand movement;
and if Vx is larger than or equal to the first preset value or Vy is larger than or equal to the second preset value, judging that the driver is performing left steering operation, and if the misoperation condition is not met, judging that the driver is performing left steering operation.
In some alternative embodiments, monitoring a change in a position of a hand-held steering wheel of the remote driver to determine a right-steering intent of the remote driver, comprises:
detecting that the left-hand feature is on the left side at time t0 ', predicting the center coordinate of the frame position to be (x 1', y1 '), detecting that the left-hand feature is above the steering wheel at time t 0' +. DELTA.t, predicting the center coordinate of the frame position to be (x2 ', y 2'), and calculating the hand movement speed to be Vx '((x 2' -x1 ')/. DELTA.t), Vy' ((y2 '-y 1')/. DELTA.t);
and if Vx 'is not less than the first preset value or Vy' is not less than the second preset value, judging that the driver is performing right steering operation.
In some optional embodiments, after sending the detected driving intent to the vehicle control instructions to control the vehicle, the method further comprises:
and receiving the operation image sent by the vehicle, and updating the projected virtual driving environment again.
According to another aspect of the present invention, there is provided a remote control vehicle system including:
the virtual driving environment generating module is used for storing the detected surrounding environment information of the vehicle in the vehicle and storing the cabin environment of the vehicle in advance;
the wireless communication module is used for sending the environmental information around the vehicle and the environmental information of the vehicle cabin to the remote signal receiving device;
the remote signal receiving device is used for projecting the vehicle surrounding environment information and the vehicle cabin environment information to the periphery of a remote driver by using a projection technology after the position of the remote signal receiving device is fixed, so that the remote driver can remotely control according to a virtual driving environment and a cabin environment;
and the remote signal receiving device is also used for carrying out remote driving operation according to the ambient environment information of the vehicle and the ambient environment information of the vehicle cabin after system authentication and remote control permission, monitoring the driving intention of a remote driver and sending a control instruction to the vehicle by using the detected driving intention so as to control the vehicle.
In some alternative embodiments, the remote signal receiving device is configured to monitor a change in a position of a hand-held steering wheel of the remote driver to determine a left steering intention and a right steering intention of the remote driver.
In some alternative embodiments, the remote signal receiving device is configured to detect that the right-hand feature is on the right side at time t0, predict frame position center coordinates as (x1, y1), detect that the right-hand feature is above the steering wheel at time t0 +. DELTA.t, predict frame position center coordinates as (x2, y2), calculate hand movement speed as Vx ((x2-x 1)/. DELTA.t), and Vy ═ ((y2-y 1)/. DELTA.t), wherein the prediction frame is configured to track hand movement; and if Vx is larger than or equal to the first preset value or Vy is larger than or equal to the second preset value, judging that the driver is performing left steering operation, and if the misoperation condition is not met, judging that the driver is performing left steering operation.
In some alternative embodiments, the remote signal receiving means is arranged to detect that the left-handed feature is to the left at time t0 ', predict the frame position centre coordinates as (x 1', y1 '), detect that the left-handed feature is above the steering wheel at time t 0' +. DELTA.t, predict the frame position centre coordinates as (x2 ', y 2'), calculate the hand movement speed as Vx '((x 2' -x1 ')/. DELTA.t), Vy' ((y2 '-y 1')/. DELTA.t); and if Vx 'is not less than the first preset value or Vy' is not less than the second preset value, judging that the driver is performing right steering operation.
In some optional embodiments, the remote signal receiving device is further configured to receive the operation image transmitted by the vehicle and update the projected virtual driving environment again after the detected driving intention is transmitted to the vehicle to transmit a control instruction to control the vehicle.
In general, compared with the prior art, the above technical solution contemplated by the present invention can achieve the following beneficial effects:
the invention can simulate real driving environment by simulating the driving operation mode to carry out vehicle remote control, can feed back the vehicle running state in time, improves the operation convenience and safety, and enables a driver to control the vehicle to move in a more accurate and safe mode.
Drawings
FIG. 1 is a schematic flow chart diagram of a method for remotely controlling a vehicle according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a remotely controlled vehicle system provided by an embodiment of the present invention;
fig. 3 is a schematic diagram of an operation of a remote controller according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
In the present examples, "first", "second", etc. are used for distinguishing different objects, and are not used for describing a specific order or sequence.
In the embodiment of the invention, the system for realizing remote control of the vehicle comprises two parts, namely a virtual driving environment and a remote control function.
The virtual driving environment mainly identifies the surrounding environment in real time through the vehicle-mounted camera, and meanwhile, the cabin environment stored in the vehicle in advance is overlapped with the surrounding environment to jointly construct the virtual driving environment. As shown in fig. 3, the cabin environment includes, but is not limited to, a brake pedal, an accelerator pedal, a gear shift mechanism, a steering wheel, a handbrake, a door, etc. for controlling the vehicle. As shown in fig. 2, the virtual driving environment is transmitted to the remote control personnel through a wireless communication technology, and the remote control personnel should have an electronic device for receiving, capable of performing communication receiving, and capable of projecting.
The remote control function is that the remote control personnel utilize the image of receiving electronic device projection in the environment, the virtual driving vehicle, for example operate the gear shift, step on the throttle, step on the brake pedal etc. receiving electronic device simultaneously has an image recognition system, can discern the operation of remote control personnel, then sends the control command to the vehicle through the communication form and controls.
As shown in fig. 1, in the present example, the remote control of the vehicle can be achieved by:
step 1: monitoring the surrounding environment of the vehicle: sensor devices such as vehicle peripheral cameras are started to detect the environment, and videos are converted into digital signals to be stored in the vehicle;
step 2: vehicle cabin environmental simulation: the method comprises the steps that the cabin environment of the vehicle is stored in the vehicle in advance, the cabin environment comprises devices capable of controlling the motion of the vehicle, such as a brake, an accelerator, gear shifting, steering and the like, and other auxiliary devices, such as a vehicle door, light and other image information, and the cabin environment simulation can be zoomed and changed according to the habit of a driver;
and step 3: vehicle signal transmission: sending the image or video signals of the step 1 and the step 2 to a signal receiving device of a remote driver through a wireless radio frequency signal, such as the receiving device in fig. 2;
and 4, step 4: adjusting the posture of the driver: the remote driver fixes the signal receiving device at a proper position, then uses the receiving device projection technology to project the video signal received in the step 3 to the periphery of the driver, and the driver adjusts the cabin environment according to the virtual driving environment, simulates the driving operation posture, such as the driving posture in fig. 2, and prepares to control the vehicle;
and 5: remote driving operation: after system authentication and remote control permission, a driver simulates and operates a steering wheel, an accelerator, gear shifting and the like according to a cockpit image transmitted by a vehicle;
step 6: detecting the driving intention: the receiving device should be capable of performing driver monitoring, such as the monitoring device in fig. 2, such as fig. 3, when the driver operates the steering wheel to move from point a to point B, the driver monitoring system detects that the position of the steering wheel held by the driver is changed, detects the hand feature, when the right hand feature is on the right side at the time t0, the center coordinates of the position of the bounding box of the prediction frame are (x1, y1), when the device detects that the right hand feature is above the steering wheel at the time t0 +. DELTA.t, the center coordinates of the position of the bounding box of the prediction frame are (x2, y2), and calculates the moving speed of the hand as: the driver gesture control method comprises the following steps that Vx ═ ((x2-x 1)/. DELTA.t), Vy ═ ((y2-y 1)/. DELTA.t), and considering that excessive speed is neglected due to possible actual misoperation, when Vx ═ ((x2-x 1)/. DELTA.t) > is larger than or equal to a first preset value (such as 0.5m/s) or Vy ≧ Vy (y2-y 1)/. DELTA.t) > is larger than or equal to a second preset value (such as 0.5m/s), misoperation is judged, when the system does not meet the conditions, the driver is judged to be in left-turn operation according to the graph 3, and the monitoring device continues to convert the judging logic into a driver operation intention control signal at the moment, and can be a camera or other devices capable of identifying the change of the posture of the driver;
detecting that the left-hand feature is on the left side at time t0 ', predicting the center coordinate of the frame position to be (x 1', y1 '), detecting that the left-hand feature is above the steering wheel at time t 0' +. DELTA.t, predicting the center coordinate of the frame position to be (x2 ', y 2'), and calculating the hand movement speed to be Vx '((x 2' -x1 ')/. DELTA.t), Vy' ((y2 '-y 1')/. DELTA.t);
and if Vx 'is not less than the first preset value or Vy' is not less than the second preset value, judging that the driver is performing right steering operation.
And 7: vehicle control: and (3) the receiving device sends the operation intention of the driver in the step (6) to the vehicle, on one hand, the vehicle works according to the operation intention, such as steering and braking, on the other hand, the vehicle transmits an operation image, such as a cabin image with a steering wheel rotating change, back to the receiving device, and the receiving device renews the projected virtual driving environment, so that the driver can observe the state of the vehicle in real time conveniently.
It should be noted that, according to the implementation requirement, each step/component described in the present application can be divided into more steps/components, and two or more steps/components or partial operations of the steps/components can be combined into new steps/components to achieve the purpose of the present invention.
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (10)
1. A method of remotely controlling a vehicle, comprising:
storing the detected vehicle surrounding environment information in the vehicle, storing the vehicle cabin environment in the vehicle in advance, and sending the vehicle surrounding environment information and the vehicle cabin environment information to a remote signal receiving device;
after the position of the remote signal receiving device is fixed, the remote signal receiving device projection technology is used for projecting the vehicle surrounding environment information and the vehicle cabin environment information to the periphery of a remote driver so that the remote driver can carry out remote control according to the virtual driving environment and the cabin environment;
after system authentication and remote control permission, remote driving operation is carried out according to the surrounding environment information of the vehicle and the environment information of the vehicle cabin, the driving intention of a remote driver is monitored, and a control instruction is sent to the vehicle by the detected driving intention so as to control the vehicle.
2. The method of claim 1, wherein monitoring the driving intent of the remote driver comprises:
and monitoring the position change of the remote driver holding the steering wheel to determine the left steering intention and the right steering intention of the remote driver.
3. The method of claim 2, wherein monitoring changes in the position of the remote driver's hand gripping the steering wheel to determine the remote driver's left steering intent comprises:
detecting that a right-hand feature is on the right side at the time t0, predicting frame position center coordinates to be (x1, y1), detecting that the right-hand feature is above a steering wheel at the time t0 +. DELTA.t, predicting frame position center coordinates to be (x2, y2), and calculating hand movement speeds to be (x2-x 1)/. DELTA.t) and Vy ((y2-y 1)/. DELTA.t), wherein the prediction frame is used for tracking hand movement;
and if Vx is larger than or equal to the first preset value or Vy is larger than or equal to the second preset value, judging that the driver is performing left steering operation, and if the misoperation condition is not met, judging that the driver is performing left steering operation.
4. The method of claim 2, wherein monitoring changes in the position of the remote driver's hand gripping the steering wheel to determine the remote driver's intent to turn right comprises:
detecting that the left-hand feature is on the left side at time t0 ', predicting the center coordinate of the frame position to be (x 1', y1 '), detecting that the left-hand feature is above the steering wheel at time t 0' +. DELTA.t, predicting the center coordinate of the frame position to be (x2 ', y 2'), and calculating the hand movement speed to be Vx '((x 2' -x1 ')/. DELTA.t), Vy' ((y2 '-y 1')/. DELTA.t);
and if Vx 'is not less than the first preset value or Vy' is not less than the second preset value, judging that the driver is performing right steering operation.
5. The method according to claim 3 or 4, wherein after sending the detected driving intention to the vehicle a control instruction to control the vehicle, the method further comprises:
and receiving the operation image sent by the vehicle, and updating the projected virtual driving environment again.
6. A remote controlled vehicle system, comprising:
the virtual driving environment generating module is used for storing the detected surrounding environment information of the vehicle in the vehicle and storing the cabin environment of the vehicle in advance;
the wireless communication module is used for sending the environmental information around the vehicle and the environmental information of the vehicle cabin to the remote signal receiving device;
the remote signal receiving device is used for projecting the vehicle surrounding environment information and the vehicle cabin environment information to the periphery of a remote driver by using a projection technology after the position of the remote signal receiving device is fixed, so that the remote driver can remotely control according to a virtual driving environment and a cabin environment;
and the remote signal receiving device is also used for carrying out remote driving operation according to the ambient environment information of the vehicle and the ambient environment information of the vehicle cabin after system authentication and remote control permission, monitoring the driving intention of a remote driver and sending a control instruction to the vehicle by using the detected driving intention so as to control the vehicle.
7. The system of claim 6, wherein the remote signal receiving device is configured to monitor changes in the position of the remote driver's hand gripping the steering wheel to determine the remote driver's left and right steering intent.
8. The system of claim 7, wherein the remote signal receiving means is configured to detect that the right-hand feature is on the right side at time t0, the predicted frame position center coordinates are (x1, y1), the right-hand feature is detected above the steering wheel at time t0 +. DELTA.t, the predicted frame position center coordinates are (x2, y2), and the hand movement speed is calculated as Vx ((x2-x 1)/. DELTA.t), and Vy ((y2-y 1)/. DELTA.t), wherein the predicted frame is configured to track hand movement; and if Vx is larger than or equal to the first preset value or Vy is larger than or equal to the second preset value, judging that the driver is performing left steering operation, and if the misoperation condition is not met, judging that the driver is performing left steering operation.
9. The system of claim 7, wherein the remote signal receiving means is configured to detect that the left-handed feature is on the left at time t0 ', the predicted frame position center coordinates are (x 1', y1 '), the left-handed feature is detected above the steering wheel at time t 0' +. DELTA.t, the predicted frame position center coordinates are (x2 ', y 2'), and hand movement speeds are calculated as Vx '((x 2' -x1 ')/. DELTA.t), Vy' ((y2 '-y 1')/. DELTA.t); and if Vx 'is not less than the first preset value or Vy' is not less than the second preset value, judging that the driver is performing right steering operation, and if the wrong operation condition is not met, judging that the driver is performing right steering operation.
10. The system of claim 8 or 9, wherein the remote signal receiving device is further configured to receive the operation image transmitted by the vehicle and update the projected virtual driving environment again after the detected driving intention is transmitted to the vehicle to control the vehicle.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210630304.0A CN114839915A (en) | 2022-06-06 | 2022-06-06 | Method and system for remotely controlling vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210630304.0A CN114839915A (en) | 2022-06-06 | 2022-06-06 | Method and system for remotely controlling vehicle |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114839915A true CN114839915A (en) | 2022-08-02 |
Family
ID=82574752
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210630304.0A Pending CN114839915A (en) | 2022-06-06 | 2022-06-06 | Method and system for remotely controlling vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114839915A (en) |
-
2022
- 2022-06-06 CN CN202210630304.0A patent/CN114839915A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109421742A (en) | Method and apparatus for monitoring autonomous vehicle | |
| US10773715B2 (en) | Parking control method and parking control device | |
| CN110678372B (en) | Vehicle control device | |
| JPWO2018066560A1 (en) | Vehicle control device | |
| CN109552308B (en) | Driving assistance apparatus and method for vehicle | |
| CN110171421B (en) | Vehicle control device | |
| US11548443B2 (en) | Display system, display method, and program for indicating a peripheral situation of a vehicle | |
| CN112046503A (en) | Vehicle control method based on artificial intelligence, related device and storage medium | |
| US11897505B2 (en) | In-vehicle operation of simulation scenarios during autonomous vehicle runs | |
| US11898850B2 (en) | Vehicle position detection device and parameter set creation device for vehicle position detection | |
| JP2019156058A (en) | Travel control system of vehicle | |
| KR20180022473A (en) | Remote controll device and remote controll method for controlling autonomous vehicle | |
| US20220058314A1 (en) | Hardware In Loop Testing and Generation of Latency Profiles for Use in Simulation | |
| CN110371018A (en) | Improve vehicle behavior using the information of other vehicle car lights | |
| US11429107B2 (en) | Play-forward planning and control system for an autonomous vehicle | |
| CN110799402B (en) | Vehicle control device | |
| US10699596B2 (en) | Vehicle operator training system | |
| US20220297599A1 (en) | Driving support device, driving support method, and storage medium | |
| US11204606B2 (en) | Vehicle control device | |
| KR20210002323A (en) | A vehicle-platoons implementation under autonomous driving system designed for single vehicle | |
| CN114839915A (en) | Method and system for remotely controlling vehicle | |
| CN109144070A (en) | Mobile device assists automatic Pilot method, automobile and storage medium | |
| US20200255016A1 (en) | Vehicle control device, vehicle, and vehicle control method | |
| KR101100646B1 (en) | Car operating robot, remote car controlling apparatus, system and method | |
| JP2022152607A (en) | Driving support device, driving support method, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |