CN112433502A - Land leveler remote control system and method based on 5G communication and VR double-ring vision technology - Google Patents

Land leveler remote control system and method based on 5G communication and VR double-ring vision technology Download PDF

Info

Publication number
CN112433502A
CN112433502A CN202011485660.5A CN202011485660A CN112433502A CN 112433502 A CN112433502 A CN 112433502A CN 202011485660 A CN202011485660 A CN 202011485660A CN 112433502 A CN112433502 A CN 112433502A
Authority
CN
China
Prior art keywords
vehicle
data
console
working condition
remote control
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011485660.5A
Other languages
Chinese (zh)
Inventor
周磊
张启亮
郭辉
姜丽萍
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jiangsu Xugong Information Technology Co ltd
Original Assignee
Jiangsu Xugong Information Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jiangsu Xugong Information Technology Co ltd filed Critical Jiangsu Xugong Information Technology Co ltd
Priority to CN202011485660.5A priority Critical patent/CN112433502A/en
Publication of CN112433502A publication Critical patent/CN112433502A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B19/00Programme-control systems
    • G05B19/02Programme-control systems electric
    • G05B19/04Programme control other than numerical control, i.e. in sequence controllers or logic controllers
    • G05B19/042Programme control other than numerical control, i.e. in sequence controllers or logic controllers using digital processors
    • G05B19/0423Input/output
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00Program-control systems
    • G05B2219/20Pc systems
    • G05B2219/23Pc programming
    • G05B2219/23051Remote control, enter program remote, detachable programmer

Abstract

The invention discloses a land leveler remote control system and a method based on 5G communication and VR double-loop vision technology.A camera acquires image data and vehicle working condition data on a vehicle CAN bus, the image data and the vehicle working condition data are transmitted to an industrial personal computer of a console through a 5G network, the image data and the vehicle working condition data are finally displayed in a VR head display through a specific image processing algorithm, and meanwhile, the internal and external visual angles CAN be switched. The remote vehicle operation can be controlled by operating the handle. The seat tilts and shakes according to data collected by the field sensors. The invention effectively avoids the visual blind area caused by the common VR, and can display the panoramic image of the vehicle body and the working condition parameters of the vehicle in real time in the VR head display while viewing from the internal view angle of the cab; the display content in the VR head display can be switched through the handle keys; by utilizing 5G communication, the time delay of image transmission and working condition data transmission can be effectively reduced.

Description

Land leveler remote control system and method based on 5G communication and VR double-ring vision technology
Technical Field
The invention relates to a land leveler remote control system based on 5G communication and VR double-ring vision technology, belonging to the field of remote control engineering machinery.
Background
The operation environment of part of engineering machinery vehicles is severe, and operators need to endure jolt, high temperature, radiation and even life danger in field operation.
The delay of the remote control adopting 4G is large, and the vehicle cannot be controlled in real time. If the remote video data is displayed only by using a large screen, the user does not have an immersive control experience.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a grader remote control system based on 5G communication and VR double-loop vision technology, which effectively avoids visual blind areas caused by common VR, and can display the panoramic image of a vehicle body and vehicle working condition parameters in real time in VR head display while viewing from the internal view angle of a cab; the display content in the VR head display can be switched through the handle keys; by utilizing 5G communication, the time delay of image transmission and working condition data transmission can be effectively reduced.
The invention is realized according to the following technical scheme:
grader remote control system based on 5G communication and VR dual-ring look technique includes:
the camera transmits the acquired video signal to an industrial personal computer of the remote console through a 5G network;
the working condition data on the vehicle CAN bus is converted into network data through a vehicle-mounted gateway and then transmitted to an industrial personal computer of a remote control console through a 5G network;
the VR head display is used for displaying the video data and the working condition parameters after the video data and the working condition parameters are combined and processed by a data processing system on the industrial personal computer;
the vehicle end controller converts data acquired by the vehicle state detector into network data through the vehicle-mounted gateway, and transmits the network data to the industrial personal computer of the remote control console by using a 5G network and controls the seat to act;
and the vehicle controller is used for acquiring a vehicle controller signal by the console end controller, converting the vehicle controller signal into network data through the console vehicle-mounted gateway and transmitting the network data to the vehicle end controller so as to control the operation of the land leveler.
Further, the camera includes the 360 degrees around installing in camera and the driver's cabin around the motorgrader automobile body looks around the camera.
Further, the VR head display has the functions of inner circular viewing, outer circular viewing and vehicle working condition parameter display, the display content in the VR head display is switched through the handle keys, the inner circular viewing, the outer circular viewing and the working condition parameter display can be realized simultaneously, and the independent display can also be switched.
Further, the vehicle body state detector comprises a vibration sensor and an inclination angle sensor; and the vehicle end controller converts data acquired by the vibration sensor and the tilt angle sensor into network data through the vehicle-mounted gateway, and transmits the network data to an industrial personal computer of the remote control console by using a 5G network and controls the seat to incline and vibrate.
Further, the vehicle manipulator comprises a handle key and a foot pedal; the console end controller collects handle keys and pedal signals, converts the signals into network data through the console vehicle-mounted gateway and then transmits the network data to the vehicle end controller, so that the operation of the land leveler is controlled.
Furthermore, a vehicle-mounted 5G router is arranged in the land scraper, and a console 5G router is arranged on the console, so that a 5G network transmission route is realized.
A land leveler remote control method based on 5G communication and VR double-ring vision technology comprises the following steps:
1) establishing a communication link based on 5G by utilizing a 5G router and a vehicle-mounted gateway;
2) the camera collects video signals and sends the video signals to a data processing system of the console through a 5G network;
3) the working condition data on the vehicle CAN bus is converted into network data through the vehicle-mounted gateway, and the network data is sent to a data processing system of the console by using a 5G network;
4) the data processing system of the console combines and processes the image data and the vehicle working condition data, and displays the image data and the vehicle working condition data to a VR head for display;
5) the vibration and angle signals collected by the vehicle end controller are sent to a data processing system of the console through a vehicle-mounted gateway, and the data processing system controls the inclination and vibration of the seat;
6) the console end controller collects handle keys and pedal signals, and transmits the signals to the vehicle end controller through the vehicle-mounted gateway, so as to control the movement and work of the grader.
Furthermore, the VR head display has the functions of inner circular vision, outer circular vision and vehicle working condition parameter display, namely the inner circular vision, the outer circular vision and the working condition parameters can be displayed simultaneously and can be switched to be displayed independently, the inner circular vision realizes 360-degree circular vision by rotating the head from the angle of a driver, and the outer circular vision displays 360-degree panoramic images of the vehicle; a data processing system on the industrial personal computer splices external video images of the vehicle body into a 360-degree panoramic image, then the current display mode of the VR head display is set according to the handle, the panoramic image of the vehicle body, the internal panoramic video images and the working condition parameters are merged and processed, and the images are displayed in the VR head display.
The invention has the beneficial effects that:
the invention realizes remote control by utilizing the VR double-ring vision function, not only can display the dynamic image of the angle of the driver in real time on the head display by utilizing the VR inner ring vision function, but also can display the panoramic image of the whole grader in the VR head display, thereby effectively solving the operation risk caused by the fact that the peripheral road conditions of the grader body cannot be known at any time by the common VR function because the grader body is longer. The vibration and the inclination of the seat are controlled by matching with the data of the vibration sensor and the inclination angle sensor, so that an on-the-spot control experience is brought; meanwhile, the 5G network is utilized to reduce communication delay.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention, are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the invention without limiting the invention to the right. It is obvious that the drawings in the following description are only some embodiments, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.
In the drawings:
FIG. 1 is a schematic flow chart of the present invention;
fig. 2 is a block diagram of the system of the present invention.
It should be noted that the drawings and the description are not intended to limit the scope of the inventive concept in any way, but to illustrate it by a person skilled in the art with reference to specific embodiments.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and the following embodiments are used for illustrating the present invention and are not intended to limit the scope of the present invention.
In the description of the present invention, it should be noted that the terms "upper", "lower", "front", "rear", "left", "right", "vertical", "inner", "outer", etc., indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, and are only for convenience of description and simplicity of description, but do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be construed as limiting the present invention.
In the description of the present invention, it should be noted that, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; may be directly connected or indirectly connected through an intermediate. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
As shown in fig. 2, a grader remote control system based on 5G communication and VR dual-loop vision technology includes a camera, a VR head display, a vehicle body state detector and a vehicle controller; the camera transmits the collected video signal to an industrial personal computer of a remote console through a 5G network; the working condition data on the CAN bus of the vehicle is converted into network data through a vehicle-mounted gateway and then transmitted to an industrial personal computer of a remote control console through a 5G network; a data processing system on the industrial personal computer combines and processes the video data and the working condition parameters and then displays the video data and the working condition parameters in a VR head display; the vehicle end controller converts data acquired by the vehicle body state detector into network data through the vehicle-mounted gateway, and then transmits the network data to an industrial personal computer of the remote control console by using a 5G network and controls the seat to act; the console end controller collects vehicle controller signals, converts the vehicle controller signals into network data through the console vehicle-mounted gateway and then transmits the network data to the vehicle end controller, and therefore the operation of the land leveler is controlled.
The optimization method of the invention comprises the following steps: the camera is including installing the camera around the leveler automobile body and the 360 degrees around looking at the camera in the driver's cabin.
The optimization method of the invention comprises the following steps: the VR head display comprises functions of inner circular viewing, outer circular viewing and vehicle working condition parameter display, and can simultaneously display the inner circular viewing, the outer circular viewing and the working condition parameters and also switch to display independently. A data processing system on the industrial personal computer splices external video images of the vehicle body into a 360-degree panoramic image, then the current display mode of the VR head display is set according to the handle, the panoramic image of the vehicle body, the internal panoramic video images and the working condition parameters are merged and processed, and the images are displayed in the VR head display.
The specific preferred scheme is as follows: the vehicle body state detector comprises a vibration sensor and an inclination angle sensor; and the vehicle end controller converts data acquired by the vibration sensor and the tilt angle sensor into network data through the vehicle-mounted gateway, and transmits the network data to an industrial personal computer of the remote control console by using a 5G network and controls the seat to incline and vibrate.
The specific preferred scheme is as follows: the vehicle controller comprises a handle key and a pedal; the console end controller collects handle keys and pedal signals, converts the signals into network data through the console vehicle-mounted gateway and then transmits the network data to the vehicle end controller, so that the operation of the land leveler is controlled.
The specific preferred scheme is as follows: the land leveler is provided with a vehicle-mounted 5G router, and the console is provided with a console 5G router, so that a 5G network transmission route is realized.
As shown in fig. 1, a remote control method of a grader based on 5G communication and VR dual-ring vision technology includes the following steps:
1) establishing a communication link based on 5G by utilizing a 5G router and a vehicle-mounted gateway;
2) the camera collects video signals and sends the video signals to a data processing system of the console through a 5G network;
3) the working condition data on the vehicle CAN bus is converted into network data through the vehicle-mounted gateway, and the network data is sent to a data processing system of the console by using a 5G network;
4) the data processing system of the console combines and processes the image data and the vehicle working condition data, and displays the image data and the vehicle working condition data to a VR head for display;
5) the vibration and angle signals collected by the vehicle end controller are sent to a data processing system of the console through a vehicle-mounted gateway, and the data processing system controls the inclination and vibration of the seat;
6) the console end controller collects handle keys and pedal signals, and transmits the signals to the vehicle end controller through the vehicle-mounted gateway, so as to control the movement and work of the grader.
It should be noted that the VR head display includes functions of inner circular viewing, outer circular viewing and vehicle working condition parameter display, that is, the inner circular viewing, the outer circular viewing and the working condition parameter can be displayed simultaneously, and can be switched to be displayed separately, the inner circular viewing realizes 360-degree circular viewing by rotating the head from the angle of the driver, and the outer circular viewing displays 360-degree panoramic images of the vehicle; a data processing system on the industrial personal computer splices external video images of the vehicle body into a 360-degree panoramic image, then the current display mode of the VR head display is set according to the handle, the panoramic image of the vehicle body, the internal panoramic video images and the working condition parameters are merged and processed, and the images are displayed in the VR head display.
In conclusion, the VR double-ring vision technology is adopted, so that the visual blind area during the operation of the land leveler is reduced, and the operation reliability is improved; the real-time display of the vehicle working condition data in the VR head display is used for timely knowing the current vehicle condition of the grader and further assisting the driving operation; utilize 5G transmission data greatly to reduce the time delay, adopt the simulated action of seat to bring immersive experience simultaneously.
In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than others, combinations of features of different embodiments are also meant to be within the scope of the invention and form different embodiments. For example, in the above embodiments, those skilled in the art can use the combination according to the known technical solutions and technical problems to be solved by the present application.
Although the present invention has been described with reference to a preferred embodiment, it should be understood that various changes, substitutions and alterations can be made herein without departing from the spirit and scope of the invention as defined by the appended claims.

Claims (8)

1. Grader remote control system based on 5G communication and VR dual-ring look technique, its characterized in that includes:
the camera transmits the acquired video signal to an industrial personal computer of the remote console through a 5G network;
the working condition data on the vehicle CAN bus is converted into network data through a vehicle-mounted gateway and then transmitted to an industrial personal computer of a remote control console through a 5G network;
the VR head display is used for displaying the video data and the working condition parameters after the video data and the working condition parameters are combined and processed by a data processing system on the industrial personal computer;
the vehicle end controller converts data acquired by the vehicle state detector into network data through the vehicle-mounted gateway, and transmits the network data to the industrial personal computer of the remote control console by using a 5G network and controls the seat to act;
and the vehicle controller is used for acquiring a vehicle controller signal by the console end controller, converting the vehicle controller signal into network data through the console vehicle-mounted gateway and transmitting the network data to the vehicle end controller so as to control the operation of the land leveler.
2. The remote control system of a grader based on 5G communication and VR dual-loop vision technology of claim 1, wherein:
the camera is including installing the camera around the leveler automobile body and the 360 degrees around looking at the camera in the driver's cabin.
3. The remote control system of a grader based on 5G communication and VR dual-loop vision technology of claim 1, wherein:
the VR head display comprises an inner ring view function, an outer ring view function and a vehicle working condition parameter display function, the display content in the VR head display is switched through the handle keys, the inner ring view function, the outer ring view function and the working condition parameter display function can be simultaneously displayed, and the independent display function can be switched.
4. The remote control system of a grader based on 5G communication and VR dual-loop vision technology of claim 1, wherein:
the vehicle body state detector comprises a vibration sensor and an inclination angle sensor;
and the vehicle end controller converts data acquired by the vibration sensor and the tilt angle sensor into network data through the vehicle-mounted gateway, and transmits the network data to an industrial personal computer of the remote control console by using a 5G network and controls the seat to incline and vibrate.
5. The remote control system of a grader based on 5G communication and VR dual-loop vision technology of claim 1, wherein:
the vehicle controller comprises a handle key and a pedal;
the console end controller collects handle keys and pedal signals, converts the signals into network data through the console vehicle-mounted gateway and then transmits the network data to the vehicle end controller, so that the operation of the land leveler is controlled.
6. The remote control system of a grader based on 5G communication and VR dual-loop vision technology of claim 1, wherein:
the land leveler is provided with a vehicle-mounted 5G router, and the console is provided with a console 5G router, so that a 5G network transmission route is realized.
7. A land leveler remote control method based on 5G communication and VR double-ring vision technology is characterized by comprising the following steps:
1) establishing a communication link based on 5G by utilizing a 5G router and a vehicle-mounted gateway;
2) the camera collects video signals and sends the video signals to a data processing system of the console through a 5G network;
3) the working condition data on the vehicle CAN bus is converted into network data through the vehicle-mounted gateway, and the network data is sent to a data processing system of the console by using a 5G network;
4) the data processing system of the console combines and processes the image data and the vehicle working condition data, and displays the image data and the vehicle working condition data to a VR head for display;
5) the vibration and angle signals collected by the vehicle end controller are sent to a data processing system of the console through a vehicle-mounted gateway, and the data processing system controls the inclination and vibration of the seat;
6) the console end controller collects handle keys and pedal signals, and transmits the signals to the vehicle end controller through the vehicle-mounted gateway, so as to control the movement and work of the grader.
8. The remote control method of a grader based on 5G communication and VR dual-ring vision technology of claim 7, wherein the remote control method comprises the following steps:
the VR head display has the functions of inner circular vision, outer circular vision and vehicle working condition parameter display, namely the inner circular vision, the outer circular vision and the working condition parameters can be displayed simultaneously and can be switched to be displayed independently, the inner circular vision realizes 360-degree circular vision by rotating the head from the angle of a driver, and the outer circular vision displays 360-degree panoramic images of the vehicle; a data processing system on the industrial personal computer splices external video images of the vehicle body into a 360-degree panoramic image, then the current display mode of the VR head display is set according to the handle, the panoramic image of the vehicle body, the internal panoramic video images and the working condition parameters are merged and processed, and the images are displayed in the VR head display.
CN202011485660.5A 2020-12-16 2020-12-16 Land leveler remote control system and method based on 5G communication and VR double-ring vision technology Pending CN112433502A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011485660.5A CN112433502A (en) 2020-12-16 2020-12-16 Land leveler remote control system and method based on 5G communication and VR double-ring vision technology

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011485660.5A CN112433502A (en) 2020-12-16 2020-12-16 Land leveler remote control system and method based on 5G communication and VR double-ring vision technology

Publications (1)

Publication Number Publication Date
CN112433502A true CN112433502A (en) 2021-03-02

Family

ID=74691239

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011485660.5A Pending CN112433502A (en) 2020-12-16 2020-12-16 Land leveler remote control system and method based on 5G communication and VR double-ring vision technology

Country Status (1)

Country Link
CN (1) CN112433502A (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106422322A (en) * 2016-09-21 2017-02-22 苏州哈工众志自动化科技有限公司 VR (virtual reality) helmet and dynamic seat integrated 3D (three-dimensional) visual driving platform
CN110191307A (en) * 2018-02-23 2019-08-30 罗罗艺术计划株式会社 Pass through the miniature land Navigating System of the manipulation of wireless remotecontrol vehicle and real-time imaging transmission
CN110750153A (en) * 2019-09-11 2020-02-04 杭州博信智联科技有限公司 Dynamic virtualization device of unmanned vehicle
CN110805079A (en) * 2019-11-15 2020-02-18 徐州康博智能控制仪表有限公司 Excavator remote operation system based on VR technology
CN110853385A (en) * 2018-08-21 2020-02-28 上海擎感智能科技有限公司 Road condition video sharing method and system based on 5G technology, and first vehicle and second vehicle
CN111152820A (en) * 2020-04-03 2020-05-15 北京全路通信信号研究设计院集团有限公司 Multi-source data integrated remote train driving method and system

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106422322A (en) * 2016-09-21 2017-02-22 苏州哈工众志自动化科技有限公司 VR (virtual reality) helmet and dynamic seat integrated 3D (three-dimensional) visual driving platform
CN110191307A (en) * 2018-02-23 2019-08-30 罗罗艺术计划株式会社 Pass through the miniature land Navigating System of the manipulation of wireless remotecontrol vehicle and real-time imaging transmission
CN110853385A (en) * 2018-08-21 2020-02-28 上海擎感智能科技有限公司 Road condition video sharing method and system based on 5G technology, and first vehicle and second vehicle
CN110750153A (en) * 2019-09-11 2020-02-04 杭州博信智联科技有限公司 Dynamic virtualization device of unmanned vehicle
CN110805079A (en) * 2019-11-15 2020-02-18 徐州康博智能控制仪表有限公司 Excavator remote operation system based on VR technology
CN111152820A (en) * 2020-04-03 2020-05-15 北京全路通信信号研究设计院集团有限公司 Multi-source data integrated remote train driving method and system

Similar Documents

Publication Publication Date Title
JP7252137B2 (en) Perimeter monitoring device
JP6427597B2 (en) Peripheral monitoring device for working machine and peripheral monitoring method for working machine
EP2717570A1 (en) Device for monitoring area around working machine
US9776566B2 (en) Environment monitoring device for operating machinery
EP1298001B1 (en) Method and apparatus for monitoring vehicle rear
WO2016013686A1 (en) Display system for work machine, display device for work machine, and display method for work machine
JPWO2012157379A1 (en) Work machine ambient monitoring device
JP6071786B2 (en) Work machine ambient monitoring device
JP6577034B2 (en) Work machine display system and work machine display method
WO2021176883A1 (en) Work support server and work support method
EP4084470A1 (en) Remote operation assistance server, remote operation assistance system, and remote operation assistance method
CN112433502A (en) Land leveler remote control system and method based on 5G communication and VR double-ring vision technology
WO2022176586A1 (en) Surroundings monitoring system for work machine, work machine, and surroundings monitoring method for work machine
WO2021230093A1 (en) Image processing device and image processing method
US20220412049A1 (en) Work assisting server and method for selecting imaging device
JP2023040971A (en) Remote control support device, remote control support system, and remote control support method
JP2010070929A (en) Monitor device for confirming rear side
WO2022195988A1 (en) Remote operation assistance server and remote operation assistance system
US20240141616A1 (en) Remote operation assistance server and remote operation asistance system
US11939744B2 (en) Display system, remote operation system, and display method
WO2021181916A1 (en) Work assistance server and work assistance method
CN220067522U (en) Remote monitoring system
CN113359691B (en) Remote image display device and method of non-road mobile equipment
JP2023103876A (en) Remote operation support system
JP6188858B2 (en) Work machine ambient monitoring device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20210302

RJ01 Rejection of invention patent application after publication