CN113900431A - Remote control information processing method and device, electronic equipment and automatic driving vehicle - Google Patents
Remote control information processing method and device, electronic equipment and automatic driving vehicle Download PDFInfo
- Publication number
- CN113900431A CN113900431A CN202111165106.3A CN202111165106A CN113900431A CN 113900431 A CN113900431 A CN 113900431A CN 202111165106 A CN202111165106 A CN 202111165106A CN 113900431 A CN113900431 A CN 113900431A
- Authority
- CN
- China
- Prior art keywords
- information
- time
- vehicle
- remote control
- vehicle end
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000010365 information processing Effects 0.000 title claims abstract description 33
- 238000003672 processing method Methods 0.000 title claims abstract description 20
- 238000000034 method Methods 0.000 claims description 38
- 230000005540 biological transmission Effects 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 12
- 238000012545 processing Methods 0.000 claims description 12
- 239000000126 substance Substances 0.000 claims description 8
- 238000013473 artificial intelligence Methods 0.000 abstract description 3
- 230000002349 favourable effect Effects 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 15
- 238000004891 communication Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 230000000153 supplemental effect Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000001934 delay Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
- G05D1/0055—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot with safety arrangements
- G05D1/0061—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot with safety arrangements for transition from automatic pilot to manual pilot and vice versa
Abstract
The disclosure provides a remote control information processing method and device, electronic equipment and a computer storage medium automatic driving vehicle, relates to the field of computers, and particularly relates to the technical fields of artificial intelligence, automatic driving, Internet of things, intelligent transportation and the like. The specific implementation scheme is as follows: acquiring time information according to the vehicle-end information; sending the vehicle end information and the time information to a remote control end, wherein the remote control end is used for carrying out remote parallel control on the vehicle end; receiving feedback information returned by the remote control end according to the vehicle end information, wherein the feedback information comprises the time information; and determining the time delay of the remote control end for controlling the vehicle end according to the time information. The scheme is favorable for improving the safety of vehicle remote control.
Description
Technical Field
The present disclosure relates to the field of computer technology, and more particularly to the field of artificial intelligence, autopilot, internet of things, and intelligent transportation.
Background
With the development of computer technology, various aspects of people's life have changed. For example, in the field of vehicle technology, technologies such as automatic driving appear, and more choices and convenience are provided for vehicle driving. In automatic driving, a driving scheme for remote control of a vehicle has emerged in consideration of unexpected emergencies that may occur during driving.
Whether automatic driving, manual driving or remote driving, safety is always the first important factor, and remote driving can improve or solve the problem that automatic driving is difficult to deal with complex emergencies, but other factors may affect the safety of remote driving.
Disclosure of Invention
The disclosure provides a remote control information processing method and device, an electronic device and a computer storage medium automatic driving vehicle.
According to an aspect of the present disclosure, there is provided a remote control information processing method including:
acquiring time information according to the vehicle-end information;
the vehicle end information and the time information are sent to a remote control end, and the remote control end is used for carrying out remote parallel control on the vehicle end;
receiving feedback information returned by the remote control end according to the vehicle end information, wherein the feedback information comprises time information;
and determining the time delay of the remote control end for controlling the vehicle end according to the time information.
According to another aspect of the present disclosure, there is provided a remote control information processing method including:
receiving vehicle end information and time information sent by a vehicle end, wherein the time information is generated according to the vehicle end information;
generating feedback information according to the vehicle end information, wherein the feedback information contains the time information of the vehicle end;
and sending the feedback information to the vehicle end, so that the time delay of the remote control end for controlling the vehicle end can be calculated according to the time information.
According to another aspect of the present disclosure, there is provided a remote control information processing apparatus including:
the time information module is used for obtaining time information according to the vehicle-end information;
the first sending module is used for sending the vehicle end information and the time information to the remote control end, and the remote control end is used for carrying out remote parallel control on the vehicle end;
the first receiving module is used for receiving feedback information returned by the remote control end according to the vehicle end information, and the feedback information comprises time information;
and the time delay module is used for determining the time delay of the remote control end for controlling the vehicle end according to the time information.
According to another aspect of the present disclosure, there is provided a remote control information processing apparatus including:
the second receiving module is used for receiving the vehicle end information sent by the vehicle end;
the feedback module is used for generating feedback information according to the vehicle end information, and the feedback information contains the time information of the vehicle end;
and the second sending unit is used for sending the feedback information to the vehicle end, so that the time delay of the remote control end for controlling the vehicle end can be calculated according to the feedback information.
According to another aspect of the present disclosure, there is provided an electronic device including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method according to any one of the embodiments of the present disclosure.
According to another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to perform a method in any of the embodiments of the present disclosure.
According to another aspect of the present disclosure, there is provided a computer program product comprising computer programs/instructions which, when executed by a processor, implement the method in any of the embodiments of the present disclosure.
According to another aspect of the present disclosure, there is provided an autonomous vehicle including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method applicable to a vehicle end as provided by any one of the embodiments of the present disclosure.
According to another aspect of the present disclosure, there is provided a remote control server including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method provided by any one of the embodiments of the disclosure and applicable to the remote control terminal.
According to the technology disclosed by the invention, the problem that the time delay is difficult to calculate in the vehicle remote control process is solved, and the safety of remote control on the vehicle is improved.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
FIG. 1 is a schematic diagram of a remote control information processing method according to an embodiment of the present disclosure;
FIG. 2 is a schematic diagram of a remote control information processing method according to another embodiment of the present disclosure;
FIG. 3 is a schematic diagram of a remote control information processing method according to yet another embodiment of the present disclosure;
FIG. 4 is a schematic diagram of a remote control information processing method according to yet another embodiment of the present disclosure;
FIG. 5 is a schematic diagram of a remote control information processing method according to yet another embodiment of the present disclosure;
FIG. 6 is a schematic illustration of a vehicle end and a remote control end according to an example of the present disclosure;
FIG. 7 is a schematic diagram of an image frame and time of day information according to an example of the present disclosure;
FIG. 8 is a schematic diagram of a remote control information processing apparatus according to an embodiment of the present disclosure;
FIG. 9 is a schematic diagram of a remote control information processing apparatus according to another embodiment of the present disclosure;
FIG. 10 is a schematic diagram of a remote control information processing apparatus according to yet another embodiment of the present disclosure;
FIG. 11 is a schematic diagram of a remote control information processing apparatus according to yet another embodiment of the present disclosure;
FIG. 12 is a schematic diagram of a remote control information processing apparatus according to yet another embodiment of the present disclosure;
FIG. 13 is a schematic diagram of a remote control information processing apparatus according to yet another embodiment of the present disclosure;
fig. 14 is a block diagram of an electronic device for implementing a remote control information processing method of an embodiment of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
The embodiment of the present disclosure first provides a method for processing remote control information, as shown in fig. 1, including:
step S11: acquiring time information according to the vehicle-end information;
step S12: the vehicle end information and the time information are sent to a remote control end, and the remote control end is used for carrying out remote parallel control on the vehicle end;
step S13: receiving feedback information returned by the remote control end according to the vehicle end information, wherein the feedback information comprises time information;
step S14: and determining the time delay of the remote control end for controlling the vehicle end according to the time information.
In this embodiment, the vehicle-side information may be information that needs to be sent from the vehicle side to the remote control side of the vehicle side and may need the remote control side to respond or operate, for example, the vehicle-side information may be video information, sensor information, and the like collected by the vehicle side.
The vehicle-end information can be information which is actively sent to the remote control end by the vehicle end or information which is actively pulled from the vehicle end by the remote control end.
The vehicle-end information can be acquired by the vehicle end itself or by a third party. In one embodiment, all information related to remote driving and required to be transmitted to the remote control end through the vehicle end can be vehicle end information.
The time information may be obtained from the time at which any operation related to the vehicle-side information occurs. For example, the time information may be determined based on the acquisition time of the main content in the vehicle-side information. The time information can be determined according to the time when the vehicle-end information is ready to be sent. The time information can also be determined according to the sending time of the vehicle-side information.
In this embodiment, before obtaining the time information according to the vehicle-side information, obtaining the vehicle-side information may be further included. Specifically, the method may include obtaining the vehicle-end information through various devices capable of detecting and obtaining the vehicle surrounding environment information or other information related to the driving strategy. The method can comprise the steps of obtaining one type of vehicle-end information or more than two types of sub-information, and combining the more than two types of sub-information into the vehicle-end information.
The vehicle end information and the time information are sent to the remote control end, and the time information and the coded vehicle end information are combined and sent to the remote control end after the vehicle end information is coded.
For example, the time information may be added to the front part of the encoded vehicle-side information, and then the combined information may be transmitted to the remote control side.
In this embodiment, the remote control terminal may be used to remotely control the vehicle.
Under the condition that the vehicle end is the vehicle end of a common vehicle, when a driver at the vehicle end encounters an emergency inconvenient to drive, the driver at the vehicle end can request the control right of the vehicle managed by the remote control end, and after the control right of the vehicle is managed by the remote control end, the corresponding control right of the vehicle to the vehicle is closed. For example, if the remote control end manages the control right of the accelerator, the steering wheel and the brake device of the vehicle, the control right of the accelerator, the steering wheel and the brake device of the vehicle end is closed.
Under the condition that the vehicle end is the vehicle end of the automatic driving vehicle, if the vehicle encounters sudden weather changes, travels to a section of a complex road environment or encounters other special conditions in the automatic driving process, the vehicle end of the automatic driving vehicle can send a remote control request to the remote control end when detecting the special conditions, and the remote control end manages the control right of the vehicle end of the automatic driving vehicle to control corresponding driving parameters of the vehicle.
In this embodiment, the remote parallel control may include a remote take-over control of the vehicle.
The feedback information returned by the remote control end according to the vehicle end information can comprise a control instruction generated by the remote control end according to the vehicle end information or information determined to be received. The time information contained in the feedback information is the time information of the vehicle end information correspondingly fed back by the feedback information, and the time information fed back by the remote control end can indicate that the remote control end receives the vehicle end information.
In a possible implementation manner, the time delay of the remote control end for controlling the vehicle end is determined according to the time information, which may be the time information received by the remote control end and sent by the vehicle end, where the time information is the time information received by the remote control end and is the time information for controlling the vehicle end by the remote control end.
In this embodiment, when the vehicle end sends vehicle end information to the remote control end, send the time information that vehicle end information corresponds, the remote control end carries the same time information when returning feedback information to the vehicle end, thereby according to the time and the time information that the vehicle end received feedback information, just can calculate the time delay that obtains the remote control process, be convenient for know and master remote control's hysteresis condition, thereby make corresponding adjustment setting, be favorable to improving remote control vehicle's timeliness, improve remote control's security.
In one embodiment, as shown in fig. 2, the remote control information processing method further includes:
step S21: acquiring an image frame through an image acquisition device at a vehicle end;
step S22: coding the image frame to obtain vehicle end information;
step S23: and obtaining the time information of the vehicle end according to the time stamp when the image frame is obtained.
In this embodiment, the obtaining of the image frame by the vehicle-side image obtaining device may include obtaining a video stream by the vehicle-side image obtaining device, and obtaining at least one image frame in the video stream.
The image frames acquired by the present embodiment may be a plurality of image frames with the same time stamp as the image frame acquisition.
In this embodiment, the video encoding may refer to a mode of converting a file in an original video format into a file in another video format by a compression technique.
The image frame may be encoded by using an encoding method such as H264 or H265 to obtain encoded data. H264 may be h.264, a highly compressed digital Video codec standard proposed by the Joint Video Team (JVT) consisting of a Video Coding Experts Group (VCEG) and a Moving Picture Experts Group (MPEG) in a Joint. H265 may be a new video coding standard that VCEG makes after h.264. The h.265 standard surrounds the existing video coding standard h.264, preserving some of the original techniques, while improving some of the related techniques.
The time information of the vehicle end is obtained according to the time stamp when the image frame is obtained, and the time stamp when the image frame is to be obtained can be used as the time information, or the time information is obtained by filling the time stamp when the image frame is obtained into a set format.
When the remote control end carries out remote control on a vehicle, the remote control end cannot sense information around the vehicle and information related to driving in real time as in actual driving, the vehicle end is required to acquire information required by driving, such as the environment around the vehicle, through a sensor, a camera and other shooting devices and is transmitted to the remote control end through a network, but delay often occurs during transmission of images or videos, such as environmental information, and therefore when the vehicle end sends image frames to the remote control end, time delay is counted, and a good indication effect is achieved on the safety of the whole remote parallel control.
In the embodiment, the vehicle-end information and the time information are determined according to the image frames obtained by the vehicle-end image obtaining device, so that the time delay of video or image transmission, processing and other related steps which are most likely to generate time delay in the vehicle remote control process can be counted, and finally obtained time delay information can play a key role in guaranteeing the vehicle remote control safety.
In one embodiment, the feedback information further includes a control instruction generated by the remote control end according to the encoded image frame.
In the present embodiment, the control command includes a series of control commands related to the travel of the vehicle. The control method specifically comprises an accelerator control coefficient, a brake control coefficient, a horn control coefficient, a steering wheel control coefficient, a car light control coefficient and the like.
In the case where the control instruction includes a series of control instructions related to the travel of the vehicle, the time information may be added to the front of the control instruction, generating feedback information.
Under the real driving condition, the driver can timely sense the condition occurring outside the vehicle, so that the condition occurring outside the vehicle can be normally reacted and operated. However, in a scenario in which the vehicle end is remotely controlled, the operator in the remotely controlled cockpit cannot control the vehicle as if the operator actually operated at the vehicle end, and there may be a delay in generating the control command. In the embodiment, the feedback information comprises the control instruction generated by the remote control end, so that the time delay can be calculated and the time for generating the control instruction is included, and more valuable time delay information is provided for safe driving.
In one embodiment, obtaining time information of a vehicle end according to a time stamp when an image frame is acquired comprises:
generating a time stamp length code and a time stamp content code according to a time stamp when the image frame is acquired;
and adding the time stamp length code and the time stamp content code into a Supplemental Enhancement Information (SEI) frame to obtain time Information.
In this embodiment, the time stamp length code and the time stamp content code are generated according to the time stamp when the image frame is acquired, and may be generated according to the requirement of the supplemental enhancement information format, so that the time stamp length code and the time stamp content code of the time stamp are acquired by acquiring the image frame.
The time stamp length code may be an encoding for indicating the length of the time stamp content code. The time stamp content code may be an encoding of information indicating a time of day.
The time information is obtained by adding the time stamp length code and the time stamp content code into the supplemental enhancement information frame, and an SEI frame can be generated according to the time stamp length code and the time stamp content code.
In another possible mode, the time stamp length code can be omitted, the time stamp content code with the set length is set, and when the set length is less than the set length, a default symbol is filled in the redundant symbol bit, or the redundant symbol bit is left vacant.
In this embodiment, the structure of the SEI frame may specifically include a header, a timestamp length code, a timestamp content code, and a trailer.
In the embodiment, the information related to the timestamp is added to the SEI frame, so that the information is conveniently combined with the coding information of the image frame of the video for transmission, and the analysis is easy.
In one embodiment, determining the time delay of the remote control end controlling the vehicle end according to the time information, as shown in fig. 3, includes:
step S31: acquiring the moment when the feedback information is received;
step S32: calculating a time difference value according to the time when the feedback information is received and the time information;
step S33: and determining the information transmission time delay between the vehicle end and the remote control end according to the time difference.
The time when the feedback information is received can be determined by the timestamp of the received feedback information.
According to the embodiment, the time difference value is calculated according to the time of the received feedback information, so that the information transmission delay can be accurately determined.
In another possible implementation, after calculating the transmission delay, all transmission delays for a particular information transmission loop may be determined. For example, the transmission delay of all image frames of a transmission loop of a video stream corresponding to the whole windshield is subjected to, and the peak value and the average value of the transmission delay are determined in all the transmission delays.
An embodiment of the present disclosure further provides a remote control information processing method, as shown in fig. 4, including:
step S41: receiving vehicle end information and time information sent by a vehicle end, wherein the time information is generated according to the vehicle end information;
step S42: generating feedback information according to the vehicle end information, wherein the feedback information contains the time information;
step S43: and sending the feedback information to the vehicle end, so that the time delay of the remote control end for controlling the vehicle end can be calculated according to the time information.
In this embodiment, the feedback information is generated according to the vehicle-end information, and may be decoded according to the vehicle-end information, and the feedback information is generated according to the decoded information.
The receiving vehicle end information and the time information sent by the vehicle end can be the combined information of the vehicle end information and the time information sent by the receiving vehicle end, and the combined information is analyzed to obtain the vehicle end information and the time information.
In one embodiment, the vehicle-end information includes an image frame acquired by an image acquisition device at the vehicle end; the time information comprises a timestamp content code; generating feedback information based on the vehicle-side information, as shown in fig. 5, includes:
step S51: generating a corresponding control instruction according to the image frame;
step S52: acquiring a timestamp content code in the vehicle-side information;
step S53: and adding the content code of the timestamp into the control instruction to generate feedback information.
The control command may be a series of control commands for all vehicle components associated with vehicle travel, such as a throttle parameter control command, a brake parameter control command, and the like.
In this embodiment, the image frames received by the vehicle end may be a single image frame or a series of image frames. When the vehicle end receives a series of image frames, the time when the control command is generated and the time stamp of each received image frame can be determined according to the control command generated each time, the image frame of which the control command is consistent with the receiving time stamp is determined to be the image frame corresponding to the control command, the time stamp of the corresponding image frame is taken out, and the image frame and the control command generate feedback information and return the feedback information.
When the vehicle end receives a single image frame or a plurality of image frames with the same timestamp, a control instruction can be generated after the image frames are received and returned to the vehicle end together with the timestamp.
In one example of the present disclosure, as shown in fig. 6, the information transmission between the remote driving end 61 and the vehicle end 62 includes the following processes:
the SEI frame is added to the vehicle-side image acquisition code, the vehicle-side image acquisition code may be vehicle-side information in the foregoing embodiment, and the SEI frame may be time information. After an image frame acquired by a vehicle-end Camera (Camera) is encoded by H264, a series of image frames are generated, and meanwhile, an SEI frame is encoded into the image frame, and a vehicle-end current timestamp (which may be a timestamp during encoding) is added into the SEI frame. In the case where the image frame includes a plurality of frames, the image frame after the SEI frame is added with a timestamp acquired by each encoded H264 image frame at the end thereof as shown in fig. 7. The format of the SEI frame may include: frame header, Linux timestamp length, Linux timestamp content, frame tail.
And combining the image frame with the SEI frame by the vehicle end to form combined information, and transmitting the combined information to a remote control cabin end (remote control end). The coded image frames are combined with SEI frames and transmitted to a remote cockpit end through an RTC (Real Time Clock) data path, the remote cockpit end analyzes the combined information of the image frames and the SEI frames, the remote cockpit end renders a video at the remote cockpit end according to the image frames, then a timestamp in the SEI is taken out, and the timestamp is transmitted to a vehicle end through an Iot (Internet of Things, Internet of Things transmission protocol) data path.
After receiving the image frames and the SEI frames, the remote cockpit end identifies the SEI frames according to the format agreed by the SEI frame header and the SEI frame trailer, takes out the time stamps, splices the time stamps into the messages to be issued through the Iot access, generates feedback information by combining control messages such as steering wheel control messages and the like, and issues the feedback information to the vehicle end.
In another example, in the case of no time information, the information sent by the remote control end to the vehicle end includes parameters for controlling the throttle, brake, steering wheel angle, horn, left turn light, etc. of the vehicle end. Under the condition of adding time information, the information sent to the vehicle end by the remote control end sequentially comprises the following steps: time information, an accelerator, a brake, a steering wheel angle, a horn and a left turn lamp.
And after the vehicle end receives the feedback information sent by the remote cockpit section, calculating the time delay of the image data. The vehicle end receives the timestamp transmitted through the Iot, and the current time of the vehicle end-the received timestamp are used for end-to-end loop time delay capable of image transmission, namely the time delay for controlling the vehicle end by the remote control end, and meanwhile, the two times are calculated based on a unified clock of the vehicle end.
In this example, the time delay calculation formula is as follows:
loop time delay-the current time of the received frame _ time frame-the frame _ time value in the frame;
the overall time delay effect can be evaluated by calculating the time delay for each frame or image frame of each time stamp and then counting the mean and peak values of the loop time delay within a specific time interval.
An embodiment of the present disclosure further provides a remote control information processing apparatus, as shown in fig. 8, including:
the time information module 81 is used for obtaining time information according to the vehicle-end information;
the first sending module 82 is used for sending the vehicle end information and the time information to a remote control end, and the remote control end is used for performing remote parallel control on the vehicle end;
the first receiving module 83 is configured to receive feedback information returned by the remote control end according to the vehicle end information, where the feedback information includes time information;
and the time delay module 84 is used for determining the time delay of the remote control end for controlling the vehicle end according to the time information.
In one embodiment, as shown in fig. 9, the remote control information processing apparatus further includes:
the image frame module 91 is used for acquiring image frames through an image acquisition device at the vehicle end;
the encoding module 92 is used for encoding the image frames to obtain vehicle-end information;
the time information module is also used for obtaining the time information of the vehicle end according to the time stamp when the image frame is obtained.
In one embodiment, the feedback information further includes a control instruction generated by the remote control end according to the encoded image frame.
In one embodiment, as shown in fig. 10, the time information module further includes:
a first processing unit 101, configured to generate a timestamp length code and a timestamp content code according to a timestamp when an image frame is acquired;
and the second processing unit 102 is configured to add the timestamp length code and the timestamp content code to the supplemental enhancement information frame to obtain time information.
In one embodiment, as shown in fig. 11, the time delay module includes:
a time unit 111, configured to obtain a time when the feedback information is received;
a difference unit 112, configured to calculate a time difference according to the time when the feedback information is received and the time information;
and the time delay calculating unit 113 is configured to determine, according to the time difference, an information transmission time delay between the vehicle end and the remote control end.
An embodiment of the present disclosure further provides a remote control information processing apparatus, as shown in fig. 12, including:
the second receiving module 121 is configured to receive vehicle-end information and time information sent by a vehicle end, where the time information is generated according to the vehicle-end information;
the feedback module 122 is configured to generate feedback information according to the vehicle-end information, where the feedback information includes vehicle-end time information;
and the second sending module 123 is configured to send the feedback information to the vehicle end, so that the time delay of the remote control end controlling the vehicle end can be calculated according to the feedback information.
In one embodiment, as shown in fig. 13, the vehicle-end information includes image frames acquired by an image acquiring device at the vehicle end; the time information comprises a timestamp content code; the feedback module comprises:
an instruction unit 131, configured to generate a corresponding control instruction according to the image frame;
a content code unit 132, configured to obtain a timestamp content code in the vehicle-side information;
the generating unit 133 is configured to add the timestamp content code to the control instruction to generate feedback information.
The disclosed embodiment also provides an autonomous vehicle, including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method applicable to a vehicle end as provided by any one of the embodiments of the present disclosure.
An embodiment of the present disclosure further provides a remote control server, including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method provided by any one of the embodiments of the disclosure and applicable to the remote control terminal.
In the technical scheme of the disclosure, the acquisition, storage, application and the like of the personal information of the related user all accord with the regulations of related laws and regulations, and do not violate the good customs of the public order.
The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.
FIG. 14 shows a schematic block diagram of an example electronic device 140 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 14, the apparatus 140 includes a computing unit 141, which can perform various appropriate actions and processes in accordance with a computer program stored in a Read Only Memory (ROM)142 or a computer program loaded from a storage unit 148 into a Random Access Memory (RAM) 143. In the RAM 143, various programs and data required for the operation of the device 140 can also be stored. The calculation unit 141, the ROM 142, and the RAM 143 are connected to each other via a bus 144. An input/output (I/O) interface 145 is also connected to bus 144.
A number of components in the device 140 are connected to the I/O interface 145, including: an input unit 146 such as a keyboard, a mouse, or the like; an output unit 147 such as various types of displays, speakers, and the like; a storage unit 148 such as a magnetic disk, optical disk, or the like; and a communication unit 149 such as a network card, modem, wireless communication transceiver, etc. The communication unit 149 allows the device 140 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunication networks.
Computing unit 141 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 141 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The calculation unit 141 performs the respective methods and processes described above, such as the remote control information processing method. For example, in some embodiments, the remote control information processing method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 148. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 140 via ROM 142 and/or communications unit 149. When the computer program is loaded into RAM 143 and executed by computing unit 141, one or more steps of the remote control information processing method described above may be performed. Alternatively, in other embodiments, the computing unit 141 may be configured to perform the remote control information processing method by any other suitable means (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server with a combined blockchain.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and the present disclosure is not limited herein.
The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.
Claims (19)
1. A remote control information processing method, comprising:
acquiring time information according to the vehicle-end information;
sending the vehicle end information and the time information to a remote control end, wherein the remote control end is used for carrying out remote parallel control on the vehicle end;
receiving feedback information returned by the remote control end according to the vehicle end information, wherein the feedback information comprises the time information;
and determining the time delay of the remote control end for controlling the vehicle end according to the time information.
2. The method of claim 1, wherein the method further comprises:
acquiring an image frame through an image acquisition device at the vehicle end;
coding the image frame to obtain the vehicle end information;
the obtaining of the time information according to the vehicle end information includes:
and acquiring the time information of the vehicle end according to the time stamp when the image frame is acquired.
3. The method of claim 2, wherein the feedback information further comprises control instructions generated by the remote control from the encoded image frames.
4. The method of claim 2, wherein the obtaining the time information of the vehicle end according to the timestamp of the time when the image frame is obtained comprises:
generating a time stamp length code and a time stamp content code according to the time stamp when the image frame is obtained;
and adding the time stamp length code and the time stamp content code into a supplementary enhancement information frame to obtain the time information.
5. The method according to any one of claims 1 to 4, wherein the determining the time delay of the remote control end controlling the vehicle end according to the time information comprises:
acquiring the moment when the feedback information is received;
calculating a time difference value according to the time when the feedback information is received and the time information;
and determining the information transmission time delay between the vehicle end and the remote control end according to the time difference.
6. A remote control information processing method, comprising:
receiving vehicle end information and time information sent by a vehicle end, wherein the time information is generated according to the vehicle end information;
generating feedback information according to the vehicle end information, wherein the feedback information contains the time information of the vehicle end;
and sending the feedback information to a vehicle end, so that the time delay of the remote control end for controlling the vehicle end can be determined according to the time information.
7. The method according to claim 6, wherein the vehicle-end information comprises image frames acquired by an image acquisition device of the vehicle end; the time information comprises a timestamp content code; the generating of the feedback information according to the vehicle end information comprises:
generating a corresponding control instruction according to the image frame;
acquiring a timestamp content code in the vehicle-side information;
and adding the content code of the timestamp into the control instruction to generate the feedback information.
8. A remote control information processing apparatus comprising:
the time information module is used for obtaining time information according to the vehicle-end information;
the first sending module is used for sending the vehicle end information and the time information to a remote control end, and the remote control end is used for carrying out remote parallel control on the vehicle end;
the first receiving module is used for receiving feedback information returned by the remote control end according to the vehicle end information, and the feedback information comprises the time information;
and the time delay module is used for determining the time delay of the remote control end for controlling the vehicle end according to the time information.
9. The apparatus of claim 8, wherein the apparatus further comprises:
the image frame module is used for acquiring image frames through an image acquisition device at the vehicle end;
the coding module is used for coding the image frame to obtain the vehicle-end information;
and the time information module is also used for acquiring the time information of the vehicle end according to the timestamp when the image frame is acquired.
10. The apparatus of claim 9, wherein the feedback information further comprises control instructions generated by the remote control from the encoded image frames.
11. The apparatus of claim 9, wherein the time of day information module further comprises:
the first processing unit is used for generating a time stamp length code and a time stamp content code according to the time stamp when the image frame is acquired;
and the second processing unit is used for adding the time stamp length code and the time stamp content code into the supplementary enhancement information frame to obtain the time information.
12. The apparatus of any of claims 9-11, wherein the delay module comprises:
a time unit, configured to obtain a time when the feedback information is received;
a difference unit, configured to calculate a time difference according to the time at which the feedback information is received and the time information;
and the time delay calculating unit is used for determining the information transmission time delay between the vehicle end and the remote control end according to the time difference.
13. A remote control information processing apparatus comprising:
the second receiving module is used for receiving vehicle end information and time information sent by a vehicle end, and the time information is generated according to the vehicle end information;
the feedback module is used for generating feedback information according to the vehicle end information, and the feedback information contains the time information of the vehicle end;
and the second sending module is used for sending the feedback information to a vehicle end so as to calculate the time delay of the remote control end for controlling the vehicle end according to the time information.
14. The device according to claim 13, wherein the vehicle-end information includes image frames acquired by an image acquisition device of the vehicle end; the time information comprises a timestamp content code; the feedback module includes:
the instruction unit is used for generating a corresponding control instruction according to the image frame;
the content code unit is used for acquiring a timestamp content code in the vehicle-end information;
and the generating unit is used for adding the content code of the timestamp into the control instruction to generate the feedback information.
15. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.
16. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-7.
17. A computer program product comprising computer programs/instructions, characterized in that the computer programs/instructions, when executed by a processor, implement the steps of the method of any one of claims 1 to 7.
18. An autonomous vehicle comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.
19. A remote control server, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 6 or 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111165106.3A CN113900431A (en) | 2021-09-30 | 2021-09-30 | Remote control information processing method and device, electronic equipment and automatic driving vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111165106.3A CN113900431A (en) | 2021-09-30 | 2021-09-30 | Remote control information processing method and device, electronic equipment and automatic driving vehicle |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113900431A true CN113900431A (en) | 2022-01-07 |
Family
ID=79190169
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111165106.3A Pending CN113900431A (en) | 2021-09-30 | 2021-09-30 | Remote control information processing method and device, electronic equipment and automatic driving vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113900431A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115314422A (en) * | 2022-10-11 | 2022-11-08 | 智道网联科技(北京)有限公司 | Link delay statistical method and device for parallel driving central control vehicle instruction |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104015190A (en) * | 2014-05-13 | 2014-09-03 | 中国科学院力学研究所 | Robot remote control method and system under indeterminate bidirectional time delay condition |
| CN108604096A (en) * | 2016-01-28 | 2018-09-28 | 三星电子株式会社 | Show the method and its electronic equipment of image |
| US20190096244A1 (en) * | 2017-09-25 | 2019-03-28 | Intel Corporation | Vehicle-to-many-vehicle communication |
| CN109873733A (en) * | 2017-12-05 | 2019-06-11 | 上海欧菲智能车联科技有限公司 | Automobile bus communication means and system, remote terminal, automotive diagnostic installation |
| CN110032176A (en) * | 2019-05-16 | 2019-07-19 | 广州文远知行科技有限公司 | Long-range adapting method, device, equipment and the storage medium of pilotless automobile |
| CN111045425A (en) * | 2019-12-05 | 2020-04-21 | 中国北方车辆研究所 | Auxiliary teleoperation driving method for ground unmanned vehicle |
| CN112073142A (en) * | 2020-09-02 | 2020-12-11 | 浙江吉利汽车研究院有限公司 | Automatic parking method and system for vehicle |
| EP3772226A1 (en) * | 2019-07-30 | 2021-02-03 | Volkswagen AG | Methods, computer programs, and apparatuses for a command center and a vehicle, a vehicle and a command center for delay compensation in teleoperated driving |
| CN112486160A (en) * | 2019-08-23 | 2021-03-12 | 丰田自动车株式会社 | Vehicle remote indication system |
| CN112714282A (en) * | 2020-12-22 | 2021-04-27 | 北京百度网讯科技有限公司 | Image processing method, apparatus, device and program product in remote control |
| CN113141415A (en) * | 2021-05-07 | 2021-07-20 | 广州小鹏汽车科技有限公司 | Remote driving system, method, electronic device and storage medium for vehicle |
-
2021
- 2021-09-30 CN CN202111165106.3A patent/CN113900431A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104015190A (en) * | 2014-05-13 | 2014-09-03 | 中国科学院力学研究所 | Robot remote control method and system under indeterminate bidirectional time delay condition |
| CN108604096A (en) * | 2016-01-28 | 2018-09-28 | 三星电子株式会社 | Show the method and its electronic equipment of image |
| US20190096244A1 (en) * | 2017-09-25 | 2019-03-28 | Intel Corporation | Vehicle-to-many-vehicle communication |
| CN109873733A (en) * | 2017-12-05 | 2019-06-11 | 上海欧菲智能车联科技有限公司 | Automobile bus communication means and system, remote terminal, automotive diagnostic installation |
| CN110032176A (en) * | 2019-05-16 | 2019-07-19 | 广州文远知行科技有限公司 | Long-range adapting method, device, equipment and the storage medium of pilotless automobile |
| EP3772226A1 (en) * | 2019-07-30 | 2021-02-03 | Volkswagen AG | Methods, computer programs, and apparatuses for a command center and a vehicle, a vehicle and a command center for delay compensation in teleoperated driving |
| CN112486160A (en) * | 2019-08-23 | 2021-03-12 | 丰田自动车株式会社 | Vehicle remote indication system |
| CN111045425A (en) * | 2019-12-05 | 2020-04-21 | 中国北方车辆研究所 | Auxiliary teleoperation driving method for ground unmanned vehicle |
| CN112073142A (en) * | 2020-09-02 | 2020-12-11 | 浙江吉利汽车研究院有限公司 | Automatic parking method and system for vehicle |
| CN112714282A (en) * | 2020-12-22 | 2021-04-27 | 北京百度网讯科技有限公司 | Image processing method, apparatus, device and program product in remote control |
| CN113141415A (en) * | 2021-05-07 | 2021-07-20 | 广州小鹏汽车科技有限公司 | Remote driving system, method, electronic device and storage medium for vehicle |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115314422A (en) * | 2022-10-11 | 2022-11-08 | 智道网联科技(北京)有限公司 | Link delay statistical method and device for parallel driving central control vehicle instruction |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| RU2010129037A (en) | TRANSFER OF CHANGING VISUAL CONTENT | |
| US11197021B2 (en) | Coding resolution control method and terminal | |
| CN113900431A (en) | Remote control information processing method and device, electronic equipment and automatic driving vehicle | |
| CN111917661B (en) | Data transmission method, device, electronic equipment and computer readable storage medium | |
| CN113467875A (en) | Training method, prediction method, device, electronic equipment and automatic driving vehicle | |
| CN112714282A (en) | Image processing method, apparatus, device and program product in remote control | |
| CN112863187A (en) | Detection method of perception model, electronic equipment, road side equipment and cloud control platform | |
| CN113783651B (en) | Remote control method and device for parallel driving, cloud end, vehicle end and parallel driving system | |
| CN110737669A (en) | Data storage method, device, equipment and storage medium | |
| CN114120172A (en) | Video-based target detection method and device, electronic equipment and storage medium | |
| CN113961289A (en) | Data processing method, device, equipment and storage medium | |
| CN116866658A (en) | Video data processing method, device, equipment and medium | |
| CN113052047A (en) | Traffic incident detection method, road side equipment, cloud control platform and system | |
| CN104156188B (en) | A kind of high frame frequency Infrared scene generation and output system and method | |
| CN114834453A (en) | Data processing method, data processing device, vehicle, equipment and storage medium | |
| CN113313944B (en) | Method, device and system for processing lamp state data | |
| CN110673813B (en) | Data processing method and device, electronic equipment and medium | |
| CN112702660B (en) | Multimedia data transmission method and device, automatic driving vehicle and cloud server | |
| CN117176962B (en) | Video encoding and decoding method and device and related equipment | |
| CN113753065B (en) | Method and equipment for remotely controlling vehicle running in automatic driving and cloud control cabin | |
| KR102579124B1 (en) | Image data correction method, apparatus, electronic device, storage medium, computer program, and autonomous vehicle | |
| CN117135387A (en) | Video display method, system, vehicle and storage medium | |
| CN116033235B (en) | Data transmission method, digital person production equipment and digital person display equipment | |
| CN104093023A (en) | Vehicle-mounted video compressed transmission method and system | |
| CN115379207A (en) | Camera simulation method and device, electronic equipment and readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |