CN115208864A

CN115208864A - Data transmission method, device, equipment, vehicle and storage medium

Info

Publication number: CN115208864A
Application number: CN202210967256.4A
Authority: CN
Inventors: 黄启立
Original assignee: Apollo Zhilian Beijing Technology Co Ltd; Apollo Zhixing Technology Guangzhou Co Ltd
Current assignee: Apollo Zhilian Beijing Technology Co Ltd; Apollo Zhixing Technology Guangzhou Co Ltd
Priority date: 2022-08-12
Filing date: 2022-08-12
Publication date: 2022-10-18
Anticipated expiration: 2042-08-12
Also published as: CN115208864B

Abstract

The disclosure provides a data transmission method, a data transmission device, data transmission equipment, a vehicle and a storage medium, relates to the technical field of communication, and particularly relates to the technical fields of data transmission, cross-terminal interconnection and screen projection. The specific implementation scheme is as follows: acquiring media stream data to be transmitted, wherein the media stream data to be transmitted comprises video stream data and/or audio stream data; generating a real-time transport protocol (RTP) data packet according to media stream data to be transmitted; and sending an RTP data packet to the vehicle based on the RTP session so as to indicate the vehicle to obtain corresponding first media stream data by analyzing the RTP data packet, and playing the first media stream data by utilizing the vehicle-mounted component, wherein the RTP session is realized based on a user datagram protocol mode. The method and the device can reduce the performance overhead of the mobile terminal and the vehicle, avoid data blockage, further improve the data transmission efficiency, ensure the real-time performance of data transmission and ensure the smoothness of the vehicle when the media data are played.

Description

Data transmission method, device, equipment, vehicle and storage medium

Technical Field

The present disclosure relates to the field of communications technologies, and in particular, to the technical fields of data transmission, cross-terminal interconnection, and screen projection.

Background

With the development of vehicle-mounted scenes becoming faster and faster, the requirement of people for interconnection of the vehicle-mounted device and the mobile terminal becomes more and more obvious.

At present, in multimedia data Transmission of a vehicle-mounted scene, a mobile terminal directly transmits encoded video data and audio data to a vehicle terminal based on a Transmission Control Protocol (TCP), and the TCP is a connection-oriented reliable connection Protocol, which can ensure correctness and sequence of data Transmission.

Disclosure of Invention

The disclosure provides a data transmission method, a device, equipment and a storage medium.

According to an aspect of the present disclosure, there is provided a data transmission method, including:

acquiring media stream data to be transmitted, wherein the media stream data to be transmitted comprises video stream data and/or audio stream data;

generating a real-time transport protocol (RTP) data packet according to the media stream data to be transmitted;

and sending the RTP data packet to a vehicle based on an RTP session to indicate the vehicle to obtain corresponding first media stream data by analyzing the RTP data packet, and playing the first media stream data by utilizing a vehicle-mounted component, wherein the RTP session is realized based on a user datagram protocol mode.

According to another aspect of the present disclosure, there is provided a data transmission method including:

receiving an RTP data packet sent by a mobile terminal based on an RTP session, wherein the mobile terminal generates the RTP data packet according to acquired media stream data to be transmitted, the media stream data to be transmitted comprises video stream data and/or audio stream data, and the RTP session is realized based on a user datagram protocol mode;

obtaining corresponding first media stream data by analyzing the RTP data packet;

and playing the first media stream data by utilizing the vehicle-mounted component.

According to another aspect of the present disclosure, there is provided a data transmission apparatus including:

the device comprises a to-be-transmitted data acquisition module, a transmission module and a transmission module, wherein the to-be-transmitted data acquisition module is used for acquiring media stream data to be transmitted, and the media stream data to be transmitted comprises video stream data and/or audio stream data;

the first data packet generating module is used for generating a real-time transport protocol (RTP) data packet according to the media stream data to be transmitted;

the first data packet sending module is used for sending the RTP data packet to a vehicle based on an RTP session so as to indicate the vehicle to obtain corresponding first media stream data by analyzing the RTP data packet, and playing the first media stream data by utilizing a vehicle-mounted component, wherein the RTP session is realized based on a user datagram protocol mode.

the first data packet receiving module is used for receiving an RTP data packet sent by a mobile terminal based on a real-time transport protocol (RTP) session, wherein the mobile terminal generates the RTP data packet according to acquired media stream data to be transmitted, the media stream data to be transmitted comprises video stream data and/or audio stream data, and the RTP session is realized based on a User Datagram Protocol (UDP) mode;

the first data packet analysis module is used for obtaining corresponding first media stream data by analyzing the RTP data packet;

and the first playing module is used for playing the first media stream data by utilizing the vehicle-mounted component.

According to another aspect of the present disclosure, there is provided a mobile terminal including:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to cause the at least one processor to perform the method of the embodiments of the disclosure.

According to another aspect of the present disclosure, there is provided a vehicle including:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein, the first and the second end of the pipe are connected with each other,

the memory stores instructions executable by the at least one processor to cause the at least one processor to perform the method of the embodiments of the present disclosure.

According to another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of the embodiments of the present disclosure.

According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the method of any of the embodiments of the present disclosure.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

fig. 1a is a flowchart of a data transmission method provided according to an embodiment of the present disclosure;

fig. 1b is a schematic diagram of a data packet structure provided in accordance with an embodiment of the present disclosure;

fig. 2 is a flow chart of another data transmission method provided in accordance with an embodiment of the present disclosure;

FIG. 3 is a flow chart of yet another data transmission method provided in accordance with an embodiment of the present disclosure;

FIG. 4 is a flow chart of yet another data transmission method provided in accordance with an embodiment of the present disclosure;

fig. 5 is an interaction diagram of a data transmission method provided in accordance with an embodiment of the present disclosure;

fig. 6 is a schematic structural diagram of a data transmission device according to an embodiment of the present disclosure;

fig. 7 is a schematic structural diagram of another data transmission device provided in accordance with an embodiment of the present disclosure;

fig. 8 is a block diagram of a mobile terminal for implementing a data transmission method of an embodiment of the present disclosure;

FIG. 9 is a block diagram of a vehicle used to implement the data transmission method of an embodiment of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of embodiments of the present disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

Fig. 1a is a flowchart of a data transmission method provided according to an embodiment of the present disclosure, which is applicable to a case where data transmission is performed between a mobile terminal and a vehicle. The method may be performed by a data transmission device, which may be implemented in hardware and/or software and may be configured in a mobile terminal, where the mobile terminal may be, for example, a mobile phone, a smart watch, or a tablet computer. Referring to fig. 1a, the method specifically includes the following:

s101, media stream data to be transmitted are obtained, wherein the media stream data to be transmitted comprise video stream data and/or audio stream data;

s102, generating a real-time transport protocol (RTP) data packet according to the media stream data to be transmitted;

s103, the RTP data packet is sent to a vehicle based on an RTP session to indicate the vehicle to obtain corresponding first media stream data by analyzing the RTP data packet, and the first media stream data is played by utilizing a vehicle-mounted component, wherein the RTP session is realized based on a user datagram protocol mode.

The media stream data to be transmitted may be understood as media stream data to be sent to a vehicle in the mobile terminal, and specifically may be sent to a vehicle machine in the vehicle. The media stream data to be transmitted may include video stream data, may include audio stream data, and may also include both video stream data and audio stream data. In the embodiment of the present disclosure, the video stream data may include a plurality of video frames, and the source of the video frames is not limited, and may be, for example, a video from a video player application program, or a navigation image from a navigation application program. The audio stream data may include a plurality of audio frames, and the source of the audio frames is not limited, and may be, for example, audio from a video player type application program, or navigation speech from a navigation application program. Under the condition that the to-be-transmitted media stream data contains video stream data which is being displayed by a screen of the mobile terminal, the technical scheme of the embodiment of the disclosure can realize the effect that the mobile terminal casts a screen to a vehicle.

In the embodiment of the disclosure, the mobile terminal and the vehicle can be in communication connection in a wired or wireless mode. For example, a Real-time Transport Protocol (RTP) session implemented in a User Datagram Protocol (UDP) manner may be established between the mobile terminal and the vehicle, and the establishment process of the session is not limited. Optionally, a main data channel is first established between the mobile terminal and the vehicle, where the main data channel may serve as a channel for basic event delivery of the mobile terminal and the vehicle, and is specifically used for transmitting basic data, where the basic data may include, for example, an Internet Protocol (IP) address and a port number of the mobile terminal and/or the vehicle, and an RTP session may be established based on the IP address and the port number. Optionally, the main data channel may be established based on a Transmission Control Protocol (TCP), specifically, may be to establish a Socket (Socket) connection, and may be referred to as a MainData Socket.

For example, after acquiring the media stream data to be transmitted, the mobile terminal may generate an RTP data packet according to all data or part of data in the media stream data to be transmitted. For example, media stream data to be transmitted is encoded into data in a preset format, and then the encoded data is encapsulated into an RTP data packet. The preset format is not limited, for example, the preset format corresponding to the video stream data may be h.264, and the preset format corresponding to the audio stream data may be MP 3. When the RTP packet is encapsulated, the video stream data and the audio stream data may be encapsulated separately or together, which is not limited specifically.

Illustratively, fig. 1b is a schematic diagram of a data packet structure provided according to an embodiment of the present disclosure, as shown in fig. 1b, an RTP data packet includes a header (RTP header) and a body, media stream data is included in the RTP data packet (i.e., RTP data stream), the RTP data packet is included in a UDP data packet, and the RTP data packet and the UDP header are encapsulated into a UDP data packet.

For example, after the RTP packet is generated, the RTP packet is transmitted to the vehicle through an RTP session, so that the vehicle may obtain media stream data therein by parsing the RTP packet after receiving the RTP packet (the media stream data obtained after parsing may be the same as or different from the media stream data to be transmitted, and is referred to as the first media stream data herein for convenience of distinction), and play the first media stream data by using a vehicle-mounted component, which may include, for example, a vehicle-mounted display screen or a vehicle-mounted sound, etc.

According to the technical scheme provided by the embodiment of the disclosure, the mobile terminal can generate the RTP data packet according to the media stream data to be transmitted, and the RTP data is sent to the vehicle based on the RTP session, so that the vehicle can obtain corresponding media stream data by analyzing the RTP data packet and play the media stream data by utilizing the vehicle-mounted component.

When the mobile terminal and the vehicle perform media stream transmission, the mobile terminal and the vehicle may be divided into different transmission scenarios according to different data types in the media stream or different association relationships between different data, and in different transmission scenarios, the emphasis points of the transmission requirements may be different. Some scenes may be more focused on the fluency of the data stream playback; some scenarios may focus on the reliability of data stream transmission, and it is necessary to ensure the correctness and sequence of each frame data. In the embodiment of the present disclosure, a transmission condition that needs to be transmitted by using an RTP session may be preset and recorded as a preset transmission condition, and then the RTP session may be used for transmission when the preset transmission condition is satisfied.

In an optional implementation manner, the generating an RTP data packet according to the media stream data to be transmitted includes: and under the condition that the media stream data to be transmitted meet the preset transmission condition, generating an RTP data packet according to the video stream data in the media stream data to be transmitted. Wherein the preset transmission condition comprises at least one of the following: the media stream data to be transmitted only contains video stream data; and the association degree between the video stream data and the audio stream data contained in the media stream data to be transmitted is lower than the preset association degree. The advantage of this setting is that, adopt RTP conversation to carry out the transmission under the appropriate transmission scene, guarantee the timeliness of data transmission.

For example, if the media stream data to be transmitted only includes video stream data and does not include audio stream data, the video stream data is not associated with other data and is played independently, for example, a navigation video stream in a navigation application may allow packet loss and preferentially ensure timeliness; if the media stream data to be transmitted is the video stream data and the audio stream data at the same time, but the degree of association between the video stream data and the audio stream data is lower than the preset degree of association, that is, the degree of closeness is lower, for example, the navigation video stream and the navigation voice stream in the navigation application can allow packet loss, and the timeliness is preferentially ensured; if the association degree between the video stream data and the audio stream data included in the media stream data to be transmitted is higher than or equal to the preset association degree, that is, the closeness degree is higher, for example, if a movie picture and a character in a movie are in conversation, the viewing experience is poor, and reliability needs to be ensured, so that it is considered that the preset transmission condition is not satisfied, that is, an RTP data packet is not generated.

Optionally, the method for determining the association degree between the video stream data and the audio stream data included in the media stream data to be transmitted is lower than a preset association degree includes: the video stream data and the audio stream data contained in the media stream data to be transmitted come from a non-video player application program. The advantage of this arrangement is that it can be quickly determined whether the preset transmission condition is satisfied. The content played in the video player application is generally higher in closeness between pictures and sounds, such as movies, live video, short video works, and the like, so that if the video stream data and the audio stream data in the media stream data to be transmitted are from the same video player application, the preset transmission condition may not be satisfied, and if the video stream data and the audio stream data are from a non-video player application, such as a navigation application, the preset transmission condition may be satisfied.

Fig. 2 is a flowchart of another data transmission method according to an embodiment of the present disclosure, and this embodiment proposes an alternative scheme based on the foregoing optional embodiments, and transmits the data in a TCP manner when it is determined that the to-be-transmitted media stream data does not meet the preset transmission condition. Referring to fig. 2, the method includes:

s201, media stream data to be transmitted is obtained.

For example, the main data channel may be created and maintained after the mobile terminal and the vehicle successfully establish a wireless connection.

S202, determining whether the media stream data to be transmitted meets a preset transmission condition, if so, executing S203; otherwise, S205 is performed.

S203, generating an RTP data packet according to the video stream data in the media stream data to be transmitted.

For example, after determining that the media stream data to be transmitted satisfies the preset transmission condition, the RTP session may be established according to the IP address and the port number transmitted by the main data channel. If the RTP session exists at this time, the operation of generating the RTP packet can be directly performed.

And S204, sending an RTP data packet to the vehicle based on the RTP session so as to instruct the vehicle to obtain corresponding first media stream data by analyzing the RTP data packet, and playing the first media stream data by using the vehicle-mounted component.

Optionally, under the condition that the degree of association between the video stream data and the audio stream data contained in the to-be-transmitted media stream data is determined to be lower than the preset degree of association, a second TCP data packet is generated according to the audio stream data in the to-be-transmitted media stream data; and sending a second TCP data packet to the vehicle based on the main data channel so as to instruct the vehicle to obtain corresponding audio stream data by analyzing the second TCP data packet, and playing the obtained audio stream data by using the vehicle-mounted component. The advantage of this arrangement is that the data volume of the audio stream data is small, and the video stream data and the audio stream data can be separately transmitted by adopting different channels, thereby ensuring the reliability of audio transmission.

S205, generating a first TCP data packet according to the media stream data to be transmitted.

Optionally, the RTP session is destroyed when it is determined that the media stream data to be transmitted does not satisfy the preset transmission condition. The method has the advantages that when the RTP conversation is determined not to be adopted for data transmission, the RTP conversation is destroyed in time, corresponding resources are released, and the resource utilization rate is improved.

S206, the first TCP data packet is sent to the vehicle based on the main data channel so as to instruct the vehicle to obtain corresponding second media stream data by analyzing the first TCP data packet, and the second media stream data is played by using the vehicle-mounted component.

Illustratively, if the media stream data to be transmitted does not meet the preset transmission condition, a TCP mode can be adopted for transmission, so as to ensure the accuracy and the sequence of data transmission.

According to the technical scheme provided by the embodiment of the disclosure, whether the media stream data to be transmitted contains video stream data or not and the degree of association between the contained video stream data and the audio stream data is determined, whether the media stream data to be transmitted adopts an RTP mode or a TCP mode is determined, different transmission requirements under different transmission scenes are met, and the experience of watching or listening to the media stream by a user is improved.

In the embodiment of the present disclosure, when the RTP session is used for media stream transmission, a packet loss situation may exist, and a Real-time Transport Control Protocol (RTCP) session may be used to send an RTCP Control packet, so that a vehicle side may know packet loss information or network quality information in time. In an alternative embodiment, the method further comprises: generating an RTCP control packet, wherein the RTCP control packet comprises the sending condition information of the RTP data packet; and sending the RTCP control packet to the vehicle based on the RTCP session so as to indicate the vehicle to obtain the sending condition information by analyzing the RTCP control packet, and determining packet loss information and/or network quality information according to the sending condition information.

The sending condition information may include a sequence number of the RTP data packet, a sent number of the RTP data packet, and the like; the packet loss information may include a sequence number of an RTP packet that is not successfully received, a packet loss rate, and the like; the network quality information may be used to represent the network quality of the RTP channel, and may be determined according to the packet loss rate of the RTP packet, for example, when the packet loss rate is higher, it may be determined that the network quality is poor.

In an alternative embodiment, the RTCP session is established based on a primary data channel, which is established based on TCP; the method further comprises the following steps: receiving a data packet retransmission instruction sent by the vehicle based on the main data channel and/or the RTCP session; and carrying out retransmission operation on the target RTP data packet indicated by the data packet retransmission instruction. The method has the advantages that the target RTP data which needs to be retransmitted can be selected according to the indication of the vehicle, and retransmission is carried out, so that the integrity of data transmission is improved to a certain extent.

The target RTP packet may include all the RTP packets that are not successfully received, or may include part of the RTP packets that are not successfully received. In the case of including parts, the selection can be made according to the importance of the missing RTP packets, so that some unnecessary retransmission operations can be avoided.

In an optional embodiment, the data packet retransmission instruction is generated by the vehicle according to packet loss information. The packet loss information may include a video frame type corresponding to the lost data packet. For example, in encoding video data, in order to reduce the amount of data, it is possible to encode different types of video frames, such as key frames (I-frames), forward predictive encoded frames (P-frames), and bidirectional predictive encoded frames (B-frames), the importance of the three types decreasing in order. Retransmission can be performed on packet loss with higher importance, that is, a target RTP data packet is determined, such as an I frame, and a P frame can also be included; packet loss with lower importance can be ignored, retransmission is not carried out, for example, B frames or P frames can be included.

In an optional implementation manner, the data packet retransmission instruction is determined by the vehicle according to packet loss information and a current value of a preset vehicle performance index and/or network quality information, where the preset vehicle performance index is related to data processing capability of the vehicle. The advantage of such an arrangement is that the RTP data packet to be retransmitted can be flexibly and accurately determined according to the actual situation by combining the network quality information and/or the preset vehicle performance index.

The preset vehicle performance index may include an idle rate or a no-load rate of a processor, such as a Central Processing Unit (CPU), and may be calculated according to an occupancy rate or a load rate.

Optionally, when the vehicle determines that the current value of the preset vehicle performance index is lower than a first preset performance threshold, and/or determines that the current network quality is lower than the first preset quality threshold according to the network quality information, the vehicle determines, as a target RTP data packet, an RTP data packet that is not successfully received and to which video frame data of a target video frame type belongs according to packet loss information, so as to generate a data packet retransmission instruction, where the target video frame type includes a key frame. Therefore, the mobile terminal can carry out retransmission on the key frame in a targeted manner under the indication of the vehicle, the reliability of key frame transmission is preferentially ensured, and the pressure of the vehicle end can be relieved.

Specifically, in the first case, when determining that the current network quality is lower than the first preset quality threshold according to the network quality information, the vehicle determines, according to the packet loss information, an RTP data packet that is not successfully received and to which video frame data of a target video frame type belongs, as a target RTP data packet, so as to generate a data packet retransmission instruction, for example, if the packet loss rate is greater than the preset packet loss rate threshold, it is indicated that the network quality is poor, and reliable transmission of a key frame can be preferentially ensured; or, in the second case, under the condition that the current value of the preset vehicle performance index is determined to be lower than the first preset performance threshold, the RTP data packet which is not successfully received and to which the video frame data of the target video frame type belongs is determined as the target RTP data packet according to the packet loss information, so as to generate a data packet retransmission instruction, for example, the CPU idle rate is lower than the preset idle rate threshold, or the CPU load rate is higher than the preset load rate threshold, which indicates that the current resources used for processing media stream transmission of the vehicle are less, and can preferentially ensure reliable transmission of the key frame; or, in a third case, under the condition that the current network quality is determined to be lower than the first preset quality threshold according to the network quality information and the current value of the preset vehicle performance index is determined to be lower than the first preset performance threshold, determining an RTP data packet which is not successfully received and to which the video frame data of the target video frame type belongs as a target RTP data packet according to the packet loss information, so as to generate a data packet retransmission instruction.

Wherein the target video frame type comprises a key frame and can also comprise a forward predictive coding frame. Optionally, the target video frame types may also be different for the above three different cases. For example, in the first case and the second case, the target video frame type includes I frames and P frames, that is, does not include B frames; in the third case, the target video frame type includes I frames, i.e., P and B frames are not included.

Optionally, when the vehicle determines that the current value of the preset vehicle performance index is higher than a first preset performance threshold, or determines that the current network quality is higher than the first preset quality threshold according to the network quality information, or simultaneously meets the two requirements, the vehicle determines whether video frame data corresponding to an RTP data packet that is not successfully received is of a target video frame type according to the packet loss information, if yes, the vehicle determines the RTP data packet as a target RTP data packet to generate a data packet retransmission instruction, and if not, the vehicle does not generate the data packet retransmission instruction.

Fig. 3 is a flowchart of another data transmission method provided according to an embodiment of the present disclosure, and this embodiment proposes an alternative scheme based on the above optional embodiments, where the mobile terminal can perform targeted RTP transmission on video frame data of a target video frame type according to a data packet transmission instruction sent by a vehicle. Referring to fig. 3, the method includes:

s301, media stream data to be transmitted is obtained.

S302, under the condition that the media stream data to be transmitted meets the preset transmission condition, an RTP data packet is generated according to the video stream data in the media stream data to be transmitted.

And S303, sending an RTP data packet to the vehicle based on the RTP session to instruct the vehicle to obtain corresponding first media stream data by analyzing the RTP data packet, and playing the first media stream data by using the vehicle-mounted component.

S304, generating a real-time transport control protocol (RTCP) control packet, wherein the RTCP control packet comprises the sending condition information of the RTP data packet.

S305, sending an RTCP control packet to the vehicle based on the RTCP session so as to instruct the vehicle to obtain sending condition information by analyzing the RTCP control packet, and determining packet loss information and/or network quality information according to the sending condition information.

And S306, receiving a data packet retransmission instruction sent by the vehicle based on the main data channel and/or the RTCP session.

And the data packet retransmission instruction is generated by the vehicle according to the packet loss information, the current value of the preset vehicle performance index and/or the network quality information.

And S307, carrying out retransmission operation on the target RTP data packet indicated by the data packet retransmission instruction.

And S308, receiving a data packet transmission instruction sent by the vehicle based on the main data channel and/or the RTCP session.

The data packet transmission instruction is generated by the vehicle according to the current value and/or the network quality information of a preset vehicle performance index, the data packet transmission instruction is used for indicating the type of a target video frame, and the preset vehicle performance index is related to the data processing capacity of the vehicle.

For example, during the transmission of media stream data based on an RTP session, the network quality or the preset vehicle performance index may be dynamically changed, and the vehicle may instruct the type of the video frame transmitted by the mobile terminal according to the current actual situation.

Optionally, the vehicle generates a data packet transmission instruction when determining that the current value of the preset vehicle performance index is lower than a second preset performance threshold, and/or determining that the current network quality is lower than the second preset quality threshold according to the network quality information, where the target video frame type indicated by the data packet transmission instruction includes a key frame. Therefore, the mobile terminal can transmit the key frames under the indication of the vehicle, so that the occupation of non-key frames on a transmission channel is reduced, the successful transmission probability of the key frames is improved, and the pressure of the vehicle end is relieved.

For example, the target video frame type does not include non-key frames, so that the occupation of the non-key frames on a transmission channel is reduced, and the probability of successful transmission of the key frames is improved. Optionally, the target video frame type may also include a P frame under the condition that the network quality is not lower than the preset minimum quality threshold and/or the current value of the preset vehicle performance index is not lower than the preset minimum performance threshold.

S309, generating a new RTP data packet according to the video frame data which is in line with the target video frame type in the newly acquired media stream data to be transmitted.

It should be noted that the media stream data to be transmitted newly acquired here is used for distinguishing from the media stream data to be transmitted acquired in S301, and the acquisition timing sequence of the two is not limited; the new RTP packet is used for distinguishing from the RTP packet in S302, and the generation timing of the two is not limited. If the data packet transmission instruction is generated by the vehicle according to the current value of the preset vehicle performance index, that is, the generation of the data packet transmission instruction does not depend on the network quality information, before S302, the data packet transmission instruction sent by the vehicle may also be received, and the RTP data packet is generated according to the video frame data conforming to the target video frame type in the media stream data to be transmitted acquired in S301.

And S310, sending a new RTP data packet to the vehicle based on the RTP session so as to instruct the vehicle to obtain corresponding third media stream data by analyzing the new RTP data packet, and playing the third media stream data by using the vehicle-mounted component.

For example, after the vehicle receives a new RTP packet, the parsed third media stream data may include only an I frame, or an I frame and a P frame, and the success rate of transmitting a key frame may be improved by discarding all or part of the transmission of non-key frames, or the load and other stresses on the vehicle side may be relieved.

According to the technical scheme provided by the embodiment of the disclosure, in the process of transmitting media stream data to a vehicle by a mobile terminal in an RTP mode, an RTCP control packet is transmitted through an RTCP session, so that the vehicle can timely and accurately acquire packet loss information and/or network quality information according to the analyzed sending condition information of the RTP data packet, and further generate a data packet retransmission instruction according to the packet loss information, and the current value of the preset vehicle performance index and/or the network quality information, and after receiving the data packet retransmission instruction, the mobile terminal can purposefully perform packet loss retransmission.

Fig. 4 is a flowchart of still another data transmission method provided according to an embodiment of the present disclosure, which is applicable to a case where data transmission is performed between a mobile terminal and a vehicle. The method can be executed by a data transmission device, which can be implemented by hardware and/or software, and can be configured in a vehicle, specifically, a vehicle machine. Referring to fig. 4, the method specifically includes the following:

s401, receiving an RTP data packet sent by a mobile terminal based on an RTP session, wherein the RTP data packet is generated by the mobile terminal according to the acquired media stream data to be transmitted, the media stream data to be transmitted comprises video stream data and/or audio stream data, and the RTP session is realized based on a user datagram protocol mode;

s402, obtaining corresponding first media stream data by analyzing the RTP data packet;

and S403, playing the first media stream data by using the vehicle-mounted component.

According to the technical scheme provided by the embodiment of the disclosure, the vehicle can receive the RTP data packet which is sent by the mobile terminal based on the RTP session and is generated according to the media stream data to be transmitted, the corresponding media stream data is obtained by analyzing the RTP data packet and is played by utilizing the vehicle-mounted component, the RTP session is realized based on a UDP mode, the data packet can be transmitted at any time in a data transmission oriented mode, the correctness and the sequence of the data packet do not need to be ensured, the relevance between the data packets does not need to be processed, the performance overhead of the mobile terminal and the vehicle can be reduced, in addition, the flow control or congestion control does not need to be provided, the occurrence of data blockage can be avoided, the data transmission efficiency is further improved, the real-time performance of data transmission is ensured, and the smoothness can be ensured when the vehicle plays the media data.

In an optional implementation manner, the mobile terminal generates the RTP data packet according to video stream data in the media stream data to be transmitted, when it is determined that the media stream data to be transmitted satisfies a preset transmission condition. Wherein the preset transmission condition comprises at least one of the following: the media stream data to be transmitted only contains video stream data; and the association degree between the video stream data and the audio stream data contained in the media stream data to be transmitted is lower than the preset association degree. The advantage of this setting is that, adopt RTP conversation to carry out the transmission under the appropriate transmission scene, guarantee the timeliness of data transmission.

In an optional embodiment, the association degree between the video stream data and the audio stream data included in the media stream data to be transmitted is lower than a preset association degree, and the method includes: the video stream data and the audio stream data contained in the media stream data to be transmitted come from a non-video player application program. The advantage of this arrangement is that it can be quickly determined whether the predetermined transmission conditions are met.

In an alternative embodiment, the RTP session is established based on a primary data channel, which is established based on the transmission control protocol TCP. Wherein the method further comprises: receiving a first TCP data packet sent by the mobile terminal based on the main data channel, wherein the first TCP data packet is generated by the mobile terminal according to the media stream data to be transmitted under the condition that the media stream data to be transmitted does not meet the preset transmission condition; obtaining corresponding second media stream data by analyzing the first TCP data packet; and playing the second media stream data by utilizing the vehicle-mounted component. The advantage of setting up like this lies in, when not satisfying the transmission condition of RTP mode, adopts the TCP mode to transmit, guarantees the timely reliable transmission of media stream data, satisfies different transmission demands under the different transmission scenes, promotes the experience that the user watched or listened to the media stream.

In an alternative embodiment, the method further comprises: and under the condition that the media stream data to be transmitted are determined not to meet the preset transmission condition, destroying the RTP session. The method has the advantages that when the RTP conversation is determined not to be adopted for data transmission, the RTP conversation is destroyed in time, corresponding resources are released, and the resource utilization rate is improved.

In an alternative embodiment, the RTP session is established based on a primary data channel, which is established based on TCP; the method further comprises the following steps: receiving a second TCP data packet sent by the mobile terminal based on the main data channel, wherein the second TCP data packet is generated by the mobile terminal according to audio stream data in the to-be-transmitted media stream data under the condition that the mobile terminal determines that the degree of association between video stream data and audio stream data contained in the to-be-transmitted media stream data is lower than a preset degree of association; obtaining corresponding audio stream data by analyzing the second TCP data packet; and playing the obtained audio stream data by using the vehicle-mounted component. The advantage of this setting is that, the data bulk of audio streaming data is less, can separate the different channels that adopt of video streaming data and audio streaming data to transmit, guarantees the reliability of audio transmission.

In an alternative embodiment, the method further comprises: receiving a real-time transport control protocol (RTCP) control packet sent by the mobile terminal based on an RTCP session, wherein the RTCP control packet comprises the sending condition information of the RTP data packet; analyzing the RTCP control packet to obtain the sending condition information; and determining packet loss information and/or network quality information according to the sending condition information. The advantage of such a configuration is that when the RTP session is used for media stream transmission, there may be a packet loss situation, and the RTCP session may be used to send RTCP control packets, so that the vehicle side can know packet loss information or network quality information in time.

In an alternative embodiment, the RTCP session is established based on a primary data channel, which is established based on TCP; the method further comprises the following steps: and based on the data packet retransmission instruction sent to the mobile terminal by the main data channel and/or the RTCP session, instructing the mobile terminal to perform retransmission operation on the target RTP data packet indicated by the data packet retransmission instruction. The advantage of this arrangement is that the target RTP data to be retransmitted can be selected according to the indication of the vehicle, and retransmitted, and the integrity of data transmission is improved to some extent.

In an alternative embodiment, the method further comprises: and generating the data packet retransmission instruction according to at least one item of packet loss information and current values of preset vehicle performance indexes and/or network quality information, wherein the preset vehicle performance indexes are related to the data processing capacity of the vehicle. The advantage of such an arrangement is that the RTP data packet to be retransmitted can be flexibly and accurately determined according to the actual situation by combining the network quality information and/or the preset vehicle performance index.

In an optional implementation manner, the generating the data packet retransmission instruction according to the packet loss information and the current value of the preset vehicle performance index and/or the network quality information includes: and under the condition that the current value of the preset vehicle performance index is lower than a first preset performance threshold value and/or the current network quality is lower than the first preset quality threshold value according to the network quality information, determining an RTP data packet which is not successfully received and belongs to the video frame data of the target video frame type as a target RTP data packet according to the packet loss information so as to generate a data packet retransmission instruction, wherein the target video frame type comprises a key frame. The method has the advantages that the type of the video frame in the RTP data packet which indicates the mobile terminal to need retransmission is reasonably determined, the reliability of key frame transmission is preferentially ensured, and the pressure of a vehicle end can be relieved.

In an alternative embodiment, the RTCP session is established based on a primary data channel, which is established based on TCP. The method further comprises the following steps: generating a data packet transmission instruction according to a current value and/or network quality information of a preset vehicle performance index, wherein the data packet transmission instruction is used for indicating a target video frame type, and the preset vehicle performance index is related to the data processing capacity of the vehicle; based on the main data channel and/or the RTCP session, sending the data packet transmission instruction to the mobile terminal to indicate the mobile terminal to generate a new RTP data packet according to video frame data which is in line with the target video frame type in newly acquired media stream data to be transmitted, and returning the new RTP data packet based on the RTP session; analyzing the new RTP data packet to obtain corresponding third media stream data; and playing the third media stream data by utilizing the vehicle-mounted component. The method has the advantages that the vehicle indicates the type of the video frame needing to be transmitted by the mobile terminal in an RTP mode according to the network condition or the performance condition of the vehicle, so that the occupation of the transmission channel by the transmission of the video frame data except the target video frame type is reduced, the probability of successful transmission of the video frame of the target video frame type is improved, and the pressure of the vehicle end is relieved.

In an optional implementation manner, the generating a data packet transmission instruction according to the current value of the preset vehicle performance index and/or the network quality information includes: and generating a data packet transmission instruction under the condition that the current value of the preset vehicle performance index is lower than a second preset performance threshold value and/or the current network quality is lower than the second preset quality threshold value according to the network quality information, wherein the target video frame type indicated by the data packet transmission instruction comprises a key frame. The method has the advantages that the type of the video frame which indicates the mobile terminal to adopt the RTP mode for transmission is reasonably determined, the occupation of non-key frames on a transmission channel is reduced, the probability of successful transmission of the key frames is improved, and the pressure of a vehicle end is relieved.

Fig. 5 is an interaction diagram of a data transmission method according to an embodiment of the present disclosure. As shown in fig. 5, a vehicle and a mobile terminal first establish a main data channel, and transmit an IP address and a port number on the main data channel to establish an RTP session and an RTCP session, and after acquiring media stream data to be transmitted, the mobile terminal determines whether the media stream data to be transmitted satisfies a preset transmission condition, if so, the media stream data to be transmitted is transmitted in an RTP mode, and if not, the media stream data to be transmitted is transmitted in the main data channel. When the RTP mode is adopted for transmission, media stream data is packaged into an RTP data packet, an RTCP control packet is generated, the RTP data packet is transmitted through an RTP channel, and the RTCP control packet is transmitted through the RTCP channel. And after receiving the RTP data packet, the vehicle analyzes the RTP data packet to obtain corresponding media stream data, and performs rendering display on a vehicle-mounted display screen. When the vehicle determines that the packet loss condition exists according to the RTCP control packet, the vehicle determines whether the RTP data packet which is not successfully received corresponds to the key frame or not when the current key frame needs to be retransmitted according to the packet loss information, the CPU occupancy rate, the packet loss rate and the like, if so, a data packet retransmission instruction is generated, and the mobile terminal is instructed to retransmit the key frame which has lost the packet. When the vehicle determines that the transmission of the non-key frame needs to be abandoned according to the CPU occupancy rate, the packet loss rate and the like, the vehicle can send a data packet transmission instruction to the mobile terminal to instruct the mobile terminal to transmit the RTP data packet which does not contain the non-key frame.

In the technical scheme of the disclosure, the collection, storage, use, processing, transmission, provision, disclosure and the like of the related media stream data, the personal information of the user and the like all accord with the regulations of related laws and regulations without violating the customs of public order.

Fig. 6 is a schematic structural diagram of a data transmission device provided according to an embodiment of the present disclosure, which is applicable to a case where data transmission is performed between a mobile terminal and a vehicle. The device can be implemented by hardware and/or software, and can be configured in a mobile terminal, and the device can implement the data transmission method described in the corresponding embodiment of the disclosure. As shown in fig. 6, the data transmission apparatus 600 includes:

a to-be-transmitted data obtaining module 601, configured to obtain media stream data to be transmitted, where the media stream data to be transmitted includes video stream data and/or audio stream data;

a first data packet generating module 602, configured to generate a real-time transport protocol RTP data packet according to the media stream data to be transmitted;

a first data packet sending module 603, configured to send an RTP data packet to a vehicle based on an RTP session, so as to instruct the vehicle to obtain corresponding first media stream data by parsing the RTP data packet, and play the first media stream data by using a vehicle-mounted component, where the RTP session is implemented based on a user datagram protocol.

In an optional implementation manner, the first packet generation module is specifically configured to:

under the condition that the media stream data to be transmitted meet the preset transmission condition, generating an RTP data packet according to video stream data in the media stream data to be transmitted;

wherein the preset transmission condition comprises at least one of the following:

the media stream data to be transmitted only contains video stream data;

and the association degree between the video stream data and the audio stream data contained in the media stream data to be transmitted is lower than the preset association degree.

In an optional embodiment, the associating degree between the video stream data and the audio stream data included in the media stream data to be transmitted is lower than a preset associating degree, including:

the video stream data and the audio stream data contained in the media stream data to be transmitted come from a non-video player application program.

In an optional embodiment, the RTP session is established based on a main data channel, and the main data channel is established based on a transmission control protocol TCP;

wherein the apparatus further comprises:

the second data packet generation module is used for generating a first TCP data packet according to the media stream data to be transmitted under the condition that the media stream data to be transmitted does not meet the preset transmission condition;

and the second data packet sending module is used for sending the first TCP data packet to the vehicle based on the main data channel so as to instruct the vehicle to obtain corresponding second media stream data by analyzing the first TCP data packet, and playing the second media stream data by utilizing a vehicle-mounted component.

In an alternative embodiment, the method further comprises:

and the first session destroying module is used for destroying the RTP session under the condition that the media stream data to be transmitted does not meet the preset transmission condition.

In an alternative embodiment, the RTP session is established based on a primary data channel, which is established based on TCP;

the device further comprises:

the third data packet generating module is used for generating a second TCP data packet according to the audio stream data in the media stream data to be transmitted under the condition that the correlation degree between the video stream data and the audio stream data contained in the media stream data to be transmitted is lower than the preset correlation degree;

and the third data packet sending module is used for sending the second TCP data packet to the vehicle based on the main data channel so as to instruct the vehicle to obtain corresponding audio stream data by analyzing the second TCP data packet, and playing the obtained audio stream data by utilizing a vehicle-mounted component.

In an alternative embodiment, the method further comprises:

a control packet generating module, configured to generate an RTCP control packet, where the RTCP control packet includes information about a sending status of the RTP data packet;

and the control packet sending module is used for sending the RTCP control packet to the vehicle based on an RTCP session so as to indicate the vehicle to obtain the sending condition information by analyzing the RTCP control packet, and determining packet loss information and/or network quality information according to the sending condition information.

In an alternative embodiment, the RTCP session is established based on a primary data channel, which is established based on TCP;

the device further comprises:

a retransmission instruction receiving module, configured to receive a retransmission instruction of a data packet sent by the vehicle based on the primary data channel and/or the RTCP session;

and the retransmission module is used for carrying out retransmission operation on the target RTP data packet indicated by the data packet retransmission instruction.

In an optional implementation manner, the data packet retransmission instruction is generated by the vehicle according to packet loss information and a current value of a preset vehicle performance index and/or network quality information, where the preset vehicle performance index is related to data processing capability of the vehicle.

In an alternative embodiment, the RTCP session is established based on a primary data channel established based on TCP;

the device further comprises:

a transmission instruction receiving module, configured to receive a data packet transmission instruction sent by the vehicle based on the master data channel and/or the RTCP session, where the data packet transmission instruction is generated by the vehicle according to a current value of a preset vehicle performance index and/or network quality information, the data packet transmission instruction is used to indicate a target video frame type, and the preset vehicle performance index is related to a data processing capability of the vehicle;

the fourth data packet generating module is used for generating a new RTP data packet according to video frame data which is in line with the type of the target video frame in newly acquired media stream data to be transmitted;

and the fourth data packet sending module is used for sending the new RTP data packet to the vehicle based on the RTP session so as to indicate the vehicle to obtain corresponding third media stream data by analyzing the new RTP data packet, and playing the third media stream data by utilizing a vehicle-mounted component.

Fig. 7 is a schematic structural diagram of another data transmission device provided according to an embodiment of the present disclosure, which is applicable to a case where data transmission is performed between a mobile terminal and a vehicle. The device can be implemented by hardware and/or software, can be configured in a vehicle, and can implement the data transmission method according to the corresponding embodiment of the disclosure. As shown in fig. 7, the data transmission apparatus 700 includes:

a first data packet receiving module 701, configured to receive an RTP data packet sent by a mobile terminal based on an RTP session, where the mobile terminal generates the RTP data packet according to obtained to-be-transmitted media stream data, where the to-be-transmitted media stream data includes video stream data and/or audio stream data, and the RTP session is implemented based on a user datagram protocol (udp) mode;

a first data packet parsing module 702, configured to parse the RTP data packet to obtain corresponding first media stream data;

a first playing module 703, configured to play the first media stream data by using the vehicle-mounted component.

In an optional implementation manner, the mobile terminal generates the RTP data packet according to video stream data in the media stream data to be transmitted under the condition that it is determined that the media stream data to be transmitted satisfies a preset transmission condition;

wherein the preset transmission condition comprises at least one of:

the media stream data to be transmitted only contains video stream data;

wherein the apparatus further comprises:

a second data packet receiving module, configured to receive, based on the main data channel, a first TCP data packet sent by the mobile terminal, where the first TCP data packet is generated by the mobile terminal according to the to-be-transmitted media stream data when it is determined that the to-be-transmitted media stream data does not meet the preset transmission condition;

the second data packet analysis module is used for analyzing the first TCP data packet to obtain corresponding second media stream data;

and the second playing module is used for playing the second media stream data by utilizing the vehicle-mounted component.

In an alternative embodiment, the method further comprises:

and the second session destroying module is used for destroying the RTP session under the condition that the media stream data to be transmitted does not meet the preset transmission condition.

In an optional embodiment, the RTP session is established based on a primary data channel, and the primary data channel is established based on TCP;

the device further comprises:

a third data packet receiving module, configured to receive, based on the main data channel, a second TCP data packet sent by the mobile terminal, where the second TCP data packet is generated by the mobile terminal according to audio stream data in the to-be-transmitted media stream data when it is determined that a degree of association between video stream data and audio stream data included in the to-be-transmitted media stream data is lower than a preset degree of association;

the third data packet analysis module is used for analyzing the second TCP data packet to obtain corresponding audio stream data;

and the third playing module is used for playing the obtained audio stream data by utilizing the vehicle-mounted component.

In an alternative embodiment, the method further comprises:

a control packet receiving module, configured to receive a real-time transport control protocol RTCP control packet sent by the mobile terminal based on an RTCP session, where the RTCP control packet includes information about a sending condition of the RTP data packet;

the control packet analysis module is used for obtaining the sending condition information by analyzing the RTCP control packet;

and the information determining module is used for determining packet loss information and/or network quality information according to the sending condition information.

the device further comprises:

and the retransmission instruction sending module is used for sending a data packet retransmission instruction to the mobile terminal based on the main data channel and/or the RTCP session so as to instruct the mobile terminal to perform retransmission operation on a target RTP data packet indicated by the data packet retransmission instruction.

In an alternative embodiment, the method further comprises:

and the retransmission instruction generating module is used for generating the data packet retransmission instruction according to at least one item of packet loss information and current value of a preset vehicle performance index and/or network quality information, wherein the preset vehicle performance index is related to the data processing capacity of the vehicle.

In an optional implementation manner, the retransmission instruction generation module is specifically configured to:

under the condition that the current value of the preset vehicle performance index is lower than a first preset performance threshold value and/or the current network quality is lower than the first preset quality threshold value according to the network quality information, determining an RTP data packet which is not successfully received and belongs to video frame data of a target video frame type as a target RTP data packet according to packet loss information so as to generate a data packet retransmission instruction; wherein the target video frame type comprises a key frame.

the device further comprises:

the system comprises a transmission instruction generation module, a data packet transmission module and a data transmission module, wherein the transmission instruction generation module is used for generating a data packet transmission instruction according to a current value and/or network quality information of a preset vehicle performance index, the data packet transmission instruction is used for indicating a target video frame type, and the preset vehicle performance index is related to the data processing capacity of the vehicle;

a transmission instruction sending module, configured to send the data packet transmission instruction to the mobile terminal based on the main data channel and/or the RTCP session, so as to instruct the mobile terminal to generate a new RTP data packet according to video frame data that conforms to the target video frame type in newly acquired media stream data to be transmitted, and return the new RTP data packet based on the RTP session;

a fourth data packet receiving module, configured to obtain corresponding third media stream data by parsing the new RTP data packet;

and the fourth playing module is used for playing the third media stream data by utilizing the vehicle-mounted component.

In an optional implementation manner, the transmission instruction generating module is specifically configured to:

and generating a data packet transmission instruction under the condition that the current value of the preset vehicle performance index is lower than a second preset performance threshold value and/or the current network quality is lower than the second preset quality threshold value according to the network quality information, wherein the target video frame type indicated by the data packet transmission instruction comprises a key frame.

The present disclosure also provides a mobile terminal, a vehicle, a readable storage medium, and a computer program product according to embodiments of the present disclosure.

FIG. 8 illustrates a schematic block diagram of an example mobile terminal 800 that may be used to implement embodiments of the present disclosure. Mobile terminals are intended to represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 8, the mobile terminal 800 includes a computing unit 801 that can perform various appropriate actions and processes in accordance with a computer program stored in a Read Only Memory (ROM) 802 or a computer program loaded from a storage unit 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the mobile terminal 800 can also be stored. The calculation unit 801, the ROM 802, and the RAM 803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.

A number of components in the mobile terminal 800 are connected to the I/O interface 805, including: an input unit 806 such as a keyboard, a mouse, or the like; an output unit 807 such as various types of displays, speakers, and the like; a storage unit 808, such as a magnetic disk, optical disk, or the like; and a communication unit 809 such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows the mobile terminal 800 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunication networks.

Computing unit 801 may be a variety of general and/or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The calculation unit 801 executes the respective methods and processes described above, such as the data transmission method. For example, in some embodiments, the data transfer method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and/or installed onto the mobile terminal 800 via the ROM 802 and/or the communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the data transfer method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the data transfer method by any other suitable means (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Fig. 9 is a block diagram of a vehicle for implementing a data transmission method of an embodiment of the present disclosure, the vehicle 900 including: a processor 901, and a memory 902 communicatively connected to the processor 901, wherein the memory 902 stores a computer program executable by the processor 901, and the computer program is executed by the processor 901, so as to enable the processor 901 to execute the data transmission method in the embodiment of the present disclosure. The number of the processors may be one or more, and one processor is taken as an example in fig. 9.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

It should be understood that various forms of the flows shown above, reordering, adding or deleting steps, may be used. For example, the steps described in this disclosure may be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions provided by this disclosure can be achieved, and are not limited herein.

The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A method of data transmission, comprising:

2. The method of claim 1, wherein the generating an RTP packet according to the media stream data to be transmitted comprises:

the media stream data to be transmitted only contains video stream data;

3. The method of claim 2, wherein the degree of the correlation between the video stream data and the audio stream data contained in the media stream data to be transmitted is lower than a preset degree of correlation, and comprises the following steps:

the video stream data and the audio stream data contained in the media stream data to be transmitted come from non-video player application programs.

4. The method of claim 2, wherein the RTP session is established based on a primary data channel established based on the transmission control protocol TCP;

wherein the method further comprises:

under the condition that the media stream data to be transmitted are determined not to meet the preset transmission condition, generating a first TCP data package according to the media stream data to be transmitted;

and sending the first TCP data packet to the vehicle based on the main data channel so as to instruct the vehicle to obtain corresponding second media stream data by analyzing the first TCP data packet, and playing the second media stream data by using a vehicle-mounted component.

5. The method of claim 4, further comprising:

and under the condition that the media stream data to be transmitted are determined not to meet the preset transmission condition, destroying the RTP session.

6. The method of claim 2, wherein the RTP session is established based on a primary data channel established based on TCP;

the method further comprises the following steps:

under the condition that the correlation degree between the video stream data and the audio stream data contained in the media stream data to be transmitted is lower than the preset correlation degree, generating a second TCP data packet according to the audio stream data in the media stream data to be transmitted;

and sending the second TCP data packet to the vehicle based on the main data channel so as to instruct the vehicle to obtain corresponding audio stream data by analyzing the second TCP data packet, and playing the obtained audio stream data by using a vehicle-mounted component.

7. The method of claim 1, further comprising:

generating a real-time transport control protocol (RTCP) control packet, wherein the RTCP control packet comprises the sending condition information of the RTP data packet;

and sending the RTCP control packet to the vehicle based on the RTCP session so as to indicate the vehicle to obtain the sending condition information by analyzing the RTCP control packet, and determining packet loss information and/or network quality information according to the sending condition information.

8. The method of claim 7, wherein the RTCP session is established based on a primary data channel established based on TCP;

the method further comprises the following steps:

receiving a data packet retransmission instruction sent by the vehicle based on the main data channel and/or the RTCP session;

and carrying out retransmission operation on the target RTP data packet indicated by the data packet retransmission instruction.

9. The method according to claim 8, wherein the data packet retransmission instruction is generated by the vehicle according to packet loss information and current values of a preset vehicle performance index and/or network quality information, and the preset vehicle performance index is related to data processing capacity of the vehicle.

10. The method of claim 7, wherein the RTCP session is established based on a primary data channel established based on TCP;

the method further comprises the following steps:

receiving a data packet transmission instruction sent by the vehicle based on the main data channel and/or the RTCP session, wherein the data packet transmission instruction is generated by the vehicle according to a current value and/or network quality information of a preset vehicle performance index, the data packet transmission instruction is used for indicating a target video frame type, and the preset vehicle performance index is related to the data processing capacity of the vehicle;

generating a new RTP data packet according to video frame data which accords with the target video frame type in newly acquired media stream data to be transmitted;

and sending the new RTP data packet to the vehicle based on the RTP session so as to instruct the vehicle to obtain corresponding third media stream data by analyzing the new RTP data packet, and playing the third media stream data by utilizing a vehicle-mounted component.

11. A method of data transmission, comprising:

analyzing the RTP data packet to obtain corresponding first media stream data;

12. The method according to claim 11, wherein the mobile terminal generates the RTP data packet according to video stream data in the media stream data to be transmitted, on the condition that it is determined that the media stream data to be transmitted satisfies a preset transmission condition;

wherein the preset transmission condition comprises at least one of:

the media stream data to be transmitted only contains video stream data;

13. The method of claim 12, wherein the association degree between the video stream data and the audio stream data contained in the media stream data to be transmitted is lower than a preset association degree, comprising:

14. The method of claim 12, wherein the RTP session is established based on a primary data channel established based on the transmission control protocol TCP;

wherein the method further comprises:

receiving a first TCP data packet sent by the mobile terminal based on the main data channel, wherein the first TCP data packet is generated by the mobile terminal according to the media stream data to be transmitted under the condition that the media stream data to be transmitted does not meet the preset transmission condition;

obtaining corresponding second media stream data by analyzing the first TCP data packet;

and playing the second media stream data by utilizing the vehicle-mounted component.

15. The method of claim 14, further comprising:

16. The method of claim 12, wherein the RTP session is established based on a primary data channel established based on TCP;

the method further comprises the following steps:

receiving a second TCP data packet sent by the mobile terminal based on the main data channel, wherein the second TCP data packet is generated by the mobile terminal according to audio stream data in the media stream data to be transmitted under the condition that the correlation degree between the video stream data and the audio stream data contained in the media stream data to be transmitted is determined to be lower than a preset correlation degree;

corresponding audio stream data is obtained by analyzing the second TCP data packet;

the resulting audio stream data is played by the in-vehicle component.

17. The method of claim 11, further comprising:

receiving a real-time transport control protocol (RTCP) control packet sent by the mobile terminal based on an RTCP session, wherein the RTCP control packet comprises the sending condition information of the RTP data packet;

analyzing the RTCP control packet to obtain the sending condition information;

and determining packet loss information and/or network quality information according to the sending condition information.

18. The method of claim 17, wherein the RTCP session is established based on a primary data channel, which is established based on TCP;

the method further comprises the following steps:

and based on the data packet retransmission instruction sent to the mobile terminal by the main data channel and/or the RTCP session, instructing the mobile terminal to perform retransmission operation on the target RTP data packet indicated by the data packet retransmission instruction.

19. The method of claim 18, further comprising:

and generating the data packet retransmission instruction according to at least one item of packet loss information, and current values of preset vehicle performance indexes and/or network quality information, wherein the preset vehicle performance indexes are related to the data processing capacity of the vehicle.

20. The method according to claim 19, wherein the generating the data packet retransmission instruction according to the packet loss information and the current value of the preset vehicle performance index and/or the network quality information includes:

determining an RTP data packet which is not successfully received and belongs to video frame data of a target video frame type as a target RTP data packet according to packet loss information under the condition that the current value of the preset vehicle performance index is lower than a first preset performance threshold value and/or the current network quality is lower than the first preset quality threshold value according to the network quality information so as to generate a data packet retransmission instruction; the unsuccessfully received RTP data packet to which the unsuccessfully received RTP data packet belongs, wherein the target video frame type comprises a key frame.

21. The method of claim 17, wherein the RTCP session is established based on a primary data channel, which is established based on TCP;

the method further comprises the following steps:

generating a data packet transmission instruction according to a current value and/or network quality information of a preset vehicle performance index, wherein the data packet transmission instruction is used for indicating a target video frame type, and the preset vehicle performance index is related to the data processing capacity of the vehicle;

based on the main data channel and/or the RTCP session, sending the data packet transmission instruction to the mobile terminal to indicate the mobile terminal to generate a new RTP data packet according to video frame data which is in line with the target video frame type in newly acquired media stream data to be transmitted, and returning the new RTP data packet based on the RTP session;

obtaining corresponding third media stream data by analyzing the new RTP data packet;

and playing the third media stream data by utilizing the vehicle-mounted component.

22. The method of claim 21, wherein generating the packet transmission instruction according to the current value of the preset vehicle performance index and/or the network quality information comprises:

23. A data transmission apparatus comprising:

24. A data transmission apparatus comprising:

25. A mobile terminal, comprising:

at least one processor; and

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-10.

26. A vehicle, comprising:

at least one processor; and

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 11-22.

27. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-22.