CN117156190A

CN117156190A - Screen projection management method and device

Info

Publication number: CN117156190A
Application number: CN202310462584.3A
Authority: CN
Inventors: 曹庆峰; 张蓓; 屈婷; 周建春; 陈衍水
Original assignee: Honor Device Co Ltd
Current assignee: Honor Device Co Ltd
Priority date: 2023-04-21
Filing date: 2023-04-21
Publication date: 2023-12-01

Abstract

The embodiment of the application provides a screen projection management method and a screen projection management device, which are applied to terminal equipment, wherein the terminal equipment and the screen projection equipment are in a screen projection connection state, and the method comprises the following steps: the terminal equipment receives a first message from the screen throwing equipment, wherein the first message comprises a first method field, and the first method field is used for supporting file transmission between the equipment; the terminal equipment receives a second message from the screen throwing equipment, wherein the second message comprises the following components: a first subfield for setting file information, and a second subfield for setting a first parameter required at the time of file transfer; the terminal equipment receives a third message from the screen throwing equipment; the terminal equipment receives a fourth message from the screen throwing equipment; and the terminal equipment sends the OTA file to the screen throwing equipment according to the first parameter. Therefore, the version upgrading of the screen throwing device can be realized through the FILE_TRANPORT field between the terminal device and the screen throwing device by expanding the FILE_TRANPORT field in the RTSP, and the experience of using the screen throwing function by a user is improved.

Description

Screen projection management method and device

Technical Field

The application relates to the technical field of terminals, in particular to a screen projection management method and device.

Background

With the popularization and development of the internet, the screen projection technology is widely applied, and the screen projection refers to projecting media resources on one device to another device for playing.

In general, the terminal device may project the media resource into a projection device such as extended reality (XR) glasses, televisions, etc. based on a real-time streaming protocol (real time streaming protocol, RTSP), or the terminal device may send the media resource to a projection adapter based on RTSP, and the projection of the media resource into the projection device is continued through the projection adapter by the projection adapter.

However, the screen-cast device cannot implement device version upgrades in the screen-cast scenario described above.

Disclosure of Invention

The embodiment of the application provides a screen-throwing management method and device, which are used for solving the problem that a screen-throwing device is difficult to realize version upgrading.

In a first aspect, an embodiment of the present application provides a screen projection management method, which is characterized in that the method is applied to a terminal device, where the terminal device and the screen projection device are in a screen projection connection state, and the method includes: the terminal equipment sends a first instruction to the screen throwing equipment; the terminal equipment receives a first message from the screen throwing equipment, wherein the first message comprises a first method field, the first method field is used for supporting file transmission between equipment, and the first message is generated by the screen throwing equipment in response to a first instruction; the terminal equipment sends a second instruction to the screen throwing equipment, wherein the second instruction comprises a first method field; the terminal equipment receives a second message from the screen throwing equipment, wherein the second message comprises the following components: a first subfield for setting file information, and a second subfield for setting a first parameter required for file transmission, the second message being generated by the screen-casting device in response to the second instruction; the terminal equipment sends a third instruction to the screen throwing equipment, wherein the third instruction comprises: file information and a first subfield corresponding to an OTA file of an over-the-air technology; the terminal equipment receives a third message from the screen throwing equipment, wherein the third message is generated by the screen throwing equipment in response to a third instruction; the terminal equipment sends a fourth instruction to the screen throwing equipment, wherein the fourth instruction comprises a second subfield; the terminal equipment receives a fourth message from the screen throwing equipment, wherein the fourth message is generated by the screen throwing equipment in response to a fourth instruction; and the terminal equipment sends the OTA file to the screen throwing equipment according to the first parameter. Therefore, the version upgrading of the screen throwing device can be realized through the FILE_TRANPORT field between the terminal device and the screen throwing device by expanding the FILE_TRANPORT field in the RTSP, and the experience of using the screen throwing function by a user is improved.

The first instruction is an OPTIONS instruction described in the embodiment of the present application, the first method field is a file_tranport instruction, the second instruction is a file_tranport instruction, the first subfield is set_file_info, the second subfield is set_transport_para, the first parameter may be a FILE transfer parameter, the third instruction may be an instruction for setting FILE information in S607, the third message may be a message for indicating whether the FILE information is set successfully in S609, the fourth instruction may be an instruction for setting FILE transfer parameters in S612, and the fourth message may be a message for indicating whether the FILE transfer parameter is set successfully in S614.

In one possible implementation manner, the second message further includes a third subfield for acquiring a second parameter required in file transmission, and the method further includes: the terminal equipment sends a fifth instruction to the screen throwing equipment, wherein the fifth instruction comprises a third subfield; the terminal equipment receives a fifth message from the screen throwing equipment, wherein the fifth message comprises a second parameter, and the fifth message is generated by the screen throwing equipment in response to a fifth instruction; the terminal equipment sends a fourth instruction to the screen throwing equipment, which comprises the following steps: under the condition that the first parameter is determined from the second parameter and the third parameter, the terminal equipment sends a fourth instruction to the screen throwing equipment, the third parameter is a parameter required by the terminal equipment when the file transmission is carried out, the second parameter is a parameter required by the screen throwing equipment when the file transmission is carried out, and the first parameter is the same parameter in the second parameter and the third parameter. Therefore, the terminal equipment and the screen throwing equipment can realize the alignment of parameters required during file transmission through the get_transport_para field, and the stability of the transmission process is ensured. The third subfield is the get_transport_para field, and the fifth instruction is the instruction for acquiring the file transfer parameter in S610.

In one possible implementation manner, the fifth message further includes: the method is used for acquiring subfields corresponding to equipment information in the screen-throwing equipment, wherein the equipment information comprises one or more of the following: version information of the device, or information for indicating an OTA upgrade status.

In one possible implementation, the first parameter includes one or more of the following: a parameter for indicating whether retransmission is supported, a parameter for indicating a maximum length of a single packet, a parameter for indicating a timeout, a parameter for indicating whether encryption is supported, or a parameter for indicating version information.

In one possible implementation, the first message further includes: the second method field is used for supporting transmission of device information between devices, and the method further comprises: the terminal equipment sends a sixth instruction to the screen throwing equipment, wherein the sixth instruction comprises: a second method field; the terminal equipment receives a sixth message from the screen throwing equipment, wherein the sixth message comprises the following components: a third subfield for acquiring sensor data, the sixth message being generated by the screen-casting device in response to the sixth instruction; the terminal equipment sends a seventh instruction to the screen throwing equipment, wherein the seventh instruction comprises a third subfield; the terminal equipment receives a seventh message from the screen throwing equipment, wherein the seventh message comprises sensor data in the screen throwing equipment, and the seventh message is generated by the screen throwing equipment in response to a seventh instruction. Therefore, the device_MANAGEMENT field in RTSP can be expanded between the terminal DEVICE and the screen throwing DEVICE to acquire the sensor data by the terminal DEVICE, so that the terminal DEVICE can adjust the screen throwing picture according to the sensor data.

The second method field may be a device_management field, the sixth instruction may be a DEVICE MANAGEMENT instruction described in S407, the third subfield may be a get_sensor field, and the seventh instruction may be an instruction for acquiring a sensor parameter described in S409.

In one possible implementation, the method further includes: the terminal equipment determines the rotation angle and the displacement of the screen throwing equipment through sensor data, wherein the sensor data comprises one or more of the following: acceleration data of the device, angular velocity data of the device, or magnetometer data of the device; the terminal equipment adjusts the screen projection picture by utilizing the rotation angle and the displacement, and sends the adjusted screen projection picture to the screen projection equipment. Therefore, the terminal equipment can utilize the sensor data generated by the user using the screen throwing equipment to adjust the screen throwing picture in real time, so that the screen throwing picture is more stereoscopic.

In one possible implementation, the first message further includes: the third method field is used for supporting encryption capability negotiation between devices, and the method further comprises: the terminal equipment sends an eighth instruction to the screen throwing equipment, wherein the eighth instruction comprises a third method field and a first encryption method supported by the terminal equipment; the terminal equipment receives an eighth message from the screen throwing equipment, wherein the eighth message comprises a second encryption method supported by the screen throwing equipment, and the eighth message is generated by the screen throwing equipment in response to an eighth instruction; the terminal equipment determines the same third encryption method in the first encryption method and the second encryption method; the terminal equipment sends a third instruction to the screen throwing equipment, which comprises the following steps: the terminal equipment encrypts the file information corresponding to the OTA file by using a third encryption method, and sends the encrypted file information corresponding to the OTA file to the screen throwing equipment through a third instruction. In this way, security of data transmission based on RTSP can be provided between the terminal device and the screen-casting device by extending the encyption_control field in RTSP.

The third method field may be an ENCRYPTION CONTROL field, and the eighth instruction may be an ENCRYPTION CONTROL instruction in S405.

In one possible implementation, the second message further includes: the fourth subfield for acquiring the file list and the fifth subfield for acquiring the log file, the method further comprises: the terminal equipment sends a ninth instruction to the screen throwing equipment, wherein the ninth instruction comprises a fourth subfield; the terminal equipment receives a ninth message from the screen throwing equipment, wherein the ninth message comprises one or more of the following: the file list, a structure body where any file in the file list is located, a file name of any file, a file type of any file and a check value corresponding to any file, and a ninth message is generated by the screen throwing device in response to a ninth instruction; the terminal equipment sends a tenth instruction to the screen throwing equipment, wherein the tenth instruction comprises a fifth subfield; the terminal equipment receives a log file from the screen throwing equipment, wherein the log file is sent by the screen throwing equipment in response to a tenth instruction. In this way, the transmission of the log FILE can be realized through the extended file_tranport field in the RTSP between the terminal equipment and the screen throwing equipment, so that the terminal equipment can check the communication state in the screen throwing process by using the log FILE.

In one possible implementation, the screening device is an augmented reality XR glasses.

In one possible implementation, the terminal device is a device that does not support the DP protocol.

In one possible implementation manner, the terminal device sends a third instruction to the screen throwing device, including: and under the condition that the terminal equipment does not support the DP protocol, the terminal equipment sends a third instruction to the screen throwing equipment. In this way, the terminal device can realize the transmission of the OTA file through the RTSP provided in the embodiment of the application under the condition of not supporting the DP protocol.

In one possible implementation manner, the terminal device has a serial bus USB Type-C interface, and the method further includes: the terminal equipment establishes connection with the screen throwing equipment based on the USB Type-C interface; and under the condition that the terminal equipment supports the DP protocol, the terminal equipment sends the OTA file to the screen throwing equipment based on the DP protocol. Therefore, under the condition that the terminal equipment supports the DP protocol, the terminal equipment can transmit OTA files through the DP protocol, and the use scene of screen projection is increased.

In a second aspect, an embodiment of the present application provides a screen projection management method, which is applied to a screen projection device, where the screen projection device and a terminal device are in a screen projection connection state, and the method includes: the screen throwing device establishes screen throwing connection with the terminal device, and the screen throwing device and the terminal device are in the same WIFI network; the screen throwing equipment receives a first instruction from the terminal equipment; the screen throwing device sends a first message to the terminal device, wherein the first message comprises a first method field, and the first method field is used for supporting file transmission between devices; the first message is generated by the screen throwing equipment in response to a first instruction; the screen throwing equipment receives a second instruction from the terminal equipment, wherein the second instruction comprises a first method field; the screen throwing device sends a second message to the terminal device, wherein the second message comprises: a first subfield for setting file information, and a second subfield for setting a first parameter required at the time of file transfer; the second message is generated by the screen throwing equipment in response to the second instruction; the screen throwing device receives a third instruction from the terminal device, wherein the third instruction comprises the following steps: file information and a first subfield corresponding to an OTA file of an over-the-air technology; the screen throwing device sends a third message to the terminal device; the third message is generated by the screen throwing equipment in response to the third instruction; the screen throwing equipment receives a fourth instruction from the terminal equipment, wherein the fourth instruction comprises a second subfield; the screen throwing device sends a fourth message to the terminal device; the fourth message is generated by the screen throwing equipment in response to the fourth instruction; and the screen throwing equipment receives the OTA file sent by the terminal equipment according to the first parameter.

In one possible implementation, the first message further includes: the second method field is used for supporting transmission of device information between devices, and the method further comprises: the screen throwing device receives a sixth instruction from the terminal device, wherein the sixth instruction comprises: a second method field; the screen throwing device sends a sixth message to the terminal device, wherein the sixth message comprises: a third subfield for acquiring sensor data, the sixth message being generated by the screen-casting device in response to the sixth instruction; the screen throwing device receives a seventh instruction from the terminal device, wherein the seventh instruction comprises a third subfield; the screen throwing device sends a seventh message to the terminal device, wherein the seventh message comprises sensor data in the screen throwing device, and the seventh message is generated by the screen throwing device in response to a seventh instruction.

In one possible implementation, the method further includes: the screen throwing device receives an adjusted screen throwing picture from the terminal device, wherein the adjusted screen throwing picture is obtained by adjusting the screen throwing picture by the terminal device based on a rotation angle and a displacement, the rotation angle and the displacement are determined based on sensor data, and the sensor data comprise one or more of the following: acceleration data of the device, angular velocity data of the device, or magnetometer data of the device.

In one possible implementation, the first message further includes: the third method field is used for supporting encryption capability negotiation between devices, and the method further comprises: the screen throwing equipment receives an eighth instruction from the terminal equipment, wherein the eighth instruction comprises a third method field and a first encryption method supported by the terminal equipment; the screen throwing device sends an eighth message to the terminal device, wherein the eighth message comprises a second encryption method supported by the screen throwing device, the third instruction comprises file information corresponding to the encrypted OTA file, the file information corresponding to the encrypted OTA file is obtained by encrypting the file information corresponding to the OTA file by using a third encryption method, the third encryption method is the same method in the first encryption method and the second encryption method, and the eighth message is generated by the screen throwing device in response to the eighth instruction.

In one possible implementation, the second message further includes: the fourth subfield for acquiring the file list and the fifth subfield for acquiring the log file, the method further comprises: the screen throwing device receives a ninth instruction from the terminal device, wherein the ninth instruction comprises a fourth subfield; the screen throwing device sends a ninth message to the terminal device, wherein the ninth message comprises one or more of the following: the file list, a structure body where any file in the file list is located, a file name of any file, a file type of any file and a check value corresponding to any file, and a ninth message is generated by the screen throwing device in response to a ninth instruction; the screen throwing device receives a tenth instruction from the terminal device, wherein the tenth instruction comprises a fifth subfield; the screen throwing device sends a log file to the terminal device according to the first parameter, wherein the log file is sent by the screen throwing device in response to a tenth instruction.

In one possible implementation manner, the terminal device is a device supporting a DP protocol, and the terminal device has a serial bus USB Type-C interface, and the method further includes: the screen throwing device establishes screen throwing connection with the terminal device based on the USB Type-C interface, and receives an OTA file from the terminal device, wherein the OTA file is sent by the terminal device based on a DP protocol.

In a third aspect, an embodiment of the present application provides a screen projection management system, where the screen projection management system includes a terminal device and a screen projection device, where the terminal device and the screen projection device are both in a screen projection connection state; the terminal equipment sends a first instruction to the screen throwing equipment; the screen throwing equipment sends a first message to the terminal equipment; the first message comprises a first method field, the first method field is used for supporting file transmission between devices, and the first message is generated by the device to be screen-thrown in response to a first instruction; the terminal equipment sends a second instruction to the screen throwing equipment; the second instruction comprises a first method field; the screen throwing device sends a second message to the terminal device; the second message includes: a first subfield for setting file information, and a second subfield for setting a first parameter required for file transmission, wherein the second message is generated by the equipment to be screen-thrown in response to the second instruction; the terminal equipment sends a third instruction to the screen throwing equipment; the third instruction includes: file information and a first subfield corresponding to an OTA file of an over-the-air technology; the screen throwing device sends a third message to the terminal device, wherein the third message is generated by the screen throwing device to be thrown in response to a third instruction; the terminal equipment sends a fourth instruction to the screen throwing equipment; the fourth instruction includes a second subfield: the screen throwing device sends a fourth message to the terminal device, wherein the fourth message is generated by the screen throwing device to be thrown in response to a fourth instruction; and the terminal equipment sends the OTA file to the screen throwing equipment according to the first parameter.

In a fourth aspect, an embodiment of the present application provides a screen-shot management apparatus, including a unit for performing the method in the first aspect, or a unit for performing the method in the second aspect.

The method described in the first aspect or the second aspect may be implemented by hardware, or may be implemented by executing corresponding software by hardware. The hardware or software includes one or more modules or units corresponding to the functions described above. Such as a processing module or unit, a display module or unit, etc.

In a fifth aspect, the present application provides a terminal device comprising a memory and one or more processors; wherein the memory is for storing computer program code, the computer program code comprising computer instructions; the computer instructions, when executed by a processor, cause a terminal device to perform the method of any of the first aspects.

In a sixth aspect, the present application provides a computer-readable storage medium comprising computer instructions. When executed on a terminal device (e.g. a computer) the computer instructions cause the terminal device to perform the method as in the first aspect and any one of its possible designs.

In a seventh aspect, the application provides a computer program product for causing a computer to carry out the method as in the first aspect and any one of its possible design approaches when the computer program product is run on a computer.

In an eighth aspect, the present application provides a chip system comprising one or more interface circuits and one or more processors. The interface circuit and the processor are interconnected by a wire. The chip system described above may be applied to a terminal device comprising a communication module and a memory. The interface circuit is for receiving signals from the memory of the terminal device and transmitting the received signals to the processor, the signals including computer instructions stored in the memory. When the processor executes the computer instructions, the terminal device may perform the method as in the first aspect and any one of its possible designs.

It should be understood that the second to eighth aspects of the present application correspond to the technical solutions of the first aspect of the present application, and the advantages obtained by each aspect and the corresponding possible embodiments are similar, and are not repeated.

Drawings

FIG. 1 is a schematic view of a scene provided in an embodiment of the present application;

fig. 2 is a schematic hardware structure of a terminal device according to an embodiment of the present application;

Fig. 3 is a schematic software structure of a terminal device according to an embodiment of the present application;

fig. 4 is a flow chart of a screen projection management method according to an embodiment of the present application;

FIG. 5 is a schematic flow chart of a screen media resource according to an embodiment of the present application;

fig. 6 is a schematic flow chart for implementing OTA upgrading according to an embodiment of the present application;

FIG. 7 is a schematic diagram of an interface for version upgrade according to an embodiment of the present application;

FIG. 8 is a schematic diagram of an interface for another version upgrade provided by an embodiment of the present application;

FIG. 9 is a schematic flow chart of log file transmission according to an embodiment of the present application;

fig. 10 is a schematic flow chart of determining whether to use a DP protocol according to an embodiment of the present application;

fig. 11 is a schematic structural diagram of a screen projection management device according to an embodiment of the present application;

fig. 12 is a schematic hardware structure of another terminal device according to an embodiment of the present application.

Detailed Description

In order to clearly describe the technical solution of the embodiments of the present application, in the embodiments of the present application, the words "first", "second", etc. are used to distinguish the same item or similar items having substantially the same function and effect. For example, the first value and the second value are merely for distinguishing between different values, and are not limited in their order. It will be appreciated by those of skill in the art that the words "first," "second," and the like do not limit the amount and order of execution, and that the words "first," "second," and the like do not necessarily differ.

In the present application, the words "exemplary" or "such as" are used to mean serving as an example, instance, or illustration. Any embodiment or design described herein as "exemplary" or "for example" should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete fashion.

In the present application, "at least one" means one or more, and "a plurality" means two or more. "and/or", describes an association relationship of an association object, and indicates that there may be three relationships, for example, a and/or B, and may indicate: a alone, a and B together, and B alone, wherein a, B may be singular or plural. The character "/" generally indicates that the context-dependent object is an "or" relationship. "at least one of" or the like means any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (one) of a, b, or c may represent: a, b, c, a and b, a and c, b and c, or a, b and c, wherein a, b, c may be single or plural.

To facilitate an understanding of the embodiments of the present application, some of the words involved in the present application will be briefly described.

1、RTSP

RTSP can be understood as a projection protocol. In particular, RTSP is an application layer protocol in the transmission control protocol/internet protocol (transmission control protocol/internet protocol, TCP/IP) protocol system, and defines how efficiently a one-to-many application program transmits multimedia data over an IP network. All operations in RTSP are accomplished by a message reply mechanism between the server and the client, where the message includes both a request and a reply, RTSP is a symmetric protocol, and both the client and the server can send and reply requests. Among them, various method fields involved in RTSP can be referred to the corresponding embodiments of table 1.

Table 1 shows a schematic table of method fields in RTSP

The method fields shown in table 1 may be standard protocol fields specified in RTSP. Wherein, C in the direction may be a client (client), such as a screen-throwing device described in the embodiment of the present application, and S in the direction may be a server (service), such as a terminal device described in the embodiment of the present application; the P in the object may be a presentation or may also be understood as a media Stream, and the S in the object may be a Stream or may also be understood as a data Stream.

For example, the server may implement screen casting for the client based on the field indicated in the RTSP, and data transmission during the screen casting.

2. Over the air technology (over the air technology, OTA)

OTA can be a technology for implementing remote management of terminal equipment or various types of software in terminal equipment through an air interface of mobile communication.

3. Screen throwing adapter

A projection switch, or also referred to as a projection switch, is a tool that enables projection of a resource in a service to another client, and may be used not only to transmit data to a client, but also to power a client. Wherein the projected resources may include: media resources, and data resources, etc.

In the embodiment of the application, under the condition that the server is a mobile phone and the client is an XR (X-ray) glasses, the mobile phone and the XR glasses can be connected through the screen throwing adapter, and in the screen throwing process, the mobile phone can send media resources to the screen throwing adapter and further transmit the media resources to the XR glasses through the screen throwing adapter.

In order to better understand the method according to the embodiment of the present application, an application scenario to which the embodiment of the present application is applicable is first described below.

Exemplary, fig. 1 is a schematic view of a scenario provided in an embodiment of the present application. As shown in fig. 1, the scene may include: a cell phone 101, a projection screen adapter 102 and XR glasses 103. The mobile phone 101 and the screen-throwing adapter 102 may be connected through a mobile hotspot (WIFI), and the screen-throwing adapter 102 and the XR glasses may be connected through a universal serial bus (universal serial bus, USB) Type-C interface in the screen-throwing adapter 102.

In the embodiment corresponding to fig. 1, when the mobile phone 101, the screen throwing adapter 102 and the XR glasses 103 are all in a connection state, the mobile phone 101 may send the media resource in the mobile phone 101 to the screen throwing adapter 102 through the WIFI screen throwing manner based on the wireless display (miracast) standard, and the screen throwing adapter 102 may send the media resource to the XR glasses 103 through the USB Type-C interface, so that the user may view the media resource in the mobile phone in the XR glasses.

In a possible implementation manner, the mobile phone 101 and the XR glasses 103 may also establish a connection in a wireless manner, and implement screen projection when the connection is established.

The mobile phone 101 may be a service end (service) described in the embodiment of the present application, the XR glasses 103 may be a client end (client) described in the embodiment of the present application, and the connection between the mobile phone 101 and the XR glasses 103 may be established based on a screen throwing adapter.

However, there are some unresolved problems in the process of screen projection based on the connection method described in fig. 1.

Problem one: the terminal device cannot manage information of various types of devices in the screen throwing device. It can be understood that, in the case where the version information of the screen throwing device, the device upgrade state, or the device information such as sensor data generated when the user wears the screen throwing device is not transmitted to the terminal device, the terminal device cannot perform planned maintenance, management and control on the screen throwing device according to the device information of the screen throwing device.

For example, since the content displayed in the screen-throwing device is generated when the screen is thrown through the terminal device, the content displayed in the screen-throwing device does not change according to the shaking condition in the case that the sensor data generated when the screen-throwing device is shaken is not transmitted to the terminal device no matter how the screen-throwing device is worn by the user.

In view of the above, an embodiment of the present application provides a screen-projection MANAGEMENT method, in which a terminal DEVICE may obtain DEVICE information of a screen-projection DEVICE through an instruction related to DEVICE MANAGEMENT (device_management), and implement DEVICE MANAGEMENT and maintenance on the screen-projection DEVICE through the DEVICE information of the screen-projection DEVICE, and may also adjust a media resource of the screen-projection according to the DEVICE information.

And a second problem: the terminal equipment cannot carry out OTA upgrading on the screen throwing equipment. It can be understood that, in the case that file data transmission is not supported between the terminal device and the screen-throwing device, the terminal device cannot realize OTA upgrading for the screen-throwing device.

In view of the second problem, the embodiment of the present application provides a screen-casting management method, so that a terminal device may send FILE information to a screen-casting device through an instruction related to FILE transfer (file_tranport), learn FILE transfer capability of the screen-casting device, and send an OTA FILE to the screen-casting device through the instruction to implement device upgrade of the screen-casting device.

It can be understood that a terminal device that does not support the Display Port (DP) protocol may establish a connection with a screen-throwing device based on the WIFI screen-throwing manner, but this connection manner does not support the screen-throwing device to reversely transmit data to the terminal device, which further results in the occurrence of the first problem and the second problem. The DP protocol is a digital video transmission protocol, and terminal equipment supporting the DP protocol can realize the screen-throwing connection with other screen-throwing equipment, and can also realize video data transmission, file transmission and the like.

The terminal device or the screen-cast device may be referred to as a terminal (terminal), a User Equipment (UE), a Mobile Station (MS), a Mobile Terminal (MT), or the like.

The terminal device may be a device having media resources, such as: mobile phone), smart tv, wearable device, tablet (Pad), computer with wireless transceiver function, etc.

The terminal device described in the embodiments of the present application may be a device that initiates screen projection, and the screen projection device may be a device that receives screen projection content, for example, the terminal device may send a media resource to the screen projection device through the screen projection, where the screen projection device receives the media resource and displays the media resource in the display screen.

The screen-casting device may be a device for receiving media assets, such as: smart televisions, car screens, virtual Reality (VR) terminals, augmented reality (augmented reality, AR) terminals such as XR glasses, devices with display screens, and the like. The embodiment of the application does not limit the specific technology and the specific equipment form adopted by the terminal equipment or the screen throwing equipment.

It can be understood that the screen projection management method provided by the embodiment of the application can also be applied to any screen projection scene in which the terminal equipment and the screen projection equipment realize two-way communication, and the screen projection management method is not limited in the embodiment of the application.

In order to better understand the embodiments of the present application, the structure of the terminal device of the embodiments of the present application is described below. Fig. 2 is a schematic hardware structure of a terminal device according to an embodiment of the present application.

It may be understood that the terminal device described in the embodiment corresponding to fig. 2 may be a terminal device or may also be a screen-throwing device, and the hardware structure in the screen-throwing device may be different from that of the terminal device, which is not limited in the embodiment of the present application. Wherein, throw the screen equipment and can include: the sensor modules such as acceleration sensor, gyro sensor, and magnetic sensor are used to obtain acceleration data, angular velocity data (or angular acceleration data), magnetometer data, and the like.

The terminal device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) interface 130, a charge management module 140, a power management module 141, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, keys 190, an indicator 192, a camera 193, a display 194, and the like.

One or more of the following may be included in the sensor module 180, for example: pressure sensors, gyroscopic sensors, barometric pressure sensors, magnetic sensors, acceleration sensors, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, or bone conduction sensors, among others.

It will be appreciated that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the terminal device. In other embodiments of the application, the terminal device may include more or less components than illustrated, or certain components may be combined, or certain components may be split, or different arrangements of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

Processor 110 may include one or more processing units. Wherein the different processing units may be separate devices or may be integrated in one or more processors. A memory may also be provided in the processor 110 for storing instructions and data.

The USB interface 130 is an interface conforming to the USB standard specification, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, or the like. The USB interface 130 may be used to connect a charger to charge a terminal device, or may be used to transfer data between the terminal device and a peripheral device. And can also be used for connecting with a headset, and playing audio through the headset. The interface may also be used to connect other devices, such as AR devices, etc.

The charge management module 140 is configured to receive a charge input from a charger. The charger can be a wireless charger or a wired charger. The power management module 141 is used for connecting the charge management module 140 and the processor 110.

The wireless communication function of the terminal device may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, and the baseband processor, etc.

The antennas 1 and 2 are used for transmitting and receiving electromagnetic wave signals. Antennas in the terminal device may be used to cover single or multiple communication bands. Different antennas may also be multiplexed to improve the utilization of the antennas.

The mobile communication module 150 may provide a solution for wireless communication including 2G/3G/4G/5G or the like applied on a terminal device. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA), etc. The mobile communication module 150 may receive electromagnetic waves from the antenna 1, perform processes such as filtering, amplifying, and the like on the received electromagnetic waves, and transmit the electromagnetic waves to the modem for demodulation.

The wireless communication module 160 may provide solutions for wireless communication including wireless local area network (wirelesslocal area networks, WLAN) (e.g., wireless fidelity (wireless fidelity, wi-Fi) network), bluetooth (BT), global navigation satellite system (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), etc. as applied on a terminal device.

The modem may include a modulator and a demodulator. The modulator is used for modulating the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used for demodulating the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low frequency baseband signal to the baseband processor for processing. The low frequency baseband signal is processed by the baseband processor and then transferred to the application processor. The application processor outputs sound signals through an audio device (not limited to speakers, receivers, etc.), or displays images or video through a display screen 194. In some embodiments, the modem may be a stand-alone device. In other embodiments, the modem may be provided in the same device as the mobile communication module 150 or other functional module, independent of the processor 110.

The terminal device implements display functions through a GPU, a display screen 194, an application processor, and the like. The GPU is a microprocessor for image processing, and is connected to the display 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering.

The display screen 194 is used to display images, videos, and the like. The display 194 includes a display panel. In some embodiments, the terminal device may include 1 or N display screens 194, N being a positive integer greater than 1.

The terminal device may implement photographing functions through an ISP, a camera 193, a video codec, a GPU, a display screen 194, an application processor, and the like.

The camera 193 is used to capture still images or video. In some embodiments, the terminal device may include 1 or N cameras 193, N being a positive integer greater than 1.

The external memory interface 120 may be used to connect an external memory card, such as a Micro SD card, to realize expansion of the memory capability of the terminal device. The external memory card communicates with the processor 110 through an external memory interface 120 to implement data storage functions. For example, files such as music, video, etc. are stored in an external memory card.

The internal memory 121 may be used to store computer-executable program code that includes instructions. The internal memory 121 may include a storage program area and a storage data area.

The terminal device may implement audio functions through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, an application processor, and the like. Such as music playing, recording, etc.

The audio module 170 is used to convert digital audio information into an analog audio signal output and also to convert an analog audio input into a digital audio signal. The speaker 170A, also referred to as a "horn," is used to convert audio electrical signals into sound signals. The terminal device can listen to music through the speaker 170A or listen to hands-free calls. A receiver 170B, also referred to as a "earpiece", is used to convert the audio electrical signal into a sound signal. When the terminal device picks up a call or voice message, the voice can be picked up by placing the receiver 170B close to the human ear. The earphone interface 170D is used to connect a wired earphone. Microphone 170C, also referred to as a "microphone" or "microphone", is used to convert sound signals into electrical signals. In the embodiment of the present application, the terminal device may have a microphone 170C.

The keys 190 include a power-on key, a volume key, etc. The keys 190 may be mechanical keys. Or may be a touch key. The terminal device may receive key inputs, generating key signal inputs related to user settings of the terminal device and function control. The indicator 192 may be an indicator light, may be used to indicate a state of charge, a change in charge, a message indicating a missed call, a notification, etc.

In addition, above the above components, the device also runs an operating system. Such as an iOS operating system, an android operating system, or a Windows operating system, etc. Running applications may be installed on the operating system.

The software system of the terminal device may adopt a layered architecture, an event driven architecture, a microkernel architecture, a microservice architecture, a cloud architecture, or the like, which will not be described herein.

Fig. 3 is a schematic software structure of a terminal device according to an embodiment of the present application.

As shown in fig. 3, the layered architecture may divide the software into several layers, each with distinct roles and branches. The layers communicate with each other through a software interface. In some embodiments, android systems are divided into multiple layers, from top to bottom, an Application (APP) layer, an application framework (frame work) layer, a system library, a kernel (kernel) layer, and so on.

As shown in fig. 3, the application layer may include: the applications such as viewing application, album, camera, telephone, bluetooth, etc. in the embodiment of the present application, other applications included in the application layer are not particularly limited. The video watching application can realize the functions of screen connection between the screen throwing equipment and the terminal equipment, setting a video watching mode, carrying out version upgrading on the screen throwing equipment and the like.

The application framework layer is used for providing application programming interfaces (application programming interface, APIs) and programming frameworks for various types of applications in the application layer, and some predefined functions can be included in the application framework layer.

As shown in fig. 3, the application framework layer may include: in a possible implementation manner, one or more of the following may be further included in the application framework layer: RTSP middleware, window manager, content provider, resource manager, view system, or notification manager, etc.

The RTSP middleware is used for controlling message interaction between the terminal equipment and the screen throwing equipment according to the indication of the video watching application. For example, in the case where the terminal device establishes a WIFI screen connection with the screen device, the viewing application may send an interaction request to the RTSP middleware, and the RTSP middleware may perform handshaking of the method field between the step shown in S401 (or S601) and the screen device when receiving the interaction request. Alternatively, the RTSP middleware may perform the step shown in S605 when receiving an instruction for OTA file upgrade initiated by the viewing application, perform the step shown in S409 when receiving an instruction for acquiring sensor data initiated by the viewing application, perform the step shown in S903 when receiving an instruction for acquiring log file initiated by the viewing application, and so on.

In a possible implementation manner, the RTSP middleware may be disposed in an application framework layer, or may be disposed in a system library or a kernel layer, which is not specifically limited in the embodiment of the present application.

The window manager is used for managing window programs. The window manager may invoke the display to display the corresponding content according to the indication of the viewing application.

The content provider is used to store and retrieve data and make such data accessible to applications. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phonebooks, etc.

The resource manager provides various resources to the application program, such as localization strings, icons, images, layout files, video files, and the like.

The view system includes visual controls, such as controls to display text, controls to display images, and the like. The view system may be used to build applications. The display interface may be composed of one or more views. For example, a display interface including a text messaging icon may include a view displaying text and a view displaying an image.

The notification manager allows the application to display notification information in a status bar, can be used to communicate notification type messages, can automatically disappear after a short dwell, and does not require user interaction. Such as notification manager is used to inform that the download is complete, message alerts, etc.

The notification manager may also be a notification in the form of a chart or scroll bar text that appears on the system top status bar, such as a notification of a background running application, or a notification that appears on the screen in the form of a dialog window. For example, a text message is presented in a status bar, a prompt tone is emitted, vibration is generated, and an indicator light blinks.

The system library may include a plurality of functional modules. For example: surface manager (surface manager), three-dimensional graphics processing library, two-dimensional graphics engine, media libraries (media libraries), and the like.

The surface manager is used for managing the display subsystem and providing fusion of 2D and 3D layers for a plurality of application programs.

The three-dimensional graphic processing library is used for realizing the functions of three-dimensional graphic drawing, image rendering, synthesis, layer processing and the like.

The two-dimensional graphics engine is a drawing engine for 2D drawing.

Media libraries support a variety of commonly used audio, video format playback and recording, still image files, and the like. The media library may support a variety of audio and video encoding formats, such as MPEG4, h.264, MP3, AAC, AMR, JPG, PNG, etc.

The kernel layer is a layer between hardware and software. The kernel layer includes one or more of the following, for example: display drive, camera drive, audio drive, sensor drive, etc.

It may be understood that other layers and/or other modules may also be included in the software structure of the terminal device, and the specific result of the terminal device is not limited in the embodiment of the present application.

The following describes the technical scheme of the present application and how the technical scheme of the present application solves the above technical problems in detail with specific embodiments. The following embodiments may be implemented independently or combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

The screen-throwing management method described in the embodiment of the application can realize the expansion of the method field in RTSP, so that the expanded method field can support the terminal equipment to carry out equipment management on the screen-throwing equipment, and the terminal equipment to carry out OTA upgrading and file data management on the screen-throwing equipment. For example, the extended method field in RTSP may be seen in table 2.

Table 2 shows an expanded method field schematic in RTSP

Method field	Direction	Object(s)	Requirements for
				ENCRYPTION CONTROL (ENCRYPTION_CONTROL)	C->S,S->C	S	Priority (priority)
DEVICE MANAGEMENT (device_management)	C->S,S->C	S	proprietary
				File transfer (FILE_TRANPORT)	S->C,C->S	S	Proprietary

As shown in table 2, the ENCRYPTION CONTROL is used for making CONTROL instructions and ENCRYPTION negotiation in the scenes of OTA upgrade and the like; device_management is used for realizing DEVICE capability query, sensor data query, version query and the like in DEVICE MANAGEMENT; FILE _ TRANPORT is used to enable the transfer of FILEs such as firmware upgrade FILEs (e.g., OTA upgrade FILEs), log (log) FILEs, and memory snapshot FILEs (or referred to as dump FILEs).

Aiming at the first problem: the terminal DEVICE cannot manage the information of various types of DEVICEs in the screen-throwing DEVICE, and the embodiment of the application can use the device_management field to realize DEVICE MANAGEMENT, and particularly, refer to the embodiment corresponding to fig. 4.

Fig. 4 is a schematic flow chart of a screen projection management method according to an embodiment of the present application. In the embodiment corresponding to fig. 4, the screen-projection management method may relate to a terminal device and a screen-projection device, and the manner in which the terminal device establishes a connection with the screen-projection device may refer to the embodiment corresponding to fig. 1.

As shown in fig. 4, the procedure of the screen-projection management method is as follows:

s401, the terminal equipment sends an OPTIONS instruction to the screen throwing equipment.

The OPTIONS instruction (or referred to as a first instruction) may include: the OPTIONS field, the OPTIONS instruction is used to obtain the method field supported by the device.

The terminal device may send an operation instruction to the screen device after the WIFI screen connection is established with the screen device, so as to implement handshaking of method fields in RTSP, where handshaking may be understood as a way of confirming identities of the terminal device and the screen device. The manner in which the terminal device establishes connection with the screen-projecting device may refer to the corresponding embodiment of fig. 1, which is not described herein.

An example is given of a message handshaking based on an OPTIONS instruction:

Request：OPTIONS*RTSP/1.0\r\n

Date：Wed,11Dec 2019 08:31:54+0000\r\n

Server：stagefright/1.2(Linux；Andriod 8.1.0)\r\n

CSeq：1\r\n

Require：org.wfa.wfd1.0\r\n

\r\n

wherein, the Request may indicate that the current message is a Request instruction, RTSP/1.0 may be a version of RTSP, date may be a Date of message transmission, server in the OPTIONS instruction may indicate a system of the device, CSeq is used to indicate a sequence number of the message, and Request in the OPTIONS instruction may be used as a tag (tag) to query capabilities supported by the peer.

S402, the screen throwing device returns a method field supported by the screen throwing device to the terminal device.

The method field supported by the screen-throwing device may also be called a first message. The response message returned based on the OPTIONS instruction in S403 may be adapted to:

Response：RTST/1.0 200OK\r\n

CSeq：1\r\n

Public：org.wfa.wfd1.0，GET_PARAMETER，SET_PARAMETER，ENCRYPTION_CONTROL，DEVICE_MANAGEMENT，FILE_TRANPORT\r\n

\r\n

wherein Response may indicate that the current message is a Response, 200 may be used to indicate a status code, OK may be used to indicate that the status description is successful, and Public may include a method field supported by the screening device.

It will be appreciated that the method fields included in Public may be related to functions that may be implemented by the screening device. In a possible implementation manner, in order to implement more functions, public may also include a method field indicated in table 1 and other possible method fields, and in this embodiment of the present application, the type of method field supported by the screen capturing device and the number of method fields supported by the screen capturing device are not specifically limited.

Optionally, S403, the screen throwing device sends an OPTIONS instruction to the terminal device.

The OPTIONS instruction is used to obtain the method field supported by the device.

S404, the terminal equipment returns a method field supported by the terminal equipment to the screen throwing equipment.

The step S403 to S404 may be optional steps, and the specific content of the method field supported by the terminal device acquired by the screen capturing device described in the step S403 to S404 may be similar to the specific content of the method field supported by the terminal device acquired by the screen capturing device described in the step S401 to S402, which is not described herein again.

It can be understood that the steps shown in S401-S402 are necessary steps for data communication between the terminal device and the screen throwing device, and after the terminal device and the screen throwing device complete handshaking of the method fields based on the steps shown in S401-S402, the terminal device can implement corresponding functions based on the method fields supported by the common device of the two devices.

For example, the terminal device may implement the acquisition of the encryption capability of the communication pipe of the screen-throwing device (see steps shown in S405-S406), the acquisition of the sub-capability set supported by the screen-throwing device (see steps shown in S407-S408), the acquisition of the data in the screen-throwing device (see steps shown in S409-S410), the OTA upgrade of the screen-throwing device (see the corresponding embodiment of fig. 6), the acquisition of the file in the screen-throwing device (see the corresponding embodiment of fig. 9), and so on.

S405, the screen throwing device sends an encryption control instruction to the terminal device.

The ENCRYPTION_CONTROL instruction may include: the ENCRYPTION_CONTROL field, the ENCRYPTION_CONTROL instruction is used to obtain the pipeline ENCRYPTION capabilities supported by the device.

The pipeline encryption capability is understood to be an encryption capability for guaranteeing the security of message transmission between the terminal equipment and the screen throwing equipment, and the pipeline encryption capability can indicate a specific encryption mode.

Illustratively, an example is given of a message that performs an ENCRYPTION handshake based on an ENCRYPTION CONTROL instruction:

Request：ENCRYPTION_CONTROL rtsp://localhost/wfd1.0 RTSP/1.0\r\n

Date：Wed,11Dec 2019 08:31:54+0000\r\n

Server：stagefright/1.2(Linux；Andriod 8.1.0)\r\n

CSeq：2\r\n

Content-type：text/parameters

Content-length：83

\r\n

Line-based text data：text/parameters(2lines)

AES128\r\n

AES256\r\n

the Content-type is used for indicating the Content type, the Content-length is used for indicating the Content length, and a specific parameter can be included in the Line-based text data. The advanced encryption standard (advanced encryption standard, AES) among the parameters is a symmetric encryption algorithm, i.e. the same key is used for encryption and decryption, AES128 is an encryption algorithm with a key length of 16 bytes (bytes), and AES256 is an encryption algorithm with a key length of 32 bytes.

S406, the screen throwing device returns the pipeline encryption capability supported by the screen throwing device to the terminal device.

The response message returned based on the encryption_control instruction in S405 may be adapted to be:

Response：RTST/1.0 200OK\r\n

CSeq：2\r\n

Content-type：text/parameters

Content-length：848

\r\n

Line-based text data：text/parameters(1lines)

AES128：CTR；ECB；OFB；CFB\r\n

Wherein, the CTR may be a computer mode (CTR), the ECB may be a codebook mode (electronic codebook book, ECB), the OFB may be an output feedback mode (OFB), and the CFB may be a cipher feedback mode (CFB). CTR, ECB, OCF and CFB may be a specific encryption method in AES128, and it may be understood that the screen-throwing device may support not only AES128, but also CTR, ECB, OCF and CFB in AES 128.

It is understood that the terminal device and the screen-throwing device may implement handshaking of the encryption capability of the communication pipe based on S405-S406. Since the terminal device can support the encryption modes of AES128 and AES256 and the projection device supports the encryption modes of CTR, ECB, OFB and CFB in AES128, the terminal device can use the encryption modes commonly supported by the two devices to perform subsequent encryption control.

In a possible implementation manner, the handshake with encryption capability may not be implemented through the steps shown in S405 to S406 between the terminal device and the screen-throwing device, which is not specifically limited in the embodiment of the present application.

S407, the terminal equipment sends an equipment management instruction to the screen throwing equipment.

The device_management instruction may include: the device_management field, the device_management instruction is used to obtain a DEVICE MANAGEMENT sub-capability set supported by the DEVICE, or may be understood as a sub-field used to obtain DEVICE support in the device_management field.

The device management sub-capability may be a field for supporting a device to implement certain device information transfer.

An example is given of a message for DEVICE sub-capability handshake based on the device_management instruction:

Request：DEVICE_MANAGEMENT rtsp://localhost/wfd1.0 RTSP/1.0\r\n

Date：Wed,11Dec 2019 08:31:54+0000\r\n

Server：stagefright/1.2(Linux；Andriod 8.1.0)\r\n

CSeq：3\r\n

Content-type：text/parameters

Content-length：83

\r\n

Line-based text data：text/parameters(5lines)

get_version\r\n

get_ota_status\r\n

set_ota_status\r\n

get_sensor\r\n

the get_version may be used to obtain version information of the device, get_ OTA _status is used to obtain an OTA upgrade state of the device, set_ OTA _status is used to set the OTA upgrade state of the device, and get_sensor is used to obtain sensor data of the device. The OTA upgrade state may include one or more of the following: version detection, upgrade file transmission, or self-starting, etc.; the sensor data of the device may include one or more of the following: acceleration data, angular velocity data (or angular acceleration data), magnetometer data, or the like.

It will be appreciated that get_version, get_ ota _status, set_ ota _status, and get_sensor may be device management sub-capabilities supported by the terminal device, and the terminal device may determine whether the device to be screened also supports the device management sub-capabilities by sending the device management sub-capabilities.

In a possible implementation manner, the device_management instruction may also include a field for indicating screen brightness information of the screen-throwing DEVICE, or a field for performing volume control, etc. other fields for acquiring DEVICE information, which is not limited in the embodiment of the present application.

S408, the screen throwing device returns a device management sub-capability set supported by the screen throwing device and field names supported by the sub-capabilities to the terminal device.

The response message returned based on the device_management instruction in S407 may be adapted as follows:

Response：RTST/1.0 200OK\r\n

CSeq：3\r\n

Content-type：text/parameters

Content-length：848

\r\n

Line-based text data：text/parameters(5lines)

get_version：software_version；hardware_version\r\n

get_ota_status：status.timeout\r\n

set_ota_status：status.timeout\r\n

get_sensor：accelerated_speed；palstance；magnetometer_data\r\n

the software_version may be software version information, the hardware_version may be hardware version information, the status. Timeout may be an OTA upgrade state and a timeout time corresponding to the OTA upgrade state, the accelerated_speed may be acceleration data, the parameter may be angular velocity data, and the magnetmeter_data may be magnetometer data.

It can be understood that the terminal device can acquire different types of device information from the screen-throwing device based on the field returned by the screen-throwing device, so as to realize the management of the device information. For example, the terminal device may acquire sensor data from the screen-casting device based on the get_sensor field to implement adjustment for the display screen; different types of device information are acquired from the screen throwing device based on the get_version field, the set_ ota _status field and the like, so that device management and the like for the screen throwing device are realized.

When the device management sub-capability returned from the screen-casting device to the terminal device does not contain the get_version, it may be determined that the screen-casting device does not support the get_version.

S409, the terminal equipment sends an instruction for acquiring the sensor parameter to the screen throwing equipment.

The instructions for acquiring the sensor parameter may include: the device_management field and the get_sensor field.

S410, the screen throwing device returns a sensor parameter to the terminal device.

The sensor parameters may include one or more of the following: acceleration data, angular velocity data, and magnetometer data. In a possible implementation manner, the sensor parameter may further include: the rotation angle of the device, the displacement of the device and other fields can be calculated by the screen projection device based on data such as acceleration data, angular velocity data, magnetometer data and the like.

S411, the terminal equipment adjusts media resources by using the sensor data.

For example, when the terminal device determines that the head of the user turns left based on the acceleration data and the angular velocity data, the terminal device may send the content, which is biased to the left of the screen, in the media resource to the screen projecting device, so as to implement adjustment of the media resource according to the sensor data, so that the user may view a more stereoscopic and real screen.

And S412, the terminal equipment sends the adjusted media resources to the screen throwing equipment.

For example, the procedure of the media resource after the terminal device screen adjustment in the step shown in S412 may refer to the corresponding embodiment of fig. 5. In the corresponding embodiment of fig. 5, the method field supported by the terminal device in the step shown in S404 for returning the terminal device to the screen-throwing device may include the method fields referred to in table 1: DESCRIBE, SETUP, PLAY and TEARDOWN.

Fig. 5 is a schematic flow chart of a screen media resource according to an embodiment of the present application.

As shown in fig. 5, the process of dropping a media asset may include the steps of:

s501, the screen throwing device sends a DESCRIBE instruction to the terminal device.

S502, the terminal equipment returns description information to the screen throwing equipment.

The description information (or referred to as session description information) may include: the information supported in the session description protocol (session description protocol, SDP), for example, may include one or more of the following, for example: session version information, session name, session time, and other supplemental information, etc.

S503, the screen throwing device sends a SETUP instruction to the terminal device.

The SETUP instruction is used for setting up session attributes and transmission modes, and prompting the terminal equipment to establish a session.

S504, the terminal equipment returns a session identifier to the screen throwing equipment.

S505, the screen throwing device sends a PLAY instruction to the terminal device.

S506, the terminal equipment returns a response message corresponding to the PLAY instruction to the screen throwing equipment.

The response message corresponding to the PLAY instruction is used for indicating the screen throwing device to receive the PLAY instruction.

S507, the terminal equipment sends the adjusted media data to the screen throwing equipment.

For example, the screen projection device may sequentially send a data packet containing the adjusted media data to the terminal device in the form of a data packet until the adjusted media data is sent.

In a possible implementation manner, the screen throwing device sends a TEARDOWN instruction to the terminal device at S508.

S509, the terminal equipment returns a response message corresponding to the TEARDOWN instruction to the screen throwing equipment.

The response message corresponding to the TEARDOWN instruction is used to indicate that the terminal device receives the TEARDOWN instruction.

The terminal device can send the adjusted media data based on the embodiment corresponding to fig. 5, so that the user can view the processed picture based on the rotation of the user in the screen throwing device.

It can be understood that, based on the embodiments corresponding to fig. 4 and fig. 5, the device_management may implement obtaining information about each type of DEVICE in the screen-throwing DEVICE, so that the terminal DEVICE may implement DEVICE MANAGEMENT for the screen-throwing DEVICE and control for a picture in the screen-throwing DEVICE. For example, the screen casting device may adjust the media assets of the screen casting device based on the sensor data of the screen casting device, and so on.

In a possible implementation manner, after the terminal DEVICE and the screen throwing DEVICE complete handshaking of sub-capabilities supported by the DEVICE based on the device_management instruction, the screen throwing DEVICE can periodically return sensor data to the terminal DEVICE through a step shown in S410, so that the terminal DEVICE can determine a rotation angle and a displacement of the screen throwing DEVICE by using the sensor data, and the rotation angle and the displacement of the screen throwing DEVICE periodically adjust a screen throwing picture, so that experience of a user for viewing the screen throwing picture by using the screen throwing DEVICE is improved.

Based on the information, the terminal equipment can acquire various types of equipment information of the screen throwing equipment through the device_MANAGEMENT instruction, and can realize equipment MANAGEMENT and maintenance of the screen throwing equipment through the equipment information of the screen throwing equipment, or can adjust the media resources of the screen throwing according to the equipment information of the screen throwing equipment.

Aiming at the second problem: and the terminal equipment cannot carry out OTA upgrading on the screen throwing equipment.

In the embodiment of the application, the terminal equipment can realize the OTA upgrading of the screen throwing equipment based on the FILE_TRANPORT field extended in the RTSP.

Fig. 6 is a schematic flow chart for implementing OTA upgrading according to an embodiment of the present application.

As shown in fig. 6, the procedure for implementing OTA upgrade is as follows:

s601, the terminal equipment sends an OPTIONS instruction to the screen throwing equipment.

S602, the screen throwing device returns a method field supported by the screen throwing device to the terminal device.

S603, the screen throwing device sends an ENCRYPTION_CONTROL instruction to the terminal device.

S604, the screen throwing device returns the pipeline encryption capability supported by the screen throwing device to the terminal device.

The steps shown in S601-S602 may be described in S401-S402, and the steps shown in S603-S604 may be described in S405-S406, which are not described herein.

S605, the terminal equipment sends a file transmission instruction to the screen throwing equipment.

The file_tranport instruction may include: the file_tranport field, the file_tranport instruction is used to obtain a FILE transfer sub-capability set supported by the device, or may be understood as a sub-field used to obtain the device support in the file_tranport field. The file transfer sub-capability set may include a field for supporting the device to implement a file transfer function, among other things.

Examples are messages of handshaking of a FILE transfer sub-capability set supported by device management based on FILE _ TRANPORT instruction:

Request：FILE_TRANPORT rtsp://localhost/wfd1.0 RTSP/1.0\r\n

Date：Wed,11Dec 2019 08:31:54+0000\r\n

Server：stagefright/1.2(Linux；Andriod 8.1.0)\r\n

CSeq：4\r\n

Content-type：text/parameters

Content-length：83

\r\n

Line-based text data：text/parameters(4lines)

get_transport_para\r\n

set_transport_para\r\n

set_file_info\r\n

set_file_data\r\n

the get_transport_para, set_transport_para, set_file_info, and set_file_data may be understood as a file transfer sub-capability set, where get_transport_para is used to obtain file transfer parameters, set_transport_para is used to set file transfer parameters, set_file_info is used to set file information, and set_file_data is used to transfer file data.

It is understood that the terminal device may perform the steps shown in S605 when receiving an instruction for OTA file upgrade. For example, the terminal device may automatically perform the step shown in S605 based on the version situation of the screen-casting device; or the terminal device may perform the steps shown in S605 based on the user operation.

In one implementation, the terminal device may perform the step shown in S605 based on the version of the projection device. Specifically, the terminal device may acquire the version information of the screen capturing device based on get_version in S408, and when determining that the version level indicated in the version information is lower than the preset version level, the RTSP middleware in the terminal device may automatically initiate an instruction for upgrading the OTA file, so that the viewing application executes the step shown in S605. The terminal device may preset a file (e.g., an OTA file) for upgrading a version of the screen device, and the preset version level may be a version level indicated by the preset file in the terminal device.

In another implementation, the terminal device may perform the steps shown in S605 based on the user operation indicated in fig. 7. Fig. 7 is an interface schematic diagram of a version upgrade according to an embodiment of the present application. In the embodiment corresponding to fig. 7, a terminal device is taken as an example for a mobile phone to be described as an example, which does not limit the embodiment of the present application.

An application program for controlling the screen throwing device, such as a video viewing application, can be preset in the terminal device, and an OTA file for upgrading the screen throwing device can be downloaded in advance in the terminal device.

In the case where the terminal device receives that the user opens the viewing application and opens the version upgrade interface, the terminal device may detect version information of the screen casting device based on get_version in S408, and display an interface as shown in a in fig. 7, which may include: the system comprises a diagram for indicating the screen throwing device and the terminal device, prompt information for indicating that version detection is being performed, and a cancel control.

Further, when the terminal device determines that the version level indicated in the version information is lower than the preset version level, the terminal device displays an interface as shown in b in fig. 7, where the interface may include one or more of the following: version upgrade control, prompt information for indicating that the version level is lower than the preset version level, etc., and the prompt information may be displayed as "the version of the screen-throwing device for which connection has been detected is lower".

As shown in b of fig. 7, when the terminal device receives the operation of the user-triggered version upgrade control, the terminal device may initiate an instruction for OTA file upgrade and perform the steps shown in S605.

In a possible implementation manner, the terminal device may also initiate a silence notification at a notification center of the terminal device when detecting that the version level of the screen-throwing device is lower than the preset version level, so as to prompt the user to perform version upgrading.

Fig. 8 is a schematic diagram of an interface for another version upgrade according to an embodiment of the present application. When the terminal device detects that the version level of the screen-throwing device is lower than the preset version level, the terminal device may display an interface as shown in fig. 8, where the interface may be a notification center of the terminal device, and the notification center may include: and the prompt message 801 is used for prompting that the version of the screen projection device is lower. Further, the terminal device may enter an interface shown in b in fig. 7 based on the triggering operation of the user on the prompt message 801, and implement version upgrade of the screen throwing device, for example, execute steps shown in S605-S614.

It can be understood that in the embodiment corresponding to fig. 8, in the case of implementing the method field interaction between the terminal device and the screen device, the terminal device may periodically acquire the version information from the screen device based on the get_version field acquired in S408, so as to implement periodic version detection. In the embodiment of the application, the specific mode of upgrading the version of the screen throwing equipment by using the terminal equipment by the user is not limited.

S606, the screen throwing device returns a file transmission sub-capability set supported by the screen throwing device and field names supported by the sub-capabilities to the terminal device.

The response message returned based on the file_tranport instruction in S601 may be adapted as follows:

Response：RTST/1.0 200OK\r\n

CSeq：4\r\n

Content-type：text/parameters

Content-length：848

\r\n

Line-based text data：text/parameters(4lines)

get_transport_para：file_trans_version；retrans_enable；max_trans_unit_size；timeout；encryption_enable\r\n

set_transport_para：file_trans_version；retrans_enable；max_trans_unit_size；timeout；encryption_enable\r\n

set_file_info：file_name；file_length\r\n

set_file_data：file_data；file_offset\r\n

the file_trans_version may be version information of a transmission protocol, the retransmission_enable may be whether retransmission is supported, the max_trans_unit_size may be a maximum length of a single packet, the timeout may be a timeout time, the encryption_enable may be whether encryption is supported, the file_name may be a file name, the file_length may be a file size, the file_data may be file data, and the file_offset may be an offset (or a file sequence number to be understood as a retransmission) corresponding to the file retransmission. The file data may be referred to as an OTA file.

In a possible implementation manner, the encryption_enable may also be used to determine an encryption type, for example, when the encryption_enable is 1, the device is instructed to support the encryption mode of AES128, when the encryption_enable is 2, the device is instructed to support the encryption mode of AES256, and so on, so that both sides of the device clearly determine the encryption mode again.

S607, the terminal equipment sends an instruction for setting file information to the screen throwing equipment.

The instructions for setting file information may include: FILE _ TRANPORT: the set_file_info may include: file_name; file_length. The file_name enables the screen throwing device to define the file name, and the file_length enables the screen throwing device to determine whether the file with the size of the file_length can be stored according to the current memory space. The file information may be information of an OTA file.

S608, setting file information by the screen throwing device.

S609, the screen throwing device returns a message for indicating whether the file information is successfully set to the terminal device.

For example, in the process of setting file information by the terminal device, when the screen throwing device determines that there is a space to store the OTA file based on the file_length in the set_file_info, the screen throwing device may return a message for indicating that the file information is set successfully to the terminal device.

Or, in the process of setting the file information, when the screen projection device determines that there is not enough space to store the OTA file, that is, the file size indicated in the file_length is greater than (or greater than or equal to) the memory space preset by the screen projection device, the screen projection device may return a message for indicating that the file information is set to fail to the terminal device. At this time, the terminal device can determine that the subsequent transmission of the OTA file cannot be performed, and the file transmission flow is interrupted; or, the terminal device may also continue to execute the transmission process of the OTA file, but the file transmission process may be interrupted due to insufficient memory detected by the terminal device.

In a possible implementation manner, the steps shown in S607-S609 may also be performed after the steps shown in S610-S614, and the specific order of performing S607-S609 is not specifically limited in the embodiment of the present application.

S610, the terminal equipment sends an instruction for acquiring file transmission parameters to the screen throwing equipment.

The instructions for obtaining the file transfer parameters may include: a file_transport_para field, and a get_transport_para field.

S611, the screen throwing device returns file transmission parameters to the terminal device.

The file transfer parameters described in S611 may be referred to as second parameters, or parameters required by the screen-casting device when performing file transfer, and the file transfer parameters may include one or more of the following: the meaning of any one of the parameters file_trans_version, retrans_enable, max_trans_unit_size, timeout, or encryption_enable may be described in the step shown in S602.

In a possible implementation, the on-screen device may encrypt the file transfer parameters, for example, the on-screen device may determine to encrypt the content transferred in the process of S607-S614 by AES128 based on the handshake of the pipe encryption capabilities in S603-S604 in the corresponding embodiment of fig. 6. Adaptively, decryption may also be based on AES128 when the terminal device receives the encrypted transmission.

S612, the terminal equipment sends an instruction for setting file transmission parameters to the screen throwing equipment.

The instructions for setting file transfer parameters may include: a file_transport_para field, and a set_transport_para field.

For example, the terminal device may determine the first parameter from the second parameter and the third parameter in S611, and return the first parameter to the screen-throwing device through the instruction in S612. The third parameter is a parameter required by the terminal equipment when the file is transmitted, the second parameter is a parameter required by the screen throwing equipment when the file is transmitted, and the first parameter is the same parameter in the second parameter and the third parameter; the content included in the first parameter or the second parameter may be similar to the content included in the first parameter described in S611, and will not be described here.

S613, the screen throwing device sets file transmission parameters.

S614, the screen throwing device returns a message for indicating whether the file transmission parameter is set successfully or not to the terminal device.

For example, when the terminal device receives a message returned by the screen projection device and indicating that the file transfer parameter setting is successful, the terminal device may perform the step shown in S615.

Or when the terminal device receives the message returned by the screen throwing device and used for indicating that the file transmission parameter setting fails, the terminal device can execute the steps shown in S612-S614 to re-set the file transmission parameter.

S615, the terminal equipment sends an instruction for setting the OTA file to the screen throwing equipment.

The instructions for setting the OTA file may include: the file_track field and the set_file_data field may also be used to set an offset corresponding to the OTA FILE by the OTA FILE, for example, the offset may indicate the first packet in the OTA FILE.

In a possible implementation manner, the screen throwing device may encrypt the OTA file based on AES128 supported by both the terminal device and the screen throwing device, so as to ensure security of data transmission.

S616, the screen throwing device returns a message for indicating whether the OTA file is successfully set to the terminal device.

When the screen throwing device returns a message indicating that the OTA file setting fails to the terminal device, the terminal device may perform the step shown in S613 to implement retransmission of the OTA file. The message for indicating the OTA file setting failure may further include: and the offset corresponding to the file with failed transmission enables the terminal equipment to realize breakpoint continuous transmission.

It can be understood that the steps shown in S615-S616 may be performed multiple times between the terminal device and the screen device, so that the OTA file may be sequentially returned from the terminal device to the screen device in the form of a data packet until the OTA file is transmitted.

Based on the above, in the case that the OTA FILE is OTA FILE data, the terminal device can realize OTA upgrading of the screen throwing device through the file_TRANPORT method field expanded in the RTSP, and the flexibility of managing the screen throwing device is improved.

Based on the corresponding embodiment of fig. 6, in a possible implementation manner, the terminal device may implement query and management of log FILEs in the screen-throwing device, and the process of FILE transmission may be implemented based on file_tranport instruction extended in RTSP. Wherein, the log file may include: RTSP messages.

Fig. 9 is a schematic flow chart for implementing log file transmission according to an embodiment of the present application.

As shown in fig. 9, the procedure for realizing log file transmission is as follows:

before log file transmission is achieved, the terminal device and the screen throwing device can handshake method fields and pipeline encryption capability in RTSP based on the steps shown in S601-S604. And upon receiving the instruction for acquiring the log file, the step shown in S901 is performed. The instruction for obtaining the log file may be an instruction that is periodically and automatically initiated by the terminal device, or may be an instruction that is initiated by the terminal device based on an operation of a user in a movie viewing application, which is not specifically limited in the embodiment of the present application.

And S901, the terminal equipment sends a FILE_TRANPORT instruction to the screen throwing equipment.

The file transfer instruction is used for acquiring a file transfer sub-capability set supported by the device.

For example, the terminal device may perform the steps shown in S901 upon receiving an instruction for acquiring the log file.

Request：FILE_TRANPORT rtsp://localhost/wfd1.0 RTSP/1.0\r\n

Date：Wed,11Dec 2019 08:31:54+0000\r\n

Server：stagefright/1.2(Linux；Andriod 8.1.0)\r\n

CSeq：4\r\n

Content-type：text/parameters

Content-length：83

\r\n

Line-based text data：text/parameters(4lines)

get_filelist_info\r\n

get_transport_para\r\n

get_file_data\r\n

the get_file_info, get_transport_para, and get_file_data may be understood as a file transfer sub-capability set, where get_file_info is used to obtain a file list, and get_file_data is used to obtain file data.

In a possible implementation manner, see description in the steps shown in S605 and S606, in order to enable the terminal device to not only send the OTA FILE to the screen-throwing device, but also collect the log FILE from the screen-throwing device, the FILE transmission sub-capability set in the file_tranport instruction may also include all the fields indicated in S401 and S402. At this time, the file transfer sub-capability set may include: get_transport_para, set_transport_para, set_file_info, set_file_data, get_file_info, and get_file_data.

S902, the screen throwing device returns a file transmission sub-capability set supported by the screen throwing device and field names supported by the sub-capabilities to the terminal device.

The response message returned based on the file_tranport instruction in S605 may be adapted as follows:

Response：RTST/1.0 200OK\r\n

CSeq：4\r\n

Content-type：text/parameters

Content-length：848

\r\n

Line-based text data：text/parameters(4lines)

get_filelist_info：filelist；filestruct；file_name；file_type；file_crc\r\n

get_file_data：file_name；file_offset；file_length；retrans_bitmap\r\n

the file list may be a file list (or an array), the file structure may be a structure body where the array is located, the file_name may be a file name or a structure body name, the file_type may be a file type, the file_crc may be a check value, the file_offset may be an offset point corresponding to the file retransmission, and the retransmission_bitmap may be the number of data packets transmitted in one transmission. For example, the retrans_bitmap may be set to a value of 16, 32, 128, or the like, and the file_offset may be set to 0 at the time of the first transmission.

S903, the terminal equipment sends an instruction for acquiring a file list to the screen throwing equipment.

The instructions for retrieving the file list may include: a file_tranport field, and a get_filelist_info field.

S904, the screen throwing device returns a file list supported by the screen throwing device to the terminal device.

S905, the terminal equipment sends an instruction for acquiring file transmission parameters to the screen throwing equipment.

S906, the screen throwing device returns file transmission parameters to the terminal device.

S907, the terminal equipment sends an instruction for setting file transmission parameters to the screen throwing equipment.

S908, the screen throwing device returns a message for indicating whether the file transmission parameter is set successfully or not to the terminal device.

The steps shown in S905 to S908 may be described in steps shown in S610 to S614, and will not be described herein.

S909, the terminal equipment sends an instruction for acquiring the log file to the screen throwing equipment.

The instructions for obtaining the log file may include: the FILE_TRANPORT field and the get_file_data field.

S910, the screen throwing device returns log files to the terminal device.

It can be understood that the steps shown in S909-S910 may be performed multiple times between the terminal device and the screen-throwing device, so that the log file may be sequentially sent from the screen-throwing device to the terminal device in the form of a data packet until the log file is transmitted.

In a possible implementation manner, after the terminal device obtains the log file in the screen-throwing device based on the steps shown in S901-S910, the RTSP message may be filtered from the log file by using a tcpdump tool, and the method field adopted in the RTSP message may be determined by analyzing the imported RTSP message using a wireshark tool. The tcpdump may be a tool for collecting and analyzing network data, and the wireshark may be a tool for analyzing network packets.

Based on the above, under the condition that the log FILE comprises the RTSP message, the terminal equipment can process and analyze the FILE of the terminal equipment through the file_TRANPORT method field expanded in the RTSP, so that various states of communication based on the RTSP can be known in time.

Based on the descriptions of the embodiments corresponding to fig. 4 to fig. 9, the terminal device described in the embodiment of the present application may be a terminal device that does not support the DP protocol. Exemplary, the terminal device sends an OPTIONS instruction to the screen throwing device, including: and under the condition that the terminal equipment does not support the DP protocol, the terminal equipment sends an OPTIONS instruction to the screen throwing equipment.

In a possible implementation manner, on the basis of the embodiment corresponding to fig. 6, the terminal device may also be a terminal device supporting a DP protocol, in this scenario, the terminal device may establish a screen-throwing connection with the screen-throwing device through a USB Type-C interface, and send an OTA file to the screen-throwing device by using the DP protocol.

Fig. 10 is a schematic flow chart of determining whether to use a DP protocol according to an embodiment of the present application.

As shown in fig. 10, in case the terminal device determines that the DP protocol is supported, the terminal device may transmit the OTA file through the DP protocol. For example, the terminal device has an USB Type-C interface, when the terminal device receives an instruction for upgrading an OTA file and determines that the terminal device supports a DP protocol under the condition that the terminal device establishes a screen connection with the screen device based on the USB Type-C interface, the terminal device sends the OTA file to the screen device based on the DP protocol.

In case the terminal device determines that the DP protocol is not supported, the terminal device may transmit the OTA file through RTSP. For example, in the case where the terminal device establishes a WIFI screen connection with the screen device, when the terminal device receives an instruction for OTA file upgrade and determines that the terminal device does not support the DP protocol, the terminal device may perform the steps shown in S601-S614 or perform the steps shown in S605-S614.

In a possible implementation manner, the terminal device may also send the OTA file through RTSP if it is determined that the DP protocol is not supported and RTSP is supported.

The driver in the kernel layer of the terminal equipment can determine whether the DP protocol is supported or not according to the line condition in the USB Type-C interface when the terminal equipment is connected with the screen throwing equipment, and send an instruction supporting the DP protocol to the framework layer and the application layer when the DP protocol is supported, so that the application layer of the terminal equipment can utilize the DP protocol in the framework layer to realize data transmission with the screen throwing equipment. Or, when the viewing application of the terminal device confirms that the instruction for supporting the DP protocol is not received, it may be determined that the device does not support the DP protocol. It is understood that different protocols may correspond to lines in different USB Type-C interfaces.

In a possible implementation manner, in the case that the terminal device may establish a screen-throwing connection with the screen-throwing device through the USB Type-C interface, the terminal device may also implement sending the OTA file to the screen-throwing device based on a USB2.0 (or USB 3.0) protocol.

In another implementation, in the case that the terminal device determines that the USB2.0 protocol is supported and determines that a connection is established with the screen throwing device through the USB Type-C interface, the terminal device may send the OTA file through the USB2.0 protocol. For example, the terminal device may interact version information with the screen-throwing device through the USB2.0 protocol, and in the case that it is determined that the version information in the screen-throwing device is lower than the preset version information, the terminal device may send the OTA file to the screen-throwing device based on an instruction for upgrading the OTA file.

In the case that the terminal device determines that the USB2.0 protocol is not supported, the terminal device may send the OTA file through RTSP, and the process of sending the OTA file by using RTSP may refer to the corresponding embodiment of fig. 6, which is not described herein.

It can be understood that the terminal device described in the embodiment of the present application may also support the DP protocol, the USB2.0 protocol, and RTSP simultaneously, so that the terminal device may determine what protocol is used to perform OTA upgrade on the screen device according to the connection situation between the terminal device and the screen device. For example, in the case that it is determined that the DP protocol and the USB2.0 protocol are simultaneously supported, and connection is established between the terminal device and the screen throwing device through the USB Type-C interface, the terminal device may send the OTA file to the screen throwing device by preferentially adopting the USB2.0 protocol. And under the condition that the DP protocol and the USB2.0 protocol are not supported or the fact that connection between the terminal equipment and the screen throwing equipment is not established through the USB Type-C interface is detected, the terminal equipment adopts RTSP to carry out OTA upgrading.

Based on this, the terminal device described in the embodiment of the present application not only can support file upgrade under DP protocol (or USB2.0 protocol), but also can upgrade file under RTSP, and improves the use experience of the user in using the screen-throwing function while expanding the use scenario of file upgrade.

Based on the descriptions in the embodiments corresponding to fig. 4 to fig. 10, the screen projection management method provided by the embodiment of the application is not limited to the scenes such as sensor data transmission, OTA upgrading, log file transmission and the like. For example, the sub-fields supported in the device_management can be extended between the terminal DEVICE and the screen-throwing DEVICE, so that other data can be transmitted, for example, in a DEVICE maintenance scene, the screen-throwing DEVICE can send the abnormal condition of the DEVICE to the terminal DEVICE through the sub-fields supported in the device_management, and the specific application scene is not limited in the embodiment of the application.

It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are information and data authorized by the user or fully authorized by each party, and the collection, use and processing of the related data need to comply with related laws and regulations and standards, and provide corresponding operation entries for the user to select authorization or rejection.

The method provided by the embodiment of the present application is described above with reference to fig. 4 to 10, and the device for performing the method provided by the embodiment of the present application is described below. As shown in fig. 11, fig. 11 is a schematic structural diagram of a screen projection management device according to an embodiment of the present application, where the screen projection management device may be a terminal device or a screen projection device in the embodiment of the present application, or may be a chip or a chip system in the terminal device, or a chip system in the screen projection device.

As shown in fig. 11, the screen-shot management apparatus 1100 may be used in a communication device, a circuit, a hardware component, or a chip, and includes: a display unit 1101, and a processing unit 1102, and a communication unit 1103. Wherein the display unit 1101 is used for supporting the step of displaying executed by the screen-throwing management apparatus 1100; the processing unit 1102 is used for supporting the screen-throwing management device 1100 to execute the steps of information processing; the communication unit is used to support the screen-drop management apparatus 1100 to perform the step of transmitting data and receiving data. The communication unit 1103 may be an input or output interface, a pin or a circuit, etc.

In a possible embodiment, the screen projection management device may further include: a storage unit 1104. The processing unit 1102 and the storage unit 1104 are connected by a line. The memory unit 1104 may include one or more memories, which may be one or more devices, circuits, or means for storing programs or data. The storage unit 1104 may exist independently and be connected to the processing unit 1102 provided in the screen management apparatus through a communication line. The memory unit 1104 may also be integrated with the processing unit 1102.

The storage unit 1104 may store computer-executable instructions of the method in the terminal device to cause the processing unit 1102 to perform the method in the above-described embodiment. The storage unit 1104 may be a register, a cache, a RAM, or the like, and the storage unit 1104 may be integrated with the processing unit 1102. The memory unit 1104 may be a read-only memory (ROM) or other type of static storage device that may store static information and instructions, and the memory unit 1104 may be independent of the processing unit 1102.

Fig. 12 is a schematic hardware structure of another terminal device according to an embodiment of the present application, as shown in fig. 12, where the terminal device includes a processor 1201, a communication line 1204 and at least one communication interface (the communication interface 1203 is exemplified in fig. 12).

The processor 1201 may be a general purpose central processing unit (central processing unit, CPU), microprocessor, application Specific Integrated Circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application.

Communication line 1204 may include circuitry to transfer information between the above-described components.

The communication interface 1203 uses any transceiver-like device for communicating with other devices or communication networks, such as ethernet, wireless local area network (wireless local area networks, WLAN), etc.

Possibly, the terminal device may also comprise a memory 1202.

The memory 1202 may be, but is not limited to, read-only memory (ROM) or other type of static storage device that can store static information and instructions, random access memory (random access memory, RAM) or other type of dynamic storage device that can store information and instructions, but may also be electrically erasable programmable read-only memory (EEPROM), compact disc-read only memory (compact disc read-only memory) or other optical disk storage, optical disk storage (including compact disc, laser disc, optical disc, digital versatile disc, blu-ray disc, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. The memory may be implemented separately and coupled to the processor via communication line 1204. The memory may also be integrated with the processor.

The memory 1202 is used for storing computer-executable instructions for performing aspects of the present application, and is controlled by the processor 1201 for execution. The processor 1201 is configured to execute computer-executable instructions stored in the memory 1202 to implement the methods provided by embodiments of the present application.

Possibly, the computer-executable instructions in the embodiments of the present application may also be referred to as application program codes, which are not limited in particular.

In a particular implementation, the processor 1201 may include one or more CPUs, such as CPU0 and CPU1 in fig. 12, as one embodiment.

In a specific implementation, as an embodiment, the terminal device may include a plurality of processors, such as processor 1201 and processor 1205 in fig. 12. Each of these processors may be a single-core (single-CPU) processor or may be a multi-core (multi-CPU) processor. A processor herein may refer to one or more devices, circuits, and/or processing cores for processing data (e.g., computer program instructions).

The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions in accordance with embodiments of the present application are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wired (e.g., coaxial cable, fiber optic, digital subscriber line (digital subscriber line, DSL), or wireless (e.g., infrared, wireless, microwave, etc.), or semiconductor medium (e.g., solid state disk, SSD)) or the like.

The embodiment of the application also provides a computer readable storage medium. The methods described in the above embodiments may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. Computer readable media can include computer storage media and communication media and can include any medium that can transfer a computer program from one place to another. The storage media may be any target media that is accessible by a computer.

As one possible design, the computer-readable medium may include compact disk read-only memory (CD-ROM), RAM, ROM, EEPROM, or other optical disk memory; the computer readable medium may include disk storage or other disk storage devices. Moreover, any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. Disk and disc, as used herein, includes Compact Disc (CD), laser disc, optical disc, digital versatile disc (digital versatile disc, DVD), floppy disk and blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers.

Combinations of the above should also be included within the scope of computer-readable media. The foregoing is merely illustrative embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily think about variations or substitutions within the technical scope of the present invention, and the invention should be covered. Therefore, the protection scope of the invention is subject to the protection scope of the claims.

Claims

1. The screen projection management method is characterized by being applied to terminal equipment, wherein the terminal equipment and the screen projection equipment are in a screen projection connection state, and the method comprises the following steps:

the terminal equipment sends a first instruction to the screen throwing equipment;

the terminal equipment receives a first message from the screen throwing equipment, wherein the first message comprises a first method field, the first method field is used for supporting file transmission between equipment, and the first message is generated by the screen throwing equipment in response to the first instruction;

the terminal equipment sends a second instruction to the screen throwing equipment, wherein the second instruction comprises the first method field;

the terminal equipment receives a second message from the screen throwing equipment, wherein the second message comprises the following components: a first subfield for setting file information, and a second subfield for setting a first parameter required at the time of file transmission, the second message being generated by the screen throwing device in response to the second instruction;

The terminal equipment sends a third instruction to the screen throwing equipment, wherein the third instruction comprises: file information corresponding to an OTA file of an over-the-air technology and the first subfield;

the terminal equipment receives a third message from the screen throwing equipment, wherein the third message is generated by the screen throwing equipment in response to the third instruction;

the terminal equipment sends a fourth instruction to the screen throwing equipment, wherein the fourth instruction comprises the second subfield;

the terminal equipment receives a fourth message from the screen throwing equipment, wherein the fourth message is generated by the screen throwing equipment in response to the fourth instruction;

and the terminal equipment sends the OTA file to the screen throwing equipment according to the first parameter.

2. The method of claim 1, wherein the second message further includes a third subfield for obtaining a second parameter required for file transfer, the method further comprising:

the terminal equipment sends a fifth instruction to the screen throwing equipment, wherein the fifth instruction comprises the third subfield;

the terminal equipment receives a fifth message from the screen throwing equipment, wherein the fifth message comprises the second parameter, and the fifth message is generated by the screen throwing equipment in response to the fifth instruction;

The terminal equipment sends a fourth instruction to the screen throwing equipment, which comprises the following steps: and under the condition that the first parameter is determined from the second parameter and the third parameter, the terminal equipment sends the fourth instruction to the screen throwing equipment, the third parameter is a parameter required by the terminal equipment when the file transmission is carried out, the second parameter is a parameter required by the screen throwing equipment when the file transmission is carried out, and the first parameter is the same parameter in the second parameter and the third parameter.

3. The method of claim 2, wherein the fifth message further comprises: the method comprises the steps of acquiring subfields corresponding to equipment information in the screen-throwing equipment, wherein the equipment information comprises one or more of the following steps: version information of the device, or information for indicating an OTA upgrade status.

4. A method according to any one of claims 1-3, wherein the first parameter comprises one or more of: a parameter for indicating whether retransmission is supported, a parameter for indicating a maximum length of a single packet, a parameter for indicating a timeout, a parameter for indicating whether encryption is supported, or a parameter for indicating version information.

5. The method according to any one of claims 1-4, wherein the first message further comprises: a second method field for supporting transmission of device information between devices, the method further comprising:

the terminal equipment sends a sixth instruction to the screen throwing equipment, wherein the sixth instruction comprises: the second method field;

the terminal equipment receives a sixth message from the screen throwing equipment, wherein the sixth message comprises: a third subfield for acquiring sensor data, wherein the sixth message is generated by the screen-throwing device in response to the sixth instruction;

the terminal equipment sends a seventh instruction to the screen throwing equipment, wherein the seventh instruction comprises the third subfield;

the terminal equipment receives a seventh message from the screen throwing equipment, wherein the seventh message comprises sensor data in the screen throwing equipment, and the seventh message is generated by the screen throwing equipment in response to the seventh instruction.

6. The method of claim 5, wherein the method further comprises:

the terminal equipment determines the rotation angle and the displacement of the screen throwing equipment through the sensor data, wherein the sensor data comprises one or more of the following: acceleration data of the device, angular velocity data of the device, or magnetometer data of the device;

And the terminal equipment adjusts the screen projection picture by utilizing the rotation angle and the displacement and sends the adjusted screen projection picture to the screen projection equipment.

7. The method according to any one of claims 1-6, wherein the first message further comprises: a third method field for supporting encryption capability negotiation between devices, the method further comprising:

the terminal equipment sends an eighth instruction to the screen throwing equipment, wherein the eighth instruction comprises the third method field and a first encryption method supported by the terminal equipment;

the terminal equipment receives an eighth message from the screen throwing equipment, wherein the eighth message comprises a second encryption method supported by the screen throwing equipment, and the eighth message is generated by the screen throwing equipment in response to the eighth instruction;

the terminal equipment determines the same third encryption method in the first encryption method and the second encryption method;

the terminal equipment sends a third instruction to the screen throwing equipment, and the third instruction comprises the following steps: and the terminal equipment encrypts the file information corresponding to the OTA file by using the third encryption method and sends the file information corresponding to the encrypted OTA file to the screen throwing equipment through the third instruction.

8. The method according to any one of claims 1-7, wherein the second message further comprises: a fourth subfield for retrieving a list of files, and a fifth subfield for retrieving a log file, the method further comprising:

the terminal equipment sends a ninth instruction to the screen throwing equipment, wherein the ninth instruction comprises the fourth subfield;

the terminal equipment receives a ninth message from the screen throwing equipment, wherein the ninth message comprises one or more of the following: the file list, a structure body where any file in the file list is located, a file name of any file, a file type of any file, and a check value corresponding to any file, and the ninth message is generated by the screen throwing device in response to the ninth instruction;

the terminal equipment sends a tenth instruction to the screen throwing equipment, wherein the tenth instruction comprises the fifth subfield;

and the terminal equipment receives a log file from the screen throwing equipment, wherein the log file is sent by the screen throwing equipment in response to the tenth instruction.

9. The method of any one of claims 1-8, wherein the screening device is an augmented reality XR glasses.

10. The method according to any of claims 1-9, wherein the terminal device is a device that does not support the DP protocol.

11. The method of claim 10, wherein the terminal device sending a third instruction to the screen-casting device comprises:

and under the condition that the terminal equipment does not support the DP protocol, or the terminal equipment does not support the DP protocol and supports a real-time streaming protocol RTSP, the terminal equipment sends the third instruction to the screen throwing equipment.

12. The method of claim 1, wherein the terminal device has a serial bus USB Type-C interface, the method further comprising:

the terminal equipment establishes connection with the screen throwing equipment based on the USB Type-C interface;

and under the condition that the terminal equipment supports a DP protocol, the terminal equipment sends the OTA file to the screen throwing equipment based on the DP protocol.

13. The screen projection management method is characterized by being applied to screen projection equipment, wherein the screen projection equipment and the terminal equipment are in a screen projection connection state, and the method comprises the following steps:

the screen projection device establishes screen projection connection with a terminal device, and the screen projection device and the terminal device are in the same WIFI network;

The screen throwing equipment receives a first instruction from the terminal equipment;

the screen throwing device sends a first message to the terminal device, wherein the first message comprises a first method field, and the first method field is used for supporting file transmission between devices; the first message is generated by the screen throwing equipment in response to the first instruction;

the screen throwing equipment receives a second instruction from the terminal equipment, wherein the second instruction comprises the first method field;

the screen throwing device sends a second message to the terminal device, wherein the second message comprises: a first subfield for setting file information, and a second subfield for setting a first parameter required at the time of file transfer; the second message is generated by the screen throwing equipment in response to the second instruction;

the screen projection device receives a third instruction from the terminal device, wherein the third instruction comprises: file information corresponding to an OTA file of an over-the-air technology and the first subfield;

the screen throwing device sends a third message to the terminal device; the third message is generated by the screen throwing equipment in response to the third instruction;

The screen throwing device receives a fourth instruction from the terminal device, wherein the fourth instruction comprises the second subfield;

the screen throwing device sends a fourth message to the terminal device; the fourth message is generated by the screen throwing equipment in response to the fourth instruction;

and the screen projection equipment receives the OTA file sent by the terminal equipment according to the first parameter.

14. The method of claim 13, wherein the first parameter comprises one or more of: a parameter for indicating whether retransmission is supported, a parameter for indicating a maximum length of a single packet, a parameter for indicating a timeout, a parameter for indicating whether encryption is supported, or a parameter for indicating version information.

15. The method according to claim 13 or 14, wherein the first message further comprises: a second method field for supporting transmission of device information between devices, the method further comprising:

the screen projection device receives a sixth instruction from the terminal device, wherein the sixth instruction comprises: the second method field;

the screen projection device sends a sixth message to the terminal device, wherein the sixth message comprises: a third subfield for acquiring sensor data, wherein the sixth message is generated by the screen-throwing device in response to the sixth instruction;

The screen throwing device receives a seventh instruction from the terminal device, wherein the seventh instruction comprises the third subfield;

the screen throwing device sends a seventh message to the terminal device, wherein the seventh message comprises sensor data in the screen throwing device, and the seventh message is generated by the screen throwing device in response to the seventh instruction.

16. The method of claim 15, wherein the method further comprises:

the screen projection device receives an adjusted screen projection picture from the terminal device, wherein the adjusted screen projection picture is obtained by adjusting the screen projection picture by the terminal device based on a rotation angle and a displacement, the rotation angle and the displacement are determined based on sensor data, and the sensor data comprise one or more of the following: acceleration data of the device, angular velocity data of the device, or magnetometer data of the device.

17. The method according to any one of claims 13-16, wherein the first message further comprises: a third method field for supporting encryption capability negotiation between devices, the method further comprising:

The screen throwing device receives an eighth instruction from the terminal device, wherein the eighth instruction comprises the third method field and a first encryption method supported by the terminal device;

the screen throwing device sends an eighth message to the terminal device, wherein the eighth message comprises a second encryption method supported by the screen throwing device, the third instruction comprises file information corresponding to an encrypted OTA file, the file information corresponding to the encrypted OTA file is obtained by encrypting the file information corresponding to the OTA file by using a third encryption method, the third encryption method is the same method in the first encryption method and the second encryption method, and the eighth message is generated by the screen throwing device in response to the eighth instruction.

18. The method according to any one of claims 13-17, wherein the second message further comprises: a fourth subfield for retrieving a list of files, and a fifth subfield for retrieving a log file, the method further comprising:

the screen throwing device receives a ninth instruction from the terminal device, wherein the ninth instruction comprises the fourth subfield;

The screen projection device sends a ninth message to the terminal device, wherein the ninth message comprises one or more of the following: the file list, a structure body where any file in the file list is located, a file name of any file, a file type of any file, and a check value corresponding to any file, and the ninth message is generated by the screen throwing device in response to the ninth instruction;

the screen projection device receives a tenth instruction from the terminal device, wherein the tenth instruction comprises the fifth subfield;

and the screen throwing device sends a log file to the terminal device according to the first parameter, wherein the log file is sent by the screen throwing device in response to the tenth instruction.

19. The method of claim 13, wherein the terminal device is a device that does not support a DP protocol.

20. The method of claim 13, wherein the terminal device is a DP protocol enabled device, the terminal device having a serial bus USB Type-C interface, the method further comprising:

the screen throwing device establishes screen throwing connection with the terminal device based on the USB Type-C interface,

And the screen throwing equipment receives the OTA file from the terminal equipment, wherein the OTA file is sent by the terminal equipment based on the DP protocol.

21. The method of any one of claims 13-20, wherein the screening device is an augmented reality XR glasses.

22. A terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program, causes the terminal device to perform the method according to any of claims 1 to 12.

23. A screen projection device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein execution of the computer program by the processor causes the screen projection device to perform the method of any one of claims 13 to 21.

24. A computer readable storage medium storing a computer program, which when executed by a processor causes the computer to perform the method of any one of claims 1 to 12 or to perform the method of any one of claims 13 to 21.