CN116708885A - Screen projection method, screen projection device, connection device, screen projection device and storage medium - Google Patents
Screen projection method, screen projection device, connection device, screen projection device and storage medium Download PDFInfo
- Publication number
- CN116708885A CN116708885A CN202310849368.4A CN202310849368A CN116708885A CN 116708885 A CN116708885 A CN 116708885A CN 202310849368 A CN202310849368 A CN 202310849368A CN 116708885 A CN116708885 A CN 116708885A
- Authority
- CN
- China
- Prior art keywords
- screen
- equipment
- data stream
- thrown
- devices
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 238000009877 rendering Methods 0.000 claims description 20
- 238000012216 screening Methods 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 7
- 230000005540 biological transmission Effects 0.000 description 16
- 238000010586 diagram Methods 0.000 description 12
- 230000000694 effects Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 2
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000013497 data interchange Methods 0.000 description 1
- 238000005538 encapsulation Methods 0.000 description 1
- 238000009432 framing Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/4104—Peripherals receiving signals from specially adapted client devices
- H04N21/4122—Peripherals receiving signals from specially adapted client devices additional display device, e.g. video projector
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
- H04N21/4334—Recording operations
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440218—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/482—End-user interface for program selection
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Human Computer Interaction (AREA)
- Controls And Circuits For Display Device (AREA)
Abstract
The invention provides a screen projection method, screen projection equipment, connection equipment, screen projection equipment and storage media, wherein the screen projection method comprises the following steps: acquiring information of one or more screen devices connected with a USB interface of the screen device; generating a device list according to the information of the one or more devices to be screened and displaying the device list through a user interface; acquiring one or more target screen-projected devices which are selected from the device list by a user through the user interface and are paired with the screen-projected devices; establishing a connection channel with the one or more target screened devices; performing screen recording operation to obtain a screen recording data stream; encoding the screen recording data stream to generate an encoded screen recording data stream; and transmitting the encoded screen recording data stream to one or more target screen-thrown devices through the connecting channel. In the invention, the screen throwing equipment and the screen throwing equipment can be in one-to-one, one-to-many or many-to-many relation, and the use scene of the screen throwing is expanded.
Description
Technical Field
The embodiment of the invention relates to the technical field of multimedia, in particular to a screen projection method, screen projection equipment, connection equipment, screen projection equipment and a storage medium.
Background
With the popularization of mobile phones, many users use the mobile phones to conduct office work or watch videos, however, the display screen of the mobile phones is smaller, which greatly influences the user experience. To solve this problem, the mobile phone screen may be projected to a large-sized display device.
In the existing screen-throwing mode, the screen-throwing equipment (such as a mobile phone) and the screen-throwing equipment (such as a large-size display device) are in one-to-one correspondence, so that the use scene of the screen is greatly limited.
Disclosure of Invention
The embodiment of the invention provides a screen projection method, screen projection equipment, connection equipment, screen projection equipment and a storage medium, which are used for solving the problem that the existing screen projection equipment and screen projection equipment are in one-to-one correspondence, and the use scene of the screen projection is greatly limited.
In order to solve the technical problems, the invention is realized as follows:
in a first aspect, an embodiment of the present invention provides a screen projection method, which is applied to a screen projection device, where the method includes:
acquiring information of one or more screen-thrown devices connected with a USB interface of the screen-throwing device;
generating a device list according to the information of the one or more devices to be screened and displaying the device list through a user interface;
acquiring one or more target screen-projected devices which are selected from the device list by a user through the user interface and are paired with the screen-projected devices;
establishing a connection channel with the one or more target screened devices;
performing screen recording operation to obtain a screen recording data stream;
encoding the screen recording data stream to generate an encoded screen recording data stream;
and transmitting the encoded screen recording data stream to one or more target screen-thrown devices through the connecting channel.
Optionally, obtaining information of one or more screen-thrown devices connected with the USB interface of the screen-throwing device includes:
after detecting the screen-thrown equipment connected with the USB interface of the screen-thrown equipment, generating inquiry information and displaying the inquiry information through a user interface, wherein the inquiry information is used for inquiring whether the screen-thrown equipment is connected with the screen-thrown equipment through an AOA protocol or not;
if the user selects to connect with the screen-thrown device through the AOA protocol through the user interface, detecting whether the screen-thrown device connected with the USB interface of the screen-thrown device supports a USB accessory mode or not;
and acquiring information of the screened device supporting the USB accessory mode.
Optionally, before generating the query information and displaying the query information through the user interface, the method further includes:
after detecting the screen-thrown equipment connected with the USB interface of the screen-thrown equipment, judging whether the screen-thrown equipment has USB permission or not;
if the screen-thrown equipment has USB authority, judging whether the screen-thrown equipment is matched with the screen-thrown equipment or not;
if the screen-thrown device is not matched with the screen-thrown device, requesting an AOA version number from the screen-thrown device; if the AOA version number of the screen-thrown device is greater than or equal to 1, sending information of the screen-thrown device to the screen-thrown device;
and if the screen-projected equipment is matched with the screen-projected equipment, sending information of the screen-projected equipment to the screen-projected equipment.
Optionally, the screen projection method further includes:
and sending the resolution information of the screen throwing equipment to the screen throwing equipment.
In a second aspect, an embodiment of the present invention provides a screen projection method, which is applied to a connection device, where the connection device includes:
the first USB interfaces are used for being connected with the screen throwing equipment;
the second USB interfaces are used for being connected with the equipment to be screened;
the method comprises the following steps:
receiving the coded screen recording data stream sent by the screen throwing equipment through the first USB interface;
and forwarding the encoded screen recording data stream to one or more corresponding screen devices through the second USB interface.
In a third aspect, an embodiment of the present invention provides a screen projection method, which is applied to a device to be projected, where the method includes:
acquiring information of one or more screen throwing devices connected with a USB interface of the screen throwing device;
generating a device list according to the information of the one or more screen throwing devices and displaying the device list through a user interface;
acquiring one or more target screen projection devices which are selected from the device list by a user through the user interface and are paired with the screen projection device;
establishing a connection channel with the one or more target screen projection devices;
receiving the coded screen recording data stream sent by the target screen projection equipment through the connecting channel;
decoding the encoded video data stream to obtain a decoded video data stream;
rendering the decoded screen recording data stream to obtain a rendered screen recording data stream;
and playing the rendered screen recording data stream.
Optionally, the screen projection method further includes: receiving resolution information of the screen throwing equipment, which is sent by the screen throwing equipment;
wherein said rendering the decoded video data stream comprises:
and rendering the decoded screen recording data stream according to the resolution information of the screen throwing equipment.
In a fourth aspect, an embodiment of the present invention provides a screen projection device, including:
the first acquisition module is used for acquiring information of one or more screen-thrown devices connected with the USB interface of the screen-throwing device;
the generating module is used for generating a device list according to the information of the one or more devices to be screened and displaying the device list through a user interface;
the second acquisition module is used for acquiring one or more target screened devices which are selected from the device list by a user through the user interface and are paired with the screened device;
the channel establishing module is used for establishing a connection channel with the one or more target screened devices;
the screen recording module is used for executing screen recording operation to obtain a screen recording data stream;
the coding module is used for coding the screen recording data stream and generating a coded screen recording data stream;
and the first sending module is used for sending the coded screen recording data stream to the one or more target screen-thrown devices through the connecting channel.
In a fifth aspect, an embodiment of the present invention provides a connection apparatus, including:
the first USB interfaces are used for being connected with the screen switching equipment in a wired mode;
the second USB interfaces are used for being connected with the equipment to be switched on in a wired mode;
the processing device is connected with the plurality of first USB interfaces and the plurality of second USB interfaces and is used for receiving the coded screen recording data stream sent by the screen throwing equipment through the first USB interfaces; and forwarding the encoded screen recording data stream to one or more corresponding screen devices through the second USB interface.
In a sixth aspect, an embodiment of the present invention provides a device to be screened, including:
the first acquisition module is used for acquiring information of one or more screen throwing devices connected with the USB interface of the screen throwing device;
the generating module is used for generating a device list according to the information of the one or more screen throwing devices and displaying the device list through a user interface;
the second acquisition module is used for acquiring one or more target screen projection devices which are selected from the device list by a user through the user interface and are paired with the screen projection device;
the channel establishing module is used for establishing a connection channel with the one or more target screen throwing devices;
the first receiving module is used for receiving the coded screen recording data stream sent by the target screen projection equipment through the connecting channel;
the decoding module is used for decoding the encoded video data stream to obtain a decoded video data stream;
the rendering module is used for rendering the decoded screen recording data stream to obtain a rendered screen recording data stream;
and the playing module is used for playing the rendered screen recording data stream.
In a seventh aspect, an embodiment of the present invention provides an electronic device, including: a processor, a memory and a program stored on the memory and executable on the processor, which when executed by the processor implements the steps of the method of screening as described in the first, second or third aspects above.
In an eighth aspect, embodiments of the present invention provide a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method of screening as described in the first, second or third aspects above.
In the embodiment of the invention, the one-to-one, one-to-many or many-to-many relation can be formed between the screen throwing equipment and the screen throwing equipment, so that the use scene of the screen throwing is expanded. In addition, the screen throwing equipment and the screen throwing equipment are transmitted through a USB interface, so that the rapid transmission of the screen recording data can be realized.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
fig. 1, fig. 2, fig. 3, and fig. 4 are schematic structural diagrams of a screen projection system according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of the overall structure of a projection system according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of the overall structure of a screen projection system according to another embodiment of the present invention;
FIG. 7 is a schematic flow chart of a screen projection method applied to a screen projection device according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of a Manager of a USB master/slave managing the USB master/slave according to an embodiment of the present invention;
FIG. 9 is a second flowchart of a screen projection method applied to a screen projection device according to an embodiment of the present invention;
FIG. 10 is a diagram illustrating the data structure of H264 and ACC according to an embodiment of the present invention;
FIG. 11 is a diagram illustrating a codec process according to an embodiment of the present invention;
fig. 12 is a flowchart of a screen projection method applied to a connection device according to an embodiment of the present invention;
fig. 13 is a schematic structural view of a connection device according to an embodiment of the present invention;
FIG. 14 is a flowchart of a screen projection method applied to a device to be projected according to an embodiment of the present invention;
FIG. 15 is a schematic structural diagram of a screen projection device according to an embodiment of the present invention;
FIG. 16 is a second schematic diagram of a connecting apparatus according to an embodiment of the present invention;
FIG. 17 is a schematic diagram of a device to be screened according to an embodiment of the present invention;
fig. 18 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1, fig. 2, fig. 3 and fig. 4, fig. 1, fig. 2, fig. 3 and fig. 4 are schematic structural diagrams of a screen projection system according to an embodiment of the present invention, where the screen projection system includes: the screen throwing device is connected with the screen throwing device through the connecting device. In the screen projection system shown in fig. 1, a connection device is connected with one screen projection device, and a plurality of screen projection devices are projected, and one screen projection device can project a screen to the plurality of screen projection devices. In the screen projection system shown in fig. 2, the screen projection device and the screen projected device are in one-to-one correspondence. In the screen projection system shown in fig. 3, the screen projection device and the screen projection device are in a many-to-many relationship, and one screen projection device can project a screen to one or more screen projection devices. In the screen projection system shown in fig. 4, one screen-projected device may receive the screen projected from one or more screen-projected devices. That is, the screen throwing device and the screen throwing device can be in one-to-one or one-to-many or many-to-many relation, and the use scene of the screen throwing is expanded.
Referring to fig. 5, fig. 5 is a schematic diagram of an overall structure of a screen projection system according to an embodiment of the invention, where the screen projection system includes: service layer, application layer, component layer, data transmission layer, and hardware layer.
Wherein:
service layer: also referred to as a business service layer, for providing business services, the business services comprising: the system comprises a screen throwing service, a rendering service, an audio/video coding/decoding service and a data transmission service; the audio and video coding service is executed by the screen throwing device, and the audio and video decoding service and the rendering service are executed by the screen throwing device;
application layer: also referred to as an application layer, in the embodiment of the present invention, optionally, the application layer may be encapsulated by using an official API (Application Programming Interface ) interface of an Android (Android) system.
Component layer: the system comprises an audio and video encoding and decoding component, a surface view rendering component, a screen recording component and a channel switching component. The screen recording assembly and the audio/video encoding assembly are arranged at the screen throwing equipment end, and the surface eV and the iew rendering assembly and the audio/video decoding assembly are arranged at the screen throwing equipment end. The channel switching component can be arranged at the screen throwing equipment end and/or the screen throwing equipment end.
Data transmission layer: and adopting USB AOA (Android Open Accessory Protocol ) protocol data to connect at high speed.
Hardware layer: including hardware and basic operating system (e.g., linux) environments.
Referring to fig. 6, in some embodiments, the screen projection system may be divided into the following parts: the system comprises a front end UI (User Interface), a service layer, a control layer and a data transmission layer, wherein the front end comprises a service end and a client, the service end is a screen throwing end, the client is a screen throwing end, the service end can start screen recording service after USB connection is successful, meanwhile, the control layer is also divided into the service end and the client, the service end mainly executes audio and video coding service, codes screen recording data streams into corresponding formats of images and sound, then the data are transmitted through the USB data transmission layer, the corresponding data are transmitted to the client, the client decodes the received data, and finally the screen throwing effect is realized through rendering.
Referring to fig. 7, an embodiment of the present invention provides a screen projection method, which is applied to a screen projection device, and the method includes:
step 11: acquiring information of one or more screen-thrown devices connected with a USB interface of the screen-throwing device;
step 12: generating a device list according to the information of the one or more devices to be screened and displaying the device list through a user interface; the list information comprises information of one or more screened devices.
Step 13: acquiring one or more target screen-projected devices which are selected from the device list by a user through the user interface and are paired with the screen-projected devices;
step 14: establishing a connection channel with the one or more target screened devices;
step 15: performing screen recording operation to obtain a screen recording data stream;
in the embodiment of the invention, optionally, the screen projection device can adopt a media projection to perform screen recording operation.
Step 16: encoding the screen recording data stream to generate an encoded screen recording data stream;
step 17: and transmitting the encoded screen recording data stream to one or more target screen-thrown devices through the connecting channel.
In the embodiment of the invention, the screen throwing equipment is used as a trigger end for establishing a connection channel with the screen throwing equipment, the screen throwing equipment can establish the connection channel with one or more screen throwing equipment and simultaneously throw the screen to the one or more screen throwing equipment, so that the application scene of screen throwing is expanded.
In an embodiment of the present invention, optionally, in step 11, obtaining information of one or more devices to be screened connected to a USB interface of the device to be screened includes:
step 111: after detecting the screen-thrown equipment connected with the USB interface of the screen-thrown equipment, generating inquiry information and displaying the inquiry information through a user interface, wherein the inquiry information is used for inquiring whether the screen-thrown equipment is connected with the screen-thrown equipment through an AOA protocol or not;
in the embodiment of the invention, the inquiry information can be displayed in a popup window mode.
Step 112: if the user selects to connect with the screen-thrown device through the AOA protocol through the user interface, detecting whether the screen-thrown device connected with the USB interface of the screen-thrown device supports a USB accessory mode or not;
step 113: and acquiring information of the screened device supporting the USB accessory mode.
The Accessory (Access) mode ultimately depends on the hardware device and not all devices support the Accessory mode. If the device supports the accessory mode, the uses-feature element can be used in the android management of the application to mark.
Android starts from version 3.1 and introduces support for AOA, which is a special Accessory (access) mode that allows external USB hardware to interact with Android devices. When an Android device (e.g., a screen-cast device) is in accessory mode, the connected accessory (e.g., the screen-cast device) then plays the role of a Host (Host) (responsible for powering the bus and enumerating devices), while the Android device plays the role of a USB device (device).
That is, in the embodiment of the present invention, the device to be screened may be a host, and the device to be screened may be a USB device, however, in other embodiments, the device to be screened may be a host, and the device to be screened may be a USB device.
In the embodiment of the invention, one of the screen throwing device and the screen throwing device is used as a USB host (also called a USB master end), the other one is used as a slave end, and corresponding Manager is arranged between the master end and the slave end to manage the USB master end and the slave end, so that data transmission, data byte stream transmission, file transmission and the like can be performed after the connection is successful, as shown in fig. 8.
In an embodiment of the present invention, referring to fig. 9, optionally, before generating the query information and displaying the query information through the user interface, the method further includes:
step 011: detecting a device to be screened connected with a USB interface of the device to be screened;
step 012: judging whether the screen-thrown equipment has USB permission or not; if the device to be screened has USB authority, the step 013 is entered; otherwise, go to step 017;
step 013: judging whether the screen-thrown device is matched with the screen-throwing device, optionally judging whether the screen-thrown device is matched with the screen-throwing device according to a vendor identification code (VID) and a Product identification code (PID) of the screen-thrown device; if the screen-thrown device is not matched with the screen-thrown device, step 014 is entered, otherwise step 016 is entered;
step 014: requesting an AOA version number from the screen-thrown equipment;
step 015: judging that the AOA version number of the screen-thrown equipment is larger than or equal to 1; if the AOA version number of the screen-thrown equipment is greater than or equal to 1, entering a step 016, otherwise, entering a step 17;
step 016: transmitting information of the screen throwing device to the screen throwing device;
step 017: the procedure is ended.
In an embodiment of the present invention, optionally, the screen projection method further includes: and sending the resolution information of the screen throwing equipment to the screen throwing equipment so that the screen throwing equipment renders the received screen recording data stream according to the resolution information of the screen throwing equipment.
The recording data stream comprises audio data and/or video data, and when the recording data stream is transmitted, the audio data and/or the video data need to be encoded, and the most important purpose is to compress the data, so that the cost of data transmission and storage is reduced.
In the embodiment of the invention, the encoding and decoding of video data can be performed by adopting H264, and the encoding and decoding of audio data can be performed by adopting ACC (Advanced Audio Coding ). 264 is a highly compressed digital video encoder standard. AAC is an audio coding standard based on MPEG-2 lossy digital audio compression. In the embodiment of the invention, the RTP (Real-time Transport protocol) protocol can be adopted to transmit audio and video data, and the RTP protocol can be used for directly transmitting H264 and ACC data without TS (Transport Stream) Stream encapsulation.
The data structures of H264 and ACC are briefly described below.
Referring to FIG. 10, an original H.264NALU (network abstraction layer unit ) is often composed of three parts, [ StartCode ] [ NALU Header ] [ NALU Payload ]. The Start Code is used to indicate that this is the Start of a NALU, must be "00 00 0001" or "00 0001", and when encapsulated as an RTP frame, it can be discarded because it is a fixed field, and NALU type will be converted into the type field of RTP payload, and the other fields will also be converted into the corresponding fields of RTP, and decide whether to frame or not depending on whether it is larger than the MTU (Maximum Transmission Unit ) size.
The audio coding data formats of AAC include ADIF (Audio Data Interchange Format, audio data exchange format) and ADTS (Audio Data Transport Stream ), the former is suitable for file storage, the latter is suitable for transport stream, and for ADTS frames, since it is generally smaller than the MTU size definition, it can be directly encapsulated into RTP frames for transmission without framing processing, only with audio identification being filled in at the payload type.
In the embodiment of the invention, the screen projection equipment can adopt a coding and decoding tool MediaCodec to realize coding and decoding of the screen recording data stream, and can realize coding and decoding operation on the data by creating a corresponding coder and decoder according to the coding and decoding type. The call of MediaCodec is divided into synchronous mode and asynchronous mode, as shown in fig. 11, the state machine flow of the two modes is that the obtained Surface data is delivered to the MediaCodec for encoding, so that the image flow data for transmission can be obtained, and similarly, the image flow data is delivered to the MediaCodec for decoding by the receiver, and then is transmitted in the form of TCP (Transmission Control Protocol ) or UDP (user datagram protocol, user Datagram Protocol) through the channel after successful connection.
Referring to fig. 12, an embodiment of the present invention further provides a screen projection method applied to a connection device, referring to fig. 13, where the connection device includes:
the first USB interfaces are used for being connected with the screen throwing equipment;
the second USB interfaces are used for being connected with the equipment to be screened;
the method comprises the following steps:
step 21: receiving the coded screen recording data stream sent by the screen throwing equipment through the first USB interface;
step 22: and forwarding the encoded screen recording data stream to one or more corresponding screen devices through the second USB interface.
In the embodiment of the invention, the one-to-one, one-to-many or many-to-many relation between the screen throwing equipment and the screen throwing equipment can be realized through the connecting equipment, and the use scene of the screen throwing is expanded.
Referring to fig. 14, an embodiment of the present invention further provides a screen projection method applied to a device to be projected, where the method includes:
step 31: acquiring information of one or more screen throwing devices connected with a USB interface of the screen throwing device;
step 32: generating a device list according to the information of the one or more screen throwing devices and displaying the device list through a user interface;
step 33: acquiring one or more target screen projection devices which are selected from the device list by a user through the user interface and are paired with the screen projection device;
step 34: establishing a connection channel with the one or more target screen projection devices;
step 35: receiving the coded screen recording data stream sent by the target screen projection equipment through the connecting channel;
step 36: decoding the encoded video data stream to obtain a decoded video data stream;
step 37: rendering the decoded screen recording data stream to obtain a rendered screen recording data stream;
in the embodiment of the invention, optionally, surfaceView can be adopted to render the decoded screen recording data stream.
Step 38: and playing the rendered screen recording data stream.
In the embodiment of the invention, the device to be screened is used as a trigger end for establishing a connection channel with the device to be screened, the device to be screened can establish the connection channel with one or more devices to be screened, and receives the screen of the one or more devices to be screened, so that the application scene of the screen is expanded.
In an embodiment of the present invention, optionally, the screen projection method further includes: receiving resolution information of the screen throwing equipment, which is sent by the screen throwing equipment; wherein said rendering the decoded video data stream comprises: and rendering the decoded screen recording data stream according to the resolution information of the screen throwing equipment.
In the above embodiments of the present invention, the USB interface may be a USB1.0, USB1.1, USB2.0, or USB3.0 type interface, where the speed of USB1.1 reaches 12Mb/s, the transmission rate of USB2.0 reaches 480Mbps, which is converted into Mb of 60Mb/s, and the theoretical speed of USB3.0 is 5.0Gb/s, so that high-definition and fast transmission can be realized.
Referring to fig. 15, an embodiment of the present invention further provides a screen projection device 150, including:
a first obtaining module 151, configured to obtain information of one or more devices to be screened connected to a USB interface of the device to be screened;
a generating module 152, configured to generate a device list according to the information of the one or more devices to be screened and display the device list through a user interface;
a second obtaining module 153, configured to obtain one or more target screened devices paired with the screening device, where the target screened device is selected from the device list by a user through the user interface;
a channel establishment module 154, configured to establish a connection channel with the one or more target screened devices;
a screen recording module 155, configured to perform a screen recording operation to obtain a screen recording data stream;
the encoding module 156 is configured to encode the recording data stream to generate an encoded recording data stream;
and a first sending module 157, configured to send the encoded screen recording data stream to the one or more target screened devices through the connection channel.
Optionally, the first obtaining module 151 is configured to generate, after detecting a device to be screened connected to a USB interface of the device to be screened, query information and display the query information through a user interface, where the query information is used to query whether the device to be screened is connected to the device to be screened through an AOA protocol; if the user selects to connect with the screen-thrown device through the AOA protocol through the user interface, detecting whether the screen-thrown device connected with the USB interface of the screen-thrown device supports a USB accessory mode or not; and acquiring information of the screened device supporting the USB accessory mode.
Optionally, the screen projection device 150 further includes:
the first judging module is used for judging whether the screen-thrown equipment has USB permission or not after detecting the screen-thrown equipment connected with the USB interface of the screen-thrown equipment;
the second judging module is used for judging whether the screen equipment to be screened is matched with the screen equipment to be screened or not if the screen equipment to be screened has USB permission;
the second sending module is used for requesting the AOA version number from the screen-thrown device if the screen-thrown device is not matched with the screen-thrown device; if the AOA version number of the screen-thrown device is greater than or equal to 1, sending information of the screen-thrown device to the screen-thrown device;
and the third sending module is used for sending the information of the screen throwing equipment to the screen throwing equipment if the screen throwing equipment is matched with the screen throwing equipment.
Optionally, the screen projection device 150 further includes:
and the fourth sending module is used for sending the resolution information of the screen throwing equipment to the screen throwing equipment.
Referring to fig. 16, an embodiment of the present invention further provides a connection device 160, including:
a plurality of first USB interfaces 161, where the first USB interfaces 161 are used for wired connection with a screen switching device;
a plurality of second USB interfaces 162, where the second USB interfaces 162 are used for wired connection with the device to be switched;
processing means 163, connected to the plurality of first USB interfaces 161 and the plurality of second USB interfaces 162, for receiving, through the first USB interfaces 161, the encoded video data stream sent by the video device; and forwarding the encoded screen recording data stream to one or more corresponding screen devices through the second USB interface 162.
Referring to fig. 17, an embodiment of the present invention further provides a device 170 for being thrown on a screen, including:
a first obtaining module 171, configured to obtain information of one or more screen capturing devices connected to the USB interface of the screen capturing device;
a generating module 172, configured to generate a device list according to the information of the one or more screen-projecting devices and display the device list through a user interface;
a second obtaining module 173, configured to obtain, through the user interface, one or more target screen capturing devices that are selected by a user from the device list and paired with the screen capturing device;
a channel establishment module 174, configured to establish a connection channel with the one or more target screen projection devices;
a first receiving module 175, configured to receive, through the connection channel, the encoded screen recording data stream sent by the target screen-throwing device;
a decoding module 176, configured to decode the encoded video data stream to obtain a decoded video data stream;
a rendering module 177, configured to render the decoded screen recording data stream, to obtain a rendered screen recording data stream;
and a playing module 178, configured to play the rendered recording data stream.
Optionally, the projected screen device 170 further includes:
the second receiving module is used for receiving the resolution information of the screen throwing equipment, which is sent by the screen throwing equipment;
the rendering module 177 is configured to render the decoded recording data stream according to resolution information of the screen-projection device.
Referring to fig. 14, the embodiment of the present invention further provides an electronic device 140, including a processor 141, a memory 142, and a computer program stored in the memory 142 and capable of running on the processor 141, where the computer program, when executed by the processor 141, implements the respective processes of the above-mentioned embodiment of the screen projection method, and can achieve the same technical effects, and for avoiding repetition, will not be repeated here.
The embodiment of the invention also provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements each process of the above-mentioned screen projection method embodiment, and can achieve the same technical effects, so that repetition is avoided, and no further description is provided herein. Wherein the computer readable storage medium is selected from Read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic disk or optical disk.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The embodiments of the present invention have been described above with reference to the accompanying drawings, but the present invention is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those having ordinary skill in the art without departing from the spirit of the present invention and the scope of the claims, which are to be protected by the present invention.
Claims (12)
1. A method of screening applied to a screening apparatus, the method comprising:
acquiring information of one or more screen-thrown devices connected with a USB interface of the screen-throwing device;
generating a device list according to the information of the one or more devices to be screened and displaying the device list through a user interface;
acquiring one or more target screen-projected devices which are selected from the device list by a user through the user interface and are paired with the screen-projected devices;
establishing a connection channel with the one or more target screened devices;
performing screen recording operation to obtain a screen recording data stream;
encoding the screen recording data stream to generate an encoded screen recording data stream;
and transmitting the encoded screen recording data stream to one or more target screen-thrown devices through the connecting channel.
2. The method of claim 1, wherein obtaining information of one or more screened devices connected to a USB interface of the screening device comprises:
after detecting the screen-thrown equipment connected with the USB interface of the screen-thrown equipment, generating inquiry information and displaying the inquiry information through a user interface, wherein the inquiry information is used for inquiring whether the screen-thrown equipment is connected with the screen-thrown equipment through an AOA protocol or not;
if the user selects to connect with the screen-thrown device through the AOA protocol through the user interface, detecting whether the screen-thrown device connected with the USB interface of the screen-thrown device supports a USB accessory mode or not;
and acquiring information of the screened device supporting the USB accessory mode.
3. The method of claim 2, wherein generating the query information and displaying via the user interface further comprises:
after detecting the screen-thrown equipment connected with the USB interface of the screen-thrown equipment, judging whether the screen-thrown equipment has USB permission or not;
if the screen-thrown equipment has USB authority, judging whether the screen-thrown equipment is matched with the screen-thrown equipment or not;
if the screen-thrown device is not matched with the screen-thrown device, requesting an AOA version number from the screen-thrown device; if the AOA version number of the screen-thrown device is greater than or equal to 1, sending information of the screen-thrown device to the screen-thrown device;
and if the screen-projected equipment is matched with the screen-projected equipment, sending information of the screen-projected equipment to the screen-projected equipment.
4. The method as recited in claim 1, further comprising:
and sending the resolution information of the screen throwing equipment to the screen throwing equipment.
5. A method of screening, characterized by being applied to a connection device, the connection device comprising:
the first USB interfaces are used for being connected with the screen throwing equipment;
the second USB interfaces are used for being connected with the equipment to be screened;
the method comprises the following steps:
receiving the coded screen recording data stream sent by the screen throwing equipment through the first USB interface;
and forwarding the encoded screen recording data stream to one or more corresponding screen devices through the second USB interface.
6. A method of screening applied to a screened device, the method comprising:
acquiring information of one or more screen throwing devices connected with a USB interface of the screen throwing device;
generating a device list according to the information of the one or more screen throwing devices and displaying the device list through a user interface;
acquiring one or more target screen projection devices which are selected from the device list by a user through the user interface and are paired with the screen projection device;
establishing a connection channel with the one or more target screen projection devices;
receiving the coded screen recording data stream sent by the target screen projection equipment through the connecting channel;
decoding the encoded video data stream to obtain a decoded video data stream;
rendering the decoded screen recording data stream to obtain a rendered screen recording data stream;
and playing the rendered screen recording data stream.
7. The method of claim 6, wherein the step of providing the first layer comprises,
further comprises: receiving resolution information of the screen throwing equipment, which is sent by the screen throwing equipment;
wherein said rendering the decoded video data stream comprises:
and rendering the decoded screen recording data stream according to the resolution information of the screen throwing equipment.
8. A screen projection device, comprising:
the first acquisition module is used for acquiring information of one or more screen-thrown devices connected with the USB interface of the screen-throwing device;
the generating module is used for generating a device list according to the information of the one or more devices to be screened and displaying the device list through a user interface;
the second acquisition module is used for acquiring one or more target screened devices which are selected from the device list by a user through the user interface and are paired with the screened device;
the channel establishing module is used for establishing a connection channel with the one or more target screened devices;
the screen recording module is used for executing screen recording operation to obtain a screen recording data stream;
the coding module is used for coding the screen recording data stream and generating a coded screen recording data stream;
and the first sending module is used for sending the coded screen recording data stream to the one or more target screen-thrown devices through the connecting channel.
9. A connection apparatus, characterized by comprising:
the first USB interfaces are used for being connected with the screen switching equipment in a wired mode;
the second USB interfaces are used for being connected with the equipment to be switched on in a wired mode;
the processing device is connected with the plurality of first USB interfaces and the plurality of second USB interfaces and is used for receiving the coded screen recording data stream sent by the screen throwing equipment through the first USB interfaces; and forwarding the encoded screen recording data stream to one or more corresponding screen devices through the second USB interface.
10. A screened device, comprising:
the first acquisition module is used for acquiring information of one or more screen throwing devices connected with the USB interface of the screen throwing device;
the generating module is used for generating a device list according to the information of the one or more screen throwing devices and displaying the device list through a user interface;
the second acquisition module is used for acquiring one or more target screen projection devices which are selected from the device list by a user through the user interface and are paired with the screen projection device;
the channel establishing module is used for establishing a connection channel with the one or more target screen throwing devices;
the first receiving module is used for receiving the coded screen recording data stream sent by the target screen projection equipment through the connecting channel;
the decoding module is used for decoding the encoded video data stream to obtain a decoded video data stream;
the rendering module is used for rendering the decoded screen recording data stream to obtain a rendered screen recording data stream;
and the playing module is used for playing the rendered screen recording data stream.
11. An electronic device, comprising: a processor, a memory, and a program stored on the memory and executable on the processor, the program implementing the steps of the method of any one of claims 1 to 4 when executed by the processor, or the program implementing the steps of the method of claim 5 when executed by the processor, or the program implementing the steps of the method of any one of claims 6 to 7 when executed by the processor.
12. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the screen projection method according to any of claims 1 to 4; alternatively, the computer program when executed by a processor implements the steps of the screen projection method according to claim 5; alternatively, the computer program when executed by a processor implements the steps of the screen projection method according to any of claims 6 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310849368.4A CN116708885A (en) | 2023-07-11 | 2023-07-11 | Screen projection method, screen projection device, connection device, screen projection device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310849368.4A CN116708885A (en) | 2023-07-11 | 2023-07-11 | Screen projection method, screen projection device, connection device, screen projection device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116708885A true CN116708885A (en) | 2023-09-05 |
Family
ID=87834053
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310849368.4A Pending CN116708885A (en) | 2023-07-11 | 2023-07-11 | Screen projection method, screen projection device, connection device, screen projection device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116708885A (en) |
-
2023
- 2023-07-11 CN CN202310849368.4A patent/CN116708885A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101971618B (en) | Method for implementing rich video on mobile terminals | |
| TWI423679B (en) | Scalable video coding and decoding | |
| US20030220971A1 (en) | Method and apparatus for video conferencing with audio redirection within a 360 degree view | |
| CN108200447B (en) | Live broadcast data transmission method and device, electronic equipment, server and storage medium | |
| US6580756B1 (en) | Data transmission method, data transmission system, data receiving method, and data receiving apparatus | |
| CN110784718B (en) | Video data encoding method, apparatus, device and storage medium | |
| TW201714456A (en) | Transporting coded audio data | |
| US6661448B2 (en) | Method and system for providing and transmitting alternative video data during interruptions in video transmissions | |
| CN114503599A (en) | Using extensions in the GLTF2 scene description to support video and audio data | |
| CN113766317A (en) | Video transmission method, video transmission device, electronic equipment and storage medium | |
| KR101296059B1 (en) | System and method for storing multisource multimedia presentations | |
| CN112019877A (en) | Screen projection method, device and equipment based on VR equipment and storage medium | |
| US20040001091A1 (en) | Method and apparatus for video conferencing system with 360 degree view | |
| CN110996122B (en) | Video frame transmission method, device, computer equipment and storage medium | |
| CN110505441B (en) | Visual communication method, device and system | |
| EP3316546B1 (en) | Multimedia information live method and system, collecting device and standardization server | |
| CN112203126A (en) | Screen projection method, screen projection device and storage medium | |
| CN116708885A (en) | Screen projection method, screen projection device, connection device, screen projection device and storage medium | |
| US9118803B2 (en) | Video conferencing system | |
| TW202249493A (en) | Anchoring a scene description to a user environment for streaming immersive media content | |
| CN105812922A (en) | Multimedia file data processing method, system, player and client | |
| CN115599326A (en) | Information co-screen method, sending end, receiving end and computer readable storage medium | |
| US20210112109A1 (en) | Av1 codec for real-time video communication | |
| CN114009030A (en) | Techniques for signaling and identifying access unit boundaries | |
| CN112135163A (en) | Video playing starting method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |