WO2021249318A1 - 一种投屏方法和终端 - Google Patents

一种投屏方法和终端 Download PDF

Info

Publication number
WO2021249318A1
WO2021249318A1 PCT/CN2021/098499 CN2021098499W WO2021249318A1 WO 2021249318 A1 WO2021249318 A1 WO 2021249318A1 CN 2021098499 W CN2021098499 W CN 2021098499W WO 2021249318 A1 WO2021249318 A1 WO 2021249318A1
Authority
WO
WIPO (PCT)
Prior art keywords
screen
display
data
text information
video
Prior art date
Application number
PCT/CN2021/098499
Other languages
English (en)
French (fr)
Inventor
牟磊
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to EP21821880.8A priority Critical patent/EP4148551A4/en
Priority to US18/000,981 priority patent/US20230217081A1/en
Publication of WO2021249318A1 publication Critical patent/WO2021249318A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/4104—Peripherals receiving signals from specially adapted client devices
    • H04N21/4122—Peripherals receiving signals from specially adapted client devices additional display device, e.g. video projector
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/4104—Peripherals receiving signals from specially adapted client devices
    • H04N21/4126—The peripheral being portable, e.g. PDAs or mobile phones
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H04N21/43615—Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H04N21/4363—Adapting the video stream to a specific local network, e.g. a Bluetooth® network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/47217—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/485—End-user interface for client configuration
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/485—End-user interface for client configuration
    • H04N21/4854—End-user interface for client configuration for modifying image parameters, e.g. image brightness, contrast
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/485—End-user interface for client configuration
    • H04N21/4858—End-user interface for client configuration for modifying screen layout parameters, e.g. fonts, size of the windows
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/488—Data services, e.g. news ticker
    • H04N21/4882—Data services, e.g. news ticker for displaying messages, e.g. warnings, reminders
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/488—Data services, e.g. news ticker
    • H04N21/4884—Data services, e.g. news ticker for displaying subtitles

Definitions

  • This application relates to the field of terminal technology, and in particular to a screen projection method and terminal.
  • Airplay and DLNA can push streaming media for accessing video applications (such as iQiyi, Youku, Youtube, etc.) on a smart phone to a large screen, such as a smart TV, to provide users with a smooth streaming media push experience. Therefore, devices that support Airplay or DLNA can discover other devices that support the same protocol in the same local area network and initiate video projection. At the same time, some screencasting applications can also implement streaming media push between two smart devices through the above protocols or private protocols.
  • video applications such as iQiyi, Youku, Youtube, etc.
  • the screencasting application when the screencasting application is pre-installed on a smart phone and a smart TV, the user can scan the smart TV
  • the two-dimensional code on the screen is used for screen projection, and the screen projection through the screen projection application does not require the two devices to be in the same local area network.
  • the development of streaming media applications has further promoted users' demand for entertainment and participation in various streaming media.
  • Barrage refers to the commentary subtitles that pop up when watching a video on the Internet. The user can view the video while watching the video. The barrage can also be commented simultaneously to form a barrage. Unlike traditional video comments, barrage will only appear at a specific point in time in the video. Therefore, barrage sent at the same time basically has the same theme.
  • participating in comments it will provide users with a The experience of other users interacting in real time.
  • This application provides a screen projection method, an electronic device and a terminal for screen projection.
  • the screen projection method realizes the screen projection of text information and improves the user's screen projection experience.
  • the present application provides a screen projection method for a source device with a display screen and a speaker, characterized in that the method includes: receiving and decoding first data from a server, and displaying on the display screen The first data that has been decoded, and/or the first data that has been decoded is output through the speaker; text information is received from the server and parsed, and the parsed text information is displayed on the display screen , Wherein the text information is associated with the first data; the first operation of projecting the target device is detected; and in response to the first operation, the first data is sent to the target device The download address and the download address of the text information.
  • the first data is a video stream
  • the text information is a bullet screen or subtitles
  • the parsed text information is displayed on the display screen
  • the method includes: superimposing and displaying the bullet screen or subtitle on the video stream displayed on the display screen, wherein the superimposed display of the bullet screen or subtitle is associated with the video stream in time.
  • the download address of the video stream and the bullet screen/subtitles can be sent to the target device, thereby realizing the simultaneous projection of the video stream and the bullet screen/subtitles.
  • the method further includes: sending a display parameter of the text information to the target device, wherein the display parameter includes the font displayed by the text information, At least one of font size, color, transparency, animation speed, and display area.
  • the display parameter includes the font displayed by the text information, At least one of font size, color, transparency, animation speed, and display area.
  • the method further includes: sending first text information to the target device, the first text information being associated with the first data.
  • the user can also send text messages with the source device to enhance the user's interactive experience.
  • the method further includes: detecting a second operation of terminating the screen projection on the target device; and receiving the display parameter of the text information from the target device, wherein The display parameter includes at least one of the font, font size, color, transparency, animation speed, and display area of the text information display.
  • the display parameter includes at least one of the font, font size, color, transparency, animation speed, and display area of the text information display.
  • the present application provides a screen projection method for a source device having a first display screen and a first speaker.
  • the method is characterized in that the method includes: The first display screen displays the decoded first data, and/or outputs the decoded first data through the first speaker; in response to the first operation of projecting the first data, Sending the download address of the first data to the target device; and sending the download address of the text information to the target device in response to the second operation of projecting the text information, wherein the text information and the The first data is associated.
  • the first data is a video stream
  • the text information is a bullet screen or subtitles
  • the method further includes: passing the download address of the first data ,
  • the target device downloads the first part of the video stream, and displays the first part of the video stream on the second display screen of the target device; through the download address of the text information, the target device downloads the Barrage or subtitles, and superimpose and display the first part of the barrage or subtitles on the second display screen, wherein the superimposed display of the first part of the barrage or subtitles and the first part of the video stream are in time ⁇ Related.
  • the download address of the video stream and the bullet screen/subtitles can be sent to the target device, thereby realizing the simultaneous projection of the video stream and the bullet screen/subtitles.
  • the method further includes: sending a display parameter of the text information to the target device, wherein the display parameter includes the font displayed by the text information, At least one of font size, color, transparency, animation speed, and display area.
  • the display parameter includes the font displayed by the text information, At least one of font size, color, transparency, animation speed, and display area.
  • the method further includes: sending first text information to the target device, the first text information being associated with the first data.
  • the user can also send text messages with the source device to enhance the user's interactive experience.
  • the method further includes: detecting a third operation of terminating the screen projection on the target device; and receiving the display parameter of the text information from the target device, wherein The display parameter includes at least one of the font, font size, color, transparency, animation speed, and display area of the text information display.
  • the display parameter includes at least one of the font, font size, color, transparency, animation speed, and display area of the text information display.
  • the present application provides a screen projection method for projecting on a target device with a display screen and a speaker, characterized in that the method includes: receiving the download address of the first data; The download address of the first data, requesting the delivery of the first data from the server, and displaying the first data through the display screen, and/or outputting the first data through the speaker; Download address, wherein the text information is associated with the first data; and through the download address of the text information, request the server to issue the text information, and display the text on the display screen information.
  • the first data is a video stream
  • the text information is a bullet screen or subtitles
  • displaying the text information on the display screen includes: The barrage or subtitles are superimposed and displayed on the video stream displayed on the display screen, wherein the barrage or subtitles that are superimposedly displayed are temporally related to the displayed video stream.
  • the method further includes: receiving a display parameter of the text information, and displaying the text information on the display screen according to the display parameter, wherein, the display parameter includes at least one of the font, font size, color, transparency, animation speed, and display area of the text information display.
  • the method further includes: detecting a first operation of sending the first text information; and in response to the first operation, superimposing on the display screen The first text information is displayed, wherein the first text information is associated with the first data. After the screen is cast, the user can also send text messages through the target device to enhance the user's interactive experience.
  • the method further includes: detecting a second operation of setting the display parameter of the text information; and in response to the second operation, according to the set The display parameter displays the text information on the display screen, wherein the display parameter includes at least one of the font, font size, color, transparency, animation speed, and display area of the text information display.
  • the user can also set the display parameters of the text information through the target device, thereby enhancing the user's interactive experience.
  • the method further includes: receiving a request from the source device to terminate the screen projection; and sending the set display parameters to the source device.
  • the parameters set by the user on the target device can be sent to the source device, so that the user's setting of displaying the text information on the target device can be continued on the source device.
  • the present application provides a terminal including a display screen, a memory, and a processor, wherein the memory stores one or more programs, and is characterized in that when the processor is executing the one or more programs
  • the terminal is made to implement the method in any one of the foregoing implementation manners.
  • the method described in any of the foregoing implementation manners is implemented. In this way, the terminal can realize the projecting of bullet screens or subtitles, and superimpose the display on the projected video, which can improve the user’s screen projecting experience.
  • the present application provides a computer-readable storage medium, including instructions, characterized in that, when the instructions run on an electronic device, the electronic device is caused to execute the method described in any of the above implementations .
  • the present application provides an electronic device for projecting a screen, with a display screen
  • the electronic device includes: a communication module for receiving first data and text information from a server, wherein the text information and The first data is associated; a decoding and parsing unit for decoding the first data and parsing the text information; a display unit for displaying the decoded first data and displaying the parsed The text information, wherein the text information is superimposed and displayed on the first data; and a detection unit configured to detect the first operation of projecting the target device; wherein, in response to the detected first operation In operation, the communication module sends the download address of the first data and the download address of the text information to the target device.
  • the electronic device realizes the screen projection of text information, which can improve the user's screen projection experience.
  • the first data is a video stream
  • the text information is a bullet screen or subtitles, wherein the bullet screen or subtitles and the video stream are in time ⁇ Related.
  • the electronic device further includes: a text control unit, configured to notify the communication module to send the first text information to the communication module when the second operation of sending the first text information is detected
  • the target device sends first text information, and the first text information is associated with the first data.
  • Sending the text information edited by the user through the text control unit can provide the user with a real-time interactive experience.
  • the present application provides an electronic device for screen projection.
  • the electronic device includes: a communication module for further receiving a download of first data from the source device when a screen projection request from the source device is received Address and a download address of text information, wherein the text information is associated with the first data, and the first data download address and the download address of the text information are used to request and receive the first data from the server A data and the text information; and a display unit for displaying the first data, and superimposing and displaying the text information on the first data.
  • the electronic device can simultaneously display the first data and the text information associated with it on the screen, thereby improving the user's screen projection experience.
  • the first data is a video stream
  • the text information is a bullet screen or subtitles
  • the display unit superimposes and displays the data on the video stream.
  • the barrage or subtitles are described, and the barrage or subtitles superimposedly displayed are temporally associated with the video stream.
  • the text control unit is configured to notify the display unit when the first operation to send the first text information is detected, so as to display the data on the first data
  • the first text information is superimposed and displayed, wherein the first text information is associated with the first data.
  • Sending the text information edited by the user through the text control unit can provide the user with a real-time interactive experience.
  • FIG. 1A is a schematic structural diagram of an electronic device provided by an embodiment of this application.
  • FIG. 1B is a software structure block diagram of an electronic device provided by an embodiment of this application.
  • Figure 2 is a schematic diagram of a network architecture provided by an embodiment of the application.
  • FIG. 3 is a schematic diagram of a scene of a screen projection method provided by an embodiment of the application.
  • FIG. 4 is a schematic diagram of a module architecture of an intelligent terminal provided by an embodiment of the application.
  • FIG. 5 is a message sequence diagram of a screen projection method provided by an embodiment of the application.
  • FIG. 6 is a message sequence diagram of a screen projection method provided by an embodiment of the application.
  • FIG. 7 is a message sequence diagram of a screen projection method provided by an embodiment of the application.
  • FIG. 8 is a message sequence diagram of a screen projection method provided by an embodiment of the application.
  • FIG. 9 is a flowchart of a screen projection method provided by an embodiment of the application.
  • FIG. 10 is a flowchart of a screen projection method provided by an embodiment of the application.
  • first and second in the embodiments of the present application are used to distinguish different messages, devices, modules, applications, etc., and do not represent a sequence, nor do they limit the "first” And “second” is a different type.
  • the descriptions of “first”, “second”, etc. also do not limit the number.
  • the "first application” can be one “first application” or multiple “first applications”.
  • a and/or B in the embodiments of the present application is merely an association relationship describing associated objects, which means that there can be three types of relationships, for example, A alone, A and B exist at the same time, and B alone exists. condition.
  • the character "/" in the embodiment of the present application generally indicates that the associated objects before and after are in an "or" relationship.
  • FIG. 1A shows a schematic structural diagram of an electronic device 100.
  • the electronic device 100 such as a smart terminal, may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, and a battery 142, Antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, earphone interface 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, A display screen 194, and a subscriber identification module (SIM) card interface 195, etc.
  • SIM subscriber identification module
  • the sensor module 180 can include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, and ambient light Sensor 180L, bone conduction sensor 180M, etc.
  • the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100.
  • the electronic device 100 may include more or fewer components than shown, or combine certain components, or split certain components, or arrange different components.
  • the illustrated components can be implemented in hardware, software, or a combination of software and hardware.
  • the processor 110 may include one or more processing units.
  • the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), and an image signal processor. (image signal processor, ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural-network processing unit (NPU), etc.
  • AP application processor
  • modem processor modem processor
  • GPU graphics processing unit
  • image signal processor image signal processor
  • ISP image signal processor
  • controller video codec
  • digital signal processor digital signal processor
  • DSP digital signal processor
  • NPU neural-network processing unit
  • the different processing units may be independent devices or integrated in one or more processors.
  • the controller can generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching and executing instructions.
  • a memory may also be provided in the processor 110 to store instructions and data.
  • the memory in the processor 110 is a cache memory.
  • the memory can store instructions or data that have just been used or recycled by the processor 110. If the processor 110 needs to use the instruction or data again, it can be directly called from the memory. Repeated accesses are avoided, the waiting time of the processor 110 is reduced, and the efficiency of the system is improved.
  • the processor 110 may include one or more interfaces.
  • the interface can include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, and a universal asynchronous transmitter (universal asynchronous) interface.
  • I2C integrated circuit
  • I2S integrated circuit built-in audio
  • PCM pulse code modulation
  • UART universal asynchronous transmitter
  • MIPI mobile industry processor interface
  • GPIO general-purpose input/output
  • SIM subscriber identity module
  • USB Universal Serial Bus
  • the I2C interface is a bidirectional synchronous serial bus, which includes a serial data line (SDA) and a serial clock line (SCL).
  • the processor 110 may include multiple sets of I2C buses.
  • the processor 110 may couple the touch sensor 180K, charger, flash, camera 193, etc., respectively through different I2C bus interfaces.
  • the processor 110 may couple the touch sensor 180K through an I2C interface, so that the processor 110 and the touch sensor 180K communicate through the I2C bus interface to implement the touch function of the electronic device 100.
  • the I2S interface can be used for audio communication.
  • the PCM interface can also be used for audio communication to sample, quantize and encode analog signals.
  • the UART interface is a universal serial data bus used for asynchronous communication.
  • the MIPI interface can be used to connect the processor 110 with the display screen 194, the camera 193 and other peripheral devices.
  • the MIPI interface includes camera serial interface (camera serial interface, CSI), display serial interface (display serial interface, DSI), etc.
  • the processor 110 and the camera 193 communicate through a CSI interface to implement the shooting function of the electronic device 100.
  • the processor 110 and the display screen 194 communicate through a DSI interface to realize the display function of the electronic device 100.
  • the GPIO interface can be configured through software.
  • the GPIO interface can be configured as a control signal or as a data signal.
  • the GPIO interface can be used to connect the processor 110 with the camera 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, and so on.
  • the GPIO interface can also be configured as an I2C interface, I2S interface, UART interface, MIPI interface, etc.
  • the USB interface 130 is an interface that complies with the USB standard specification, and specifically may be a Mini USB interface, a Micro USB interface, a USB Type C interface, and so on. It can be understood that the interface connection relationship between the modules illustrated in the embodiment of the present invention is merely a schematic description, and does not constitute a structural limitation of the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection modes in the foregoing embodiments, or a combination of multiple interface connection modes.
  • the charging management module 140 is used to receive charging input from the charger.
  • the power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110.
  • the wireless communication function of the electronic device 100 can be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor.
  • the antenna 1 and the antenna 2 are used to transmit and receive electromagnetic wave signals.
  • Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization.
  • antenna 1 can be multiplexed as a diversity antenna of a wireless local area network.
  • the antenna can be used in combination with a tuning switch.
  • the mobile communication module 150 may provide a wireless communication solution including 2G/3G/4G/5G and the like applied to the electronic device 100.
  • the mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), and the like.
  • the mobile communication module 150 can receive electromagnetic waves by the antenna 1, and perform processing such as filtering, amplifying and transmitting the received electromagnetic waves to the modem processor for demodulation.
  • the mobile communication module 150 can also amplify the signal modulated by the modem processor, and convert it into electromagnetic wave radiation via the antenna 1.
  • at least part of the functional modules of the mobile communication module 150 may be provided in the processor 110.
  • at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be provided in the same device.
  • the modem processor may include a modulator and a demodulator.
  • the modulator is used to modulate the low frequency baseband signal to be sent into a medium and high frequency signal.
  • the demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal.
  • the demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing.
  • the low-frequency baseband signal is processed by the baseband processor and then passed to the application processor.
  • the application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.), or displays an image or video through the display screen 194.
  • the modem processor may be an independent device.
  • the modem processor may be independent of the processor 110 and be provided in the same device as the mobile communication module 150 or other functional modules.
  • the wireless communication module 160 can provide applications on the electronic device 100 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), and global navigation satellites.
  • WLAN wireless local area networks
  • BT wireless fidelity
  • GNSS global navigation satellite system
  • FM frequency modulation
  • NFC near field communication technology
  • infrared technology infrared, IR
  • the wireless communication module 160 may be one or more devices integrating at least one communication processing module.
  • the wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110.
  • the wireless communication module 160 may also receive a signal to be sent from the processor 110, perform frequency modulation, amplify it, and convert it into electromagnetic waves to radiate through the antenna 2.
  • the antenna 1 of the electronic device 100 is coupled with the mobile communication module 150, and the antenna 2 is coupled with the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology.
  • the wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), broadband Code division multiple access (wideband code division multiple access, WCDMA), time-division code division multiple access (time-division code division multiple access, TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC , FM, and/or IR technology, etc.
  • the GNSS may include the global positioning system (GPS), the global navigation satellite system (GLONASS), the Beidou navigation satellite system (BDS), and the quasi-zenith satellite system (quasi). -zenith satellite system, QZSS) and/or satellite-based augmentation systems (SBAS).
  • GPS global positioning system
  • GLONASS global navigation satellite system
  • BDS Beidou navigation satellite system
  • QZSS quasi-zenith satellite system
  • SBAS satellite-based augmentation systems
  • the electronic device 100 implements a display function through a GPU, a display screen 194, an application processor, and the like.
  • the GPU is a microprocessor for image processing, connected to the display 194 and the application processor.
  • the GPU is used to perform mathematical and geometric calculations and is used for graphics rendering.
  • the processor 110 may include one or more GPUs that execute program instructions to generate or change display information.
  • the display screen 194 is used to display images, videos, and the like.
  • the display screen 194 includes a display panel.
  • the display panel can adopt liquid crystal display (LCD), organic light-emitting diode (OLED), active-matrix organic light-emitting diode or active-matrix organic light-emitting diode (active-matrix organic light-emitting diode).
  • LCD liquid crystal display
  • OLED organic light-emitting diode
  • active-matrix organic light-emitting diode active-matrix organic light-emitting diode
  • AMOLED flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diode (QLED), etc.
  • the electronic device 100 may include one or N display screens 194, and N is a positive integer greater than one.
  • the electronic device 100 can realize a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.
  • the ISP is used to process the data fed back from the camera 193.
  • the camera 193 is used to capture still images or videos.
  • the object generates an optical image through the lens and is projected to the photosensitive element.
  • Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the energy of the frequency point.
  • Video codecs are used to compress or decompress digital video.
  • the electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple encoding formats, such as: moving picture experts group (MPEG) 1, MPEG2, MPEG3, MPEG4, and so on.
  • MPEG moving picture experts group
  • MPEG2 MPEG2, MPEG3, MPEG4, and so on.
  • NPU is a neural-network (NN) computing processor.
  • NN neural-network
  • the external memory interface 120 may be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100.
  • the external memory card communicates with the processor 110 through the external memory interface 120 to realize the data storage function. For example, save music, video and other files in an external memory card.
  • the internal memory 121 may be used to store computer executable program code, where the executable program code includes instructions.
  • the internal memory 121 may include a storage program area and a storage data area.
  • the storage program area can store an operating system, at least one application program (such as a sound playback function, an image playback function, etc.) required by at least one function.
  • the data storage area can store data (such as audio data, phone book, etc.) created during the use of the electronic device 100.
  • the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), and the like.
  • the processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121 and/or instructions stored in a memory provided in the processor.
  • the internal memory is also used to store the translation application and cache all pictures generated during the running of the translation application, and all the pictures cached can be automatically deleted after the user exits the translation application.
  • the electronic device 100 can implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor. For example, music playback, recording, etc.
  • the audio module 170 is used to convert digital audio information into an analog audio signal for output, and is also used to convert an analog audio input into a digital audio signal.
  • the audio module 170 can also be used to encode and decode audio signals.
  • the audio module 170 may be disposed in the processor 110, or part of the functional modules of the audio module 170 may be disposed in the processor 110.
  • the speaker 170A also called “speaker” is used to convert audio electrical signals into sound signals.
  • the electronic device 100 can listen to music through the speaker 170A, or listen to a hands-free call.
  • the receiver 170B also called a "handset" is used to convert audio electrical signals into sound signals.
  • the electronic device 100 answers a call or voice message, it can receive the voice by bringing the receiver 170B close to the human ear.
  • the microphone 170C also called “microphone”, “microphone”, is used to convert sound signals into electrical signals.
  • the earphone interface 170D is used to connect wired earphones.
  • the earphone interface 170D may be a USB interface 130, or a 3.5mm open mobile terminal platform (open mobile terminal platform, OMTP) standard interface, and a cellular telecommunications industry association (cellular telecommunications industry association of the USA, CTIA) standard interface.
  • OMTP open mobile terminal platform
  • CTIA cellular telecommunications industry association of the USA
  • the pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal.
  • the pressure sensor 180A may be provided on the display screen 194.
  • the capacitive pressure sensor may include at least two parallel plates with conductive materials.
  • the electronic device 100 determines the intensity of the pressure according to the change in capacitance.
  • the electronic device 100 detects the intensity of the touch operation according to the pressure sensor 180A.
  • the electronic device 100 may also calculate the touched position according to the detection signal of the pressure sensor 180A.
  • the gyro sensor 180B may be used to determine the movement posture of the electronic device 100.
  • the air pressure sensor 180C is used to measure air pressure.
  • the electronic device 100 calculates the altitude based on the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.
  • the magnetic sensor 180D includes a Hall sensor.
  • the acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in various directions (generally three axes).
  • Distance sensor 180F used to measure distance.
  • the electronic device 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 may use the distance sensor 180F to measure the distance to achieve fast focusing.
  • the proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector such as a photodiode.
  • the ambient light sensor 180L is used to sense the brightness of the ambient light.
  • the fingerprint sensor 180H is used to collect fingerprints.
  • the electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photographs, fingerprint answering calls, and so on.
  • the temperature sensor 180J is used to detect temperature.
  • Touch sensor 180K also called “touch device”.
  • the touch sensor 180K may be provided on the display screen 194, and the touch screen is composed of the touch sensor 180K and the display screen 194, which is also called a “touch screen”.
  • the touch sensor 180K is used to detect touch operations acting on or near it.
  • the touch sensor can pass the detected touch operation to the application processor to determine the type of touch event.
  • the visual output related to the touch operation can be provided through the display screen 194.
  • the touch sensor 180K may also be disposed on the surface of the electronic device 100, which is different from the position of the display screen 194.
  • the bone conduction sensor 180M can acquire vibration signals.
  • the button 190 includes a power-on button, a volume button, and so on.
  • the button 190 may be a mechanical button. It can also be a touch button.
  • the electronic device 100 may receive key input, and generate key signal input related to user settings and function control of the electronic device 100.
  • the motor 191 can generate vibration prompts.
  • the indicator 192 may be an indicator light, which may be used to indicate the charging status, power change, or to indicate messages, missed calls, notifications, and so on.
  • the SIM card interface 195 is used to connect to the SIM card.
  • the SIM card can be inserted into the SIM card interface 195 or pulled out from the SIM card interface 195 to achieve contact and separation with the electronic device 100.
  • the electronic device 100 may support 1 or N SIM card interfaces, and N is a positive integer greater than 1.
  • the SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc.
  • the same SIM card interface 195 can insert multiple cards at the same time. The types of the multiple cards can be the same or different.
  • the SIM card interface 195 can also be compatible with different types of SIM cards.
  • the SIM card interface 195 can also be compatible with external memory cards.
  • the electronic device 100 interacts with the network through the SIM card to implement functions such as call and data communication.
  • the electronic device 100 adopts an eSIM, that is, an embedded SIM card.
  • the eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.
  • the software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture.
  • the embodiment of the present invention takes an Android system with a layered architecture as an example to illustrate the software structure of the electronic device 100 by way of example.
  • FIG. 1B is a software structure block diagram of the electronic device 100 shown in FIG. 1A according to an embodiment of the present invention.
  • the layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Communication between layers through software interface.
  • the Android system is divided into four layers, from top to bottom, the application layer, the application framework layer, the Android runtime and system library, and the kernel layer.
  • the application layer can include a series of application packages.
  • the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc.
  • the application framework layer provides an application programming interface (application programming interface, API) and a programming framework for applications in the application layer.
  • the application framework layer includes some predefined functions.
  • the application framework layer can include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and so on.
  • the window manager is used to manage window programs.
  • the window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, take a screenshot, etc.
  • the content provider is used to store and retrieve data and make these data accessible to applications.
  • the data may include video, image, audio, phone calls made and received, browsing history and bookmarks, phone book, etc.
  • the view system includes visual controls, such as controls that display text, controls that display pictures, and so on.
  • the view system can be used to build applications.
  • the display interface can be composed of one or more views.
  • a display interface that includes a short message notification icon may include a view that displays text and a view that displays pictures.
  • the phone manager is used to provide the communication function of the electronic device 100. For example, the management of the call status (including connecting, hanging up, etc.).
  • the resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, and so on.
  • the notification manager enables the application to display notification information in the status bar, which can be used to convey notification-type messages, and it can automatically disappear after a short stay without user interaction.
  • the notification manager is used to notify download completion, message reminders, and so on.
  • the notification manager can also be a notification that appears in the status bar at the top of the system in the form of a chart or a scroll bar text, such as a notification of an application running in the background, or a notification that appears on the screen in the form of a dialog window.
  • prompt text information in the status bar sound a prompt sound, electronic device vibration, flashing indicator light, etc.
  • Android Runtime (ie Android Runtime) includes core libraries and virtual machines. Android runtime is responsible for the scheduling and management of the Android system.
  • the core library consists of two parts: one part is the function function that the java language needs to call, and the other part is the core library of Android.
  • the application layer and application framework layer run in a virtual machine.
  • the virtual machine executes the java files of the application layer and the application framework layer as binary files.
  • the virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.
  • the system library can include multiple functional modules. For example: surface manager (surface manager), media library (Media Libraries), three-dimensional graphics processing library (for example: OpenGL ES), two-dimensional graphics engine (for example: SGL), etc.
  • the surface manager is used to manage the display subsystem and provides a combination of 2D and 3D layers for multiple applications.
  • the media library supports playback and recording of a variety of commonly used audio and video formats, as well as still image files.
  • the media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.
  • the 3D graphics processing library is used to realize 3D graphics drawing, image rendering, synthesis, and layer processing.
  • the two-dimensional graphics engine is a drawing engine for 2D drawing.
  • the kernel layer is the layer between hardware and software.
  • the kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.
  • the corresponding hardware interrupt is sent to the kernel layer.
  • the kernel layer processes the touch operation into the original input event (including touch coordinates, time stamp of the touch operation, etc.).
  • the original input events are stored in the kernel layer.
  • the application framework layer obtains the original input event from the kernel layer and identifies the control corresponding to the input event. Taking the touch operation as a touch click operation, and the control corresponding to the click operation is the control of the camera application icon as an example, the camera application calls the interface of the application framework layer to start the camera application, and then starts the camera driver by calling the kernel layer.
  • the camera 193 shown in FIG. 1A captures still images or video.
  • the above-mentioned display may be displayed on a display screen, which has a display function, and the display screen may or may not have a touch function.
  • the operation on the touch screen can be realized by virtual keys, and can be realized by clicking on the touch screen.
  • the operation on the non-touch screen can be realized by physical buttons.
  • FIG. 2 shows a schematic diagram 200 of a network architecture provided by an embodiment of the present application.
  • the aforementioned electronic device 100 such as a smart terminal such as a smart phone and a smart TV, can be used to work in conjunction with the network architecture 200.
  • the network 202 can take any form, including but not limited to telecommunications networks, local area networks (LAN for short), wireless networks, and wide area networks such as the Internet, peer-to-peer networks, and wired networks. network, WAN for short). Although only one network is shown, it should be understood that two or more similar or different networks 202 may be provided.
  • the server computer 212 and the end user computer 208 may be connected to the network 202 for communication, and such end user computer 208 may include a desktop computer, a laptop computer, and/or any other type of logic.
  • various other devices may also be connected to the network 202, including a personal digital assistant (personal digital assistant, PDA for short) device 210, a mobile phone device 206, a TV set 204, and so on.
  • PDA personal digital assistant
  • a mobile phone device 206 such as a smart phone
  • a television 204 such as a smart TV
  • the network 202 such as a local area network
  • the smart phone 206 or smart TV 204 and the server 212 can also be Connect to the network 202 (for example, a wide area network) for communication for users to access server data on the terminal device, such as accessing or downloading video streams.
  • the following embodiments of this application provide a screen projection method, so that smart terminals, such as smart phones and smart TVs, can push and display streaming media data more conveniently and quickly during interconnection.
  • smart terminals such as smart phones and smart TVs
  • the large screen of the smart TV can simultaneously display video and at least one of bullet screen information or external subtitle information.
  • FIG. 3 shows a schematic diagram of a scene of a screen projection method provided by the present application, such as a streaming data screen projection method.
  • the first electronic device such as the first smart terminal 310
  • the first smart terminal 310 is watching a video on the first application.
  • the first smart terminal 310 can be a mobile phone, a tablet computer, a smart wearable, a smart home device, etc.
  • the first smart terminal 310 has a relatively high user frequency and a relatively small screen. device of.
  • the first application can be any video website client, such as iQiyi, Youku, Tencent, etc., or it can be an application that supports playing video sources in any video website, such as WeChat, Weibo, etc.
  • an operation interface can also be displayed, as shown in Figure 3a, Part of the playback control interface related to the video content is displayed in, including but not limited to the screen projection key 302, the barrage key 304, the barrage setting key 306, and the barrage sending box 308.
  • the operation interface can be displayed on the video screen 311, or can be displayed in a different display area from the video screen 311, which is not limited in this application.
  • the barrage or barrage information 313 or external subtitle information can be displayed along with the video screen by default.
  • the animation and video screen of the bullet screen 313 are played on the first application of the first smart terminal 310, the user can use the bullet screen setting key 306 and the bullet screen sending box 308 to respectively set the display parameters of the bullet screen and send new bullets. Screen information.
  • the first smart terminal 310 detects the screen projection operation and starts to search for screen projection devices.
  • the searched device results may be presented on the smart terminal 310 in the form of a list for the user to select, and based on the user's selection, the smart terminal 310 sends a screen projection request to the selected device to perform the screen projection. In other cases, based on the user's selection, the smart terminal 310 communicates with the selected device to send a screen projection request.
  • the smart terminal 310 in response to the user clicking the screen projection button 302, or in response to the user selecting a screen projection device, may also display a menu bar or pop-up window as shown in FIG. 3b or 3c.
  • the smart terminal 310 displays a menu bar as shown in 3b Or a pop-up window for the user to select 301 "Video projection” or 303 "Video and bullet screen projection", so as to perform video projection or both video and bullet screen projection according to the actual needs of the user.
  • the user selects a second electronic device, such as the second smart terminal 320, to perform screen projection.
  • the second smart terminal 320 may have a larger display than the first smart terminal 310 Screen.
  • the first smart terminal 310 determines whether barrage and/or external subtitles are displayed in the current video playback content. If the result of the determination is yes, then When pushing the video to the second smart terminal 320, barrage and/or external subtitles are simultaneously pushed. As shown in FIG. 3d, when the screen is started, the video screen 321 and the barrage 323 are displayed on the display screen of the second smart terminal 320 at the same time.
  • the first application of the first smart terminal 310 receives the user’s click on the bullet screen.
  • the barrage information of the video screen 321 will be projected onto the display screen of the second smart terminal 320, that is, the barrage or barrage information 323 will be displayed.
  • the user can still use the first smart terminal
  • the playback controls on the terminal 310 control the video screen 321 displayed on the second smart terminal 320, including but not limited to switching videos, fast forward, fast reverse, volume control, etc.
  • the user can also control the second smart terminal 310 through the first smart terminal 310.
  • the barrage 323 displayed on the smart terminal 320 is manipulated, including but not limited to adjusting barrage parameters, sending a new barrage, and so on.
  • the user can also directly control the video screen 321 and the barrage 323 on the second smart terminal 320, including but not limited to the aforementioned manipulation operations.
  • the second smart terminal 320 can display the bullet screen and/or external subtitles at the same time, it can continue to provide the user with a scene where the bullet screen interacts while watching the video, and/or view the corresponding subtitles while watching the video Scene. Therefore, the present application provides users with a convenient and quick solution for displaying bullet screens and/or subtitles on the screen, which satisfies the user's entertainment while watching videos, and can greatly improve the user's screen projection experience.
  • the first smart terminal 310 in this application may be a portable smart device, and the second smart terminal 320 may be a smart TV. After the video and barrage displayed on the portable smart device 310 are projected onto the smart TV 320 as shown in 3d, The portable smart device 310 can also exit the first application program or exit the video display screen as shown in FIG. 3e to perform other operations. Therefore, the first smart terminal 310 in the embodiment of the present application realizes projection of videos and bullet screens, and can freely and flexibly choose a projection scheme.
  • the “barrage” mentioned in this application refers to the commentary content of the video provided by the client of the video website that can be displayed at the same time as the video, usually superimposed on the top of the video screen, and any user watching the video You can add your own barrage information for interaction, and the added barrage information will be displayed in the barrage display area.
  • the settings mentioned in this application through the barrage setting key can adjust the "display parameters" of the barrage.
  • the display parameters can include but are not limited to the following barrage parameters: font, font size, color, transparency, animation speed, display area , Block user ID (account name), etc.
  • the "external subtitles” mentioned in this application is a kind of subtitles that are different from the embedded subtitles of the video.
  • the data of the external subtitles are separated from the video data, but not integrated. It needs to be imported separately to be displayed on the video screen, usually superimposed Displayed at the bottom of the video screen.
  • the first smart terminal 310 "projecting" refers to at least a part of the content displayed on the display screen of the first smart terminal 310 and at least a part of the data related to the displayed content, including But it is not limited to video, audio, text, etc., which are projected onto another electronic device, such as the second smart terminal 320.
  • the "devices that can be screened" search for can be electronic devices that support screen projection in the same LAN (including Layer 2 LAN, connected to the same router), or it can be different from the same LAN.
  • the screen-casting device searched by the first smart terminal is a device in the same local area network that supports the same screen-casting protocol, where the screen-casting protocol can be a wireless screen-casting protocol, such as Airplay, DLNA, etc. Technology or agreement.
  • FIG. 4 shows a schematic diagram of the module architecture of the smart terminal
  • FIG. 4a also shows a schematic diagram of the architecture and communication connections of the first smart terminal 401, the second smart terminal 402, and the video server 410 of the first application.
  • the first smart terminal 401 includes a communication module 411, a data judgment unit 412, a video decoding unit 413, a text parsing unit 414, a display module 415, a video control unit 416, and a text control unit 417, wherein the display module 415 also A video display unit 418 and a text display unit 419 may be included.
  • the first smart terminal 401 may also include more functional modules. As shown in Figure 1A, it will not be repeated here.
  • the first smart terminal 401 in the embodiment of the present application may be a smart phone.
  • the first communication 431 is performed with the video server 410, so as to obtain the video data and text data in the video server 410, and display them on the first terminal.
  • the display module 415 of the smart terminal 401 is performed with the video server 410, so as to obtain the video data and text data in the video server 410, and display them on the first terminal.
  • the display module 415 of the smart terminal 401 is performed with the video server 410, so as
  • the second smart terminal 402 includes a communication module 421, a data judgment unit 422, a video decoding unit 423, a text analysis unit 424, and a display module 425.
  • the display module 425 may also include a video display unit 428 and a text display unit 429.
  • the second smart terminal 402 in the embodiment of the present application may be a smart TV, which communicates with the first smart terminal 401, such as a smart phone, through a screen projection protocol. In an embodiment of the present application, the second communication 432 is performed between the smart TV and the smart phone.
  • the second smart terminal 402 then performs a third communication 433 with the video server 410 to obtain the video data and text data in the video server 410, and display them on the display module of the second smart terminal 402 425 to cast the screen.
  • the first communication 431, the second communication 432, and the third communication 433 may be any form of communication, such as a wired or wireless communication method, which is not limited in this application.
  • the first smart terminal 401 and the second smart terminal 402 are in a local area network and are connected to a router, and the above-mentioned first communication 431, second communication 432, and third communication 433 may be implemented via the router.
  • the second smart terminal 402 may also include a video control unit 426 and a text control unit 427 (indicated by a dashed box in the figure) to control the playback of the projected video and the corresponding text display .
  • FIG. 4 only shows the modules related to video playback and text display in the second smart terminal 402. It should be understood that the second smart terminal 402 may also include more functional modules. As shown in Figure 1A, it will not be repeated here.
  • the text analysis unit 414/424 may include a bullet screen analysis unit 4141 and/or a subtitle analysis unit 4142; the text display unit 419/429 may include a bullet screen display unit 4191 and/or a subtitle display unit 4192; a text control unit 417/427 may include a barrage control unit 4171 and/or a subtitle control unit 4172, where the text control unit 417/427 may also include a barrage sending unit 4173.
  • the communication modules 411, 421 of the first and second smart terminals 401, 402 are used to perform first communication 431 and third communication 433 with the video server 410, respectively, so as to communicate with each other to request and deliver streaming media and related data.
  • the first smart terminal 401 receives video data and text data from the video server 410; through the second communication 432, the second smart terminal 402 receives video from the first smart terminal 401 The data of the download address and the text download address; through the third communication 433, the second smart terminal 402 receives video data and text data from the video server 410.
  • the data judging units 412, 422 of the first and second smart terminals 401, 402 are used to judge the type of data received from the video server 410, and transmit different data to different units according to the judgment result, such as video data It is sent to the video decoding unit 413 and 423, and the text data is sent to the text parsing unit 414 and 424.
  • the data judging unit 412, 422 judges that the barrage data is received, the barrage data will be sent to the barrage parsing unit 4141 shown in Figure 4b; when the data judging unit 412, 422 judges that the received barrage data is subtitles When data, the subtitle data will be sent to the subtitle analysis unit 4142 shown in Figure 4b.
  • the video decoding unit 413, 423 decodes the received video data and transmits it to the video display unit 418, 428 for display on the display screen of the smart terminal.
  • the text analysis unit 414, 424 analyzes the received text data and transmits it to the text display unit 419, 429 to be superimposed and displayed on the video screen played on the display screen.
  • the barrage analysis unit 4141 parses the barrage data to obtain barrage information, and sends it to the barrage display unit 4191.
  • the barrage display unit 4191 renders the analyzed barrage information and displays it on the display screen of the smart terminal.
  • the subtitle analysis unit 4142 parses the subtitle data and sends it to the subtitle display unit 4192, and the subtitle display unit 4192 renders the analyzed subtitle data and displays it on the display screen of the smart terminal , And is usually superimposed on the bottom area of the video screen to display.
  • the video control units 416 and 426 are used to control the video displayed on the display screen, including but not limited to controlling the pause, play, fast forward, drag and drop, and switch of the video.
  • the text control unit 417, 427 is used to control the text displayed on the display screen.
  • the bullet screen control unit 4171 is used to control the bullet screen information on the video being played, including but not limited to controlling the playback speed, text font, color, size, display area, etc. of the bullet screen information;
  • the subtitle control unit 4172 uses To control the subtitle data of the video being played, including but not limited to controlling the font, color, size, display area, switching subtitles, etc. of the subtitle.
  • the text control units 417, 427 may also include a bullet screen sending unit 4173 for sending bullet screen information on the smart terminal.
  • the bullet screen sending unit 4173 receives the bullet screen information input by the user (for example, the user’s bullet screen as shown in Figure 3a). Send the information entered in the box 308), and send the barrage information to the video server 410 to update the barrage data packet corresponding to the video on the video server side and deliver it to the corresponding client in the form of streaming media.
  • the barrage sending unit 4173 sends the barrage information to another smart terminal and displays it on its display screen, and/or sends it to the local barrage display units 419, 429 to display the barrage information. Barrage, so that the new barrage sent by the user can be displayed on the display screen in real time, providing users with a better barrage sending experience.
  • the embodiment of this application only takes video projection as an example, it describes in detail the module architecture of the smart terminal and the projection solution of text information such as bullet screens. It should be understood that in other embodiments of this application, other data can be performed. Projection, such as audio data. When the projected audio data contains text information such as bullet screens and/or lyrics, the bullet screens and/or lyrics information can be simultaneously projected according to the above embodiment. It should be understood that, in this embodiment, the projected screen The audio data will be parsed in the local audio decoding unit of the smart terminal and played in the audio output unit. In the process of projecting audio or other data, the project of projecting text information such as bullet screens and/or lyrics at the same time should be similar to the above-mentioned embodiment, so it will not be repeated here.
  • FIGS. 5 to 8 will be performed in conjunction with the architecture diagram shown in FIG. 4 and the scene diagram shown in FIG. 3.
  • the following will take the display of the bullet screen and the sending of the bullet screen as an example to illustrate the data request and issuance between the smart terminal and the video server of the present application.
  • FIG. 5 to 8 are message sequence diagrams of the screen projection method provided by the embodiments of the application, for example, the streaming data screen projection method.
  • FIG. 5 shows a sequence diagram 500 of screen projection between the source device and the screen projection device.
  • the source device is the terminal device 501
  • the screen projection device is the terminal device 502.
  • the terminal device 501 opens the application 1 and selects the video, and then requests video data from the video server 510 in 513.
  • the video server 510 delivers video data such as video to the terminal device 501. Therefore, in 515, when the terminal device 501 determines that the received data is video data through the data determination unit 412 as shown in FIG. 4, the video data is transmitted to the video decoding unit 413 for decoding, and the video data is decoded. After that, the video is displayed on the video display unit 418 of the terminal device 501, that is, the video screen is displayed on the display screen.
  • the video server 510 sends an authorized video download address to the terminal device 501 in response to a request of the terminal device 501 to obtain a video, so that the terminal device 501 can download the video.
  • the video download address may be a URL (Uniform Resource Locator, Uniform Resource Locator) address.
  • the terminal device 501 detects the operation of opening the barrage (for example, clicking the barrage button 304 shown in FIG. 3), in 517, the video server 510 requests the barrage data, and in response to the request, in 518, the video server 510 sends the barrage data to the terminal device 501. Therefore, in 519, the terminal device 501, as shown in FIG. The unit 414 performs analysis. After the barrage data is analyzed, the barrage information is displayed on the text display unit 419 of the terminal device 501, that is, the barrage information is superimposed on the video screen to display the barrage information, as shown in Figure 3a, which is usually displayed in the video The top of the screen. The display of subtitles is not shown in the figure.
  • the video server 510 responds to a request from the terminal device 501 to obtain a bullet screen, and issues an authorized bullet screen download address (for example, URL) to the terminal device 501, so that the terminal device 501 downloads the bullet screen data.
  • the bullet screen can be an animation played on the display screen, which can roll in or slide in from the right side to the left side of the display screen of the terminal device 501, or it can be displayed on the display screen with other animation effects. Move the display in other directions. It should be understood that the animation effect of the barrage should not be used as a limitation to this application.
  • the source device 501 When the source device 501 detects the operation of clicking the screen projection button 302, in 521, the source device 501 selects the screen projection device 502 to send a screen projection request to perform a screen projection. In 525, the source device 501 determines whether the barrage information is being displayed in the application 1. If the result of the determination is yes, in 526, when requesting the authorized streaming data download address from the video server 510, it also requests the video data download address and Barrage data download address. In response to the request of the source device 501, in 526, the video server 510 issues the authorized video and barrage data download address to the source device 501. In 528, the source device 501 sends the received authorized download address to the screen projection device 502.
  • steps 526 and 527 can be skipped.
  • the video server 510 issues the authorized video data download address and the authorized barrage data download address, so ,
  • the source device 501 may directly perform step 528 without going through steps 526 and 527.
  • the authorized video data and the barrage data download address can be added to the screen projection request sent by the terminal device 501 to the terminal device 502.
  • the terminal device 501 when the user is watching iQIYI's video on the terminal device 501, when the terminal device 501 judges that the subtitles and the external barrage are displayed at the same time, the terminal device 501 reports to the terminal device 502 adds the download address of the video authorized by iQiyi in the screencast request sent by 502, as well as the download address of the barrage and external subtitles.
  • a terminal device 501 such as a smart phone, serves as a Control Point in DLNA
  • a terminal device 502 such as a smart TV
  • a video server 510 such as an iQiyi server, serves as a Media Server.
  • the smart phone Control Point after the smart phone Control Point selects the video to be played, it pushes the download address of the video source to the smart TV Media Renderer, such as judging the current video of the iQIYI APP (application)
  • the corresponding barrage information and/or external subtitle information are also superimposed and displayed, and the smartphone ControlPoint pushes the download address of the barrage source and/or the download address of the external subtitle source to the Smart TV Media Renderer.
  • the Smart TV Media Renderer obtains the video source, the barrage source, and/or the external subtitle source file from the Media Server of the iQiyi server according to the received video source, the barrage source, and/or the download address of the external subtitle source, iQiyi
  • the way that the art server Media Server transmits these files to the Smart TV Media Renderer can be, for example, HTTP/HTTPS.
  • the Smart TV Media Renderer After receiving the video source, barrage source, and/or external subtitle source files, the Smart TV Media Renderer renders these files, and displays the video image on the display screen, and superimposes the barrage and/or external subtitles.
  • the following fields can be added to the information pushed by the Smart Phone Control Point to the Smart TV Media Renderer through the DLNA protocol:
  • the projection device 502 requests the video server 510 for the video data and the bullet screen data respectively according to the received video download address and the bullet screen download address, and sends the video to the projection device 502 in 532 and 536 respectively.
  • Data and barrage data Therefore, in 533 and 537, as shown in Figure 4, the projection device 502 transmits the part determined as video data to the video decoding unit 423 through the data determination unit 422 for decoding, and transmits the part determined as barrage data.
  • the text parsing unit 424 is parsed, and the parsed video and barrage information are loaded and displayed on the display screen of the terminal device 502 through the video display unit 428 and the text display unit 429, respectively.
  • the barrage data is related to the playing time of the video, that is, part of the barrage data is only related to the video A certain segment of is related and should be displayed on the screen when the video is played.
  • the video displayed on the projection device 502 should be the video screen that is cut after the user clicks the projection button on the source device or the cut-off video screen before and after it. At the same time, it is displayed on the projection device 502 on the 537
  • the barrage information of should be the barrage information associated with the cut video screen or its front and back screens.
  • the source device 501 plays the video Video1 to 03'30′′ on the application 1
  • the user clicks the screen projection button
  • the projection device 502 performs the projection on the screen
  • the video should be from 03'30′′ or Before and after, for example, the screen starting from 03'28
  • the barrage displayed on the video screen should be the barrage information associated with the video screens before or after 03'30".
  • an example of a part of the parsed barrage file received by the projection device 502 is as follows:
  • the multiple data separated by commas are data related to the display parameter of this barrage of "Excited, Excited, Excited".
  • the data is exemplary and can be explained as follows:
  • the first parameter "51.593" is the time when the barrage appears in the video, in seconds;
  • the second parameter "5" is the animation mode of the barrage, where 1 to 3 represent three different forms of rolling barrage, 4 represents the bottom barrage, 5 represents the top barrage, 6 represents the reverse barrage, and 7 represents Precise positioning, 8 represents advanced barrage, etc.;
  • the third parameter "25" shows the font size of the bullet screen, where 12 represents very small, 16 represents very small, 18 represents small, 25 represents medium, 36 represents large, 45 represents large, 64 represents extremely large, etc. ;
  • the fourth parameter "16711680" is the font color displayed on the bullet screen, and this embodiment uses the tens digit of HTML color;
  • the fifth parameter "1408852480” is a timestamp in Unix format
  • the sixth parameter "0" is the barrage pool, where 0 represents the normal pool, 1 represents the subtitle pool, 2 represents the special pool (the special pool is dedicated to advanced barrage), and so on;
  • the seventh parameter "7fa769b4" is the account ID of the sender of the barrage, which can be used for the function of "blocking the sender of this barrage";
  • the eighth parameter "576008622" is the rowID representing the sequence of the barrage in the barrage database, which can be used for the "historical barrage” function.
  • the screen projection device 502 After the screen projection device 502 receives, for example, the above-mentioned bullet screen file, it renders it on the local player, and then superimposes it on the video screen played on the display screen.
  • the relevant code for rendering the bullet screen in the smart TV player will be shown as an example below. It should be understood that these codes are only used as examples and should not be used as a display of the present invention.
  • the source device 501 may send the current bullet screen display parameters to the screen projection device 502, so that when the screen projection device 502 displays the bullet screen, it is matched accordingly.
  • the barrage parameters displayed on the source device such as displaying four rows of barrage information in blue in the largest font, and so on.
  • the source device 501 when the source device 501 initiates a screen projection request, it actively determines whether barrage and/or external subtitles are displayed in the screen projection video, and based on the result of the determination being yes, Actively send to the screen projection device 502 the authorized download address of the bullet screen data and/or external subtitle data, so that the screen projection device 502 requests the streaming data from the video server 510.
  • the embodiment of the present application realizes that the video and the bullet screen ( And/or plug-in subtitles) data streaming project.
  • the two communicate through wired or wireless, which can ensure the smoothness of video playback and barrage display in the projection screen, and improve the user’s projection. Screen experience.
  • the source device 501 can continue to control the projection state in 539, or exit the current projection video interface, and browse other video guides in Application 1. , Or exit the application program 1 to browse the interface of other applications, or enter the screen rest state, without causing interference to the video and barrage display on the screen projection device 502.
  • the user can also use the user interaction interface displayed on the source device 501 to control, adjust the display of the barrage, and send barrage information, for example, the barrage key 304 in Figure 3a can be used to terminate or open the barrage.
  • the display parameters of the bullet screen on the projection device can be adjusted through the bullet screen setting key 306, or the bullet screen can be sent to display on the projection device through the bullet screen sending box 308.
  • an exemplary part of the code is as follows:
  • part of the pseudo code can be as follows:
  • FIG. 6 shows a timing diagram 600 of adjusting the barrage parameters during the screen projection process.
  • the user makes settings through the user interaction screen on the terminal device 501, that is, the source device, and adjusts the display parameters of the bullet screen through the text control unit 417.
  • a barrage setting window (not shown in the figure) pops up, which may include but is not limited to setting the barrage: transparency (0-100%) , Font size (small-standard-large-extra large), playback speed (slow-normal-2x-3x), display area (10-50%), color, animation effects, etc.
  • the source device 501 detects that the setting of the barrage parameters is completed, in 603, the source device 501 sends the updated barrage parameters to the screen projection device 502.
  • the text control unit 427 of the screen projection device 502 adjusts the barrage displayed on the text display unit 429 according to the barrage parameters, and the adjusted barrage display corresponds to the user’s display on the source device
  • the setting made on 501 or corresponding to the display setting after the text control unit 427 makes corresponding adaptations to some parameters.
  • the user can also set the display parameters of the bullet screen on the screen projection device 502, for example, through the remote control of the smart TV to control the user interaction screen on the display screen for setting, and adjust the bullet screen through the text display unit 429 Display parameters.
  • step 613 is further included.
  • the screen projection device 502 sends the updated barrage display parameters to the source device 501 to update the user's adjustment/change of the barrage display parameters on the source device 501.
  • the user can freely choose whether to personalize the display of the bullet screen information on the source device or the projection device. Even if the parameter settings of the bullet screen display are performed on the source device, it can be adapted accordingly. In the current projection screen on the projection device, the setting operation is very convenient and flexible. Correspondingly, the user can also update the display settings of the bullet screen information on the projection device to the source device, so that when the user switches back to the source device to watch the video, he can see the bullet screen that has been adapted accordingly. , No need to set again.
  • FIG. 7 shows a sequence diagram 700 of a user sending a bullet screen through two devices during a screen projection process.
  • the user edits the barrage information to be sent through the barrage sending box 308 on the user interaction interface of the terminal device 501, and sends the barrage information through the barrage sending unit in the text control unit 417, when
  • the terminal device 501 detects that the user inputs the bullet screen information and clicks to send, the terminal device 501 encapsulates the bullet screen information as bullet screen data and sends the bullet screen data to the terminal device 502.
  • the text parsing unit 424 of the terminal device 502 parses the barrage data to obtain new barrage information.
  • the parsed barrage information is loaded/displayed by the text display unit 429 In the currently displayed video screen, the newly added barrage information may slide into the display screen from the upper right side of the display screen with a sliding-in animation effect.
  • the terminal device 501 may also send the barrage data to the video server 510, and in 705, the video server 510 updates the corresponding barrage data in response to the received barrage data. In 706, the updated barrage data packet is sent to the terminal device 502, so that the terminal device 502 displays the newly added barrage in step 707.
  • the terminal device 501 still displays barrage information on its display screen after the screen is projected, the barrage information newly sent by the user on its user interaction interface will also appear as a sliding-in animation. The effect is displayed on the display screen of the terminal device 501.
  • the user can also edit the barrage information on the user interaction interface on the terminal device 502, and send the barrage information through the barrage sending unit in the text control unit 427.
  • the specific operation can be, for example, through The remote control of the smart TV inputs text information in the user interaction interface on the display screen and clicks to send.
  • the terminal device 502 detects that the user clicks to send the barrage information, it encapsulates it as barrage data and sends it to the video server 510.
  • the video server 510 updates the corresponding barrage data packet, and in 716, sends the updated barrage data packet to the terminal device 502.
  • the terminal device 502 controls the text display unit 429 through the text control unit 427, so that the new barrage information is displayed on the display screen of the terminal device 502 with a sliding-in animation effect.
  • steps 715 and 716 can be omitted.
  • the new barrage sent by the user on the terminal device 502 will be directly displayed on the display screen of the terminal device 502 instead of receiving the update from the video server 510. Only after the barrage data appeared.
  • the terminal device 502 may also include the barrage data of the new barrage information in 712
  • the text display unit 417 displays the barrage information newly sent by the user with the animation effect of sliding in on the terminal device 501. On the display.
  • an exemplary part of the code is as follows:
  • the user can have a real-time interactive experience through the display of the bullet screen and the transmission of the bullet screen when watching the video, the user can still send a new bullet screen from the source device or the projection device during the video projection process. And when watching the video, I saw the new barrage just sent, and continued to participate in the "real-time interaction".
  • the projection device has a larger display screen than the source device, so that users can watch videos on a large screen and "interact in real time", which enhances the interaction experience.
  • FIG. 8 shows a timing diagram 800 in which video images and barrage information continue to be displayed on the display screen of the source device 501 when the screen projection between the two devices is terminated.
  • the terminal device 501 detects that the user terminates the screen projection operation, for example, clicks the screen projection button 302 shown in FIG. 3 to end the screen projection, the terminal device 501 sends a termination request to the terminal device 502, in response to the termination request,
  • the terminal device 502 controls the display module 425 through the video control unit 426 and the text control unit 427 to stop displaying the current video screen and barrage information
  • the terminal device 502 and the video server 510 terminate data transfer Send and receive.
  • the terminal device 501 requests the video server 510 for the currently interrupted video data and barrage data.
  • the video server 510 delivers the currently interrupted video screen to the terminal device 501 or the start of the screen before and after the interruption.
  • the decoded video and barrage data are transmitted on the terminal device 501 through the video decoding unit 413 and the text parsing unit 414, and displayed on the display module 415.
  • the user sets the display parameters of the bullet screen through the screen projection device 502, because at this time no video screen and/or bullet screen is displayed on the source device 501 Therefore, the parameter changes made on the screen projection device 502 are not further sent to the source device 501.
  • the screen projection device 502 determines whether the barrage display parameters have been changed during the projection process. If the result of the judgment is yes, then in 807 , Send the currently updated barrage display parameters to the source device 501. Based on the bullet screen display parameters received from the screen projection device 502, the source device 501 will correspond to the parameters and make corresponding adaptations when displaying the bullet screen on the display screen.
  • the 613 shown in FIG. 6 may not be executed, that is, the change of the barrage display setting may not be updated to On the source device 501; and, when the user terminates the screen projection, the 805 and 807 shown in FIG. 8 may not be executed, that is, the change of the barrage display setting is still not updated on the source device 501.
  • the user can reserve a set of settings ConfigA for the barrage display on the source device 501, and reserve another set of settings ConfigB for the barrage display on the screen projection device 502, and the user can at any time in the source
  • the user can at any time in the source
  • only the parameters in ConfigA or ConfigB will be changed, and the other parameter will not be affected, so as to meet the different display parameters of the bullet screen on different devices. need.
  • FIG. 9 is a flowchart 900 of a method for projecting a screen according to an embodiment of the application, such as a method for projecting a screen of streaming data.
  • FIG. 9 shows a flowchart 900 of a method for streaming data projection performed on a terminal device 501, that is, a source device.
  • the source device 501 plays a video on a first application, such as iQiyi, Youku, and other apps.
  • the video played on the source device 501 is a video stream sent from a server such as iQiyi or Youku.
  • the server sends the download address of the video stream requested by the source device 501 to be played to the source device 501.
  • the source device 501 can download the corresponding video stream data, decode it locally by the video decoding unit 413, and play it.
  • the played video screen is displayed on the source device 501 through the video display unit 418 in the display module 415. On the display.
  • the source device 501 detects the user's screen projection operation on the first application interface. For example, when the source device 501 detects that the user clicks the screen projection button 302 on the video screen, it will pop up the screencast operation detected in the current environment. Tabs for the list of devices that can be screened. Among them, the current environment refers to, for example, the local area network where the current source device 501 is located. In another embodiment of the present application, the source device 501 can detect other smart devices logged in by the current system account, and use these devices as screen-casting devices. In step 903, if the source device 501 does not detect the screen projection operation, it continues to play the current video.
  • the first application program may respond to the user having checked the launch barrage before exiting the application last time, and the barrage is turned on by default when the video is played this time.
  • the video screen currently displayed by the source device 501 may also superimpose and display barrage information related to the video.
  • the server of the first application program may provide external subtitles related to the video, and in response to the user selecting to play the video, automatically issue the data of the external subtitles.
  • the source device 501 may also superimpose and display external subtitle information related to the video in the currently displayed video screen received.
  • the source device 501 detects that the user selects the terminal device 502 for screen projection.
  • the terminal device 502 may be a device that is in the same local area network as the source device and supports the same projection protocol, such as a smart TV.
  • step 907 the source device 501 detects under the first application whether a bullet screen is displayed in the currently played video, that is, it detects whether the bullet screen function is currently enabled.
  • step 909 is executed.
  • step 909 the source device 501 sends to the screen projection device 502 the download address of the bullet screen data and the download address of the video data authorized by the video server of the first application, such as the bullet screen download URL and the video download URL.
  • the screen projection device 502 After the screen projection device 502 receives the data download address sent by the source device 501, it obtains the video stream and barrage data through the access address and the video server.
  • the video decoding unit 423 and text parsing unit 424 of the projection device 502 decode and parse the acquired video stream and barrage data respectively, and display the decoded and parsed data through the video display unit 428 and text display unit 429 of the display module 425, respectively Video and barrage information, where the barrage information is loaded and superimposed on the video screen, and the notification is displayed on the top of the video screen.
  • FIG. 10 is a flowchart 1000 of a method for projecting a screen according to an embodiment of the application, such as a method for projecting a screen of streaming data.
  • FIG. 10 shows a flowchart 1000 of a method for streaming data screen projection executed on a terminal device 502, that is, a screen projection device.
  • step 1001 the screen projection device 502 receives a screen projection request from the terminal device 501, that is, the source device.
  • step 1003 the screen projection device 502 receives the download address of the video data and the download address of the barrage data from the source device 501.
  • the first judging unit on the screen projection device 502 respectively judges the part belonging to the video data download address and the part belonging to the barrage data download address to request the video stream data and the barrage data respectively.
  • the download address of the video data and the bullet screen data is the authorized download address issued by the video server 510 of the first application to the source device 501.
  • the download address may be used by the source device 501 when requesting the video from the video server 510.
  • the video server 510 sends to the source device 501, or when the source device 501 prepares to project to the projection device 502, the download address requested from the video server, that is, the source device 501 detects that the user clicks the projection button 302 Then, at the same time as the screen projection invitation is sent to the screen projection device 502, a request is sent to the video server 510 to request the issuance of the authorized video data download address and the barrage data download address.
  • step 1005 in response to the received video data download address and barrage data download address, the screen projection device 502 accesses the address to request video stream data and barrage data.
  • the video server 510 determines that the screen projection device 502 is an authorized device, it delivers the requested video stream and barrage data packet to the screen projection device 502.
  • step 1007 the screen projection device 502 receives the video stream and the barrage data packet sent by the video server.
  • the data judging unit on the screen projection device 502 judges the type of the received data, and sends the video data to the video decoding unit and the barrage data to the text parsing unit for respective analysis.
  • step 1009 the display screen of the screen projection device 502 plays the video, and at the same time superimposes the bullet screen on the video screen for display.
  • the video decoding unit 423 of the projection device 502 decodes the received video stream and transmits it to the video display unit 428 of the display unit 425 to display the decoded video picture on the display screen of the projection device 502.
  • the text analysis unit 424 of the screen projection device 502 parses the received barrage data and transmits it to the text display unit 429 of the display unit 425, so as to superimpose the analyzed barrage information on the video screen and display it on the projection screen.
  • the embodiments of the present application can also implement subtitle projection and subtitle projection.
  • the subsequent display and control should be understandable.
  • the process of downloading, parsing, and displaying subtitles is similar to that of a bullet screen, and the setting/parameter adjustment of subtitles is also similar to that of a bullet screen.
  • the projection process of subtitle data will not be repeated here, so In this application, a detailed description of the projecting scheme of subtitles will not be given.
  • the smart device can, according to the user’s projection requirements, not only perform video projection to the projection device, but also provide text information such as barrage and/or subtitles related to the video.
  • the projection display and update operations of these text data are beneficial to enhance the user's overall experience when watching the video, and the interactive experience when watching the video.
  • the embodiment of the present invention provides a very convenient and quick solution for projecting bullet screens and subtitles that match the video, as well as a flexible bullet screen sending solution and flexible adjustment of the bullet screen. /Subtitle display parameter scheme, etc., which greatly improves the user's experience and interactive experience when watching the video.
  • This application provides a computer program product containing instructions.
  • the computer program product runs on a terminal (for example, the aforementioned terminal device 501 or 502), the terminal executes a streaming data screen projection provided by the embodiment of this application. Steps in the method.
  • the present application provides a computer-readable storage medium, including instructions, which when run on a terminal, cause the terminal to execute the steps in a streaming data screen projection method provided in an embodiment of the present application.
  • the embodiments of the present application can be implemented in hardware, or in hardware and software. When implemented by hardware and software, the above functions can be stored in a computer-readable medium. Based on this understanding, the technical solutions of the embodiments of the present application are essentially or the part that contributes to the prior art, or all or part of the technical solutions can be embodied in the form of software products, and the computer software products are stored in a storage
  • the medium includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods described in the various embodiments of the present application.
  • the aforementioned storage media include: flash memory, mobile hard disk, read-only memory, random access memory, magnetic disk or optical disk and other media that can store program codes.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)
  • Telephone Function (AREA)

Abstract

一种投屏方法和终端。其中，投屏方法用于具有显示屏和扬声器的源设备，该投屏方法包括：从服务器接收第一数据并解码，并通过显示屏显示已解码的第一数据，和/或通过扬声器输出已解码的第一数据；从服务器接收文本信息并解析，并通过显示屏显示已解析的文本信息，其中，文本信息与第一数据相关联；检测到对目标设备进行投屏的第一操作；以及响应于该第一操作，向目标设备发送第一数据的下载地址以及文本信息的下载地址。该方法实现了对文本信息与第一数据的投屏。

Description

一种投屏方法和终端
本申请要求在2020年6月8日提交中国国家知识产权局、申请号为202010514417.5的中国专利申请的优先权，发明名称为“一种投屏方法和终端”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及终端技术领域，特别是涉及一种投屏方法和终端。
背景技术
随着智能终端的发展，例如手机、平板电脑、智能穿戴、智能电视、智能家居等等，用户将多种终端设备进行有线或无线连接、以及在终端设备之间分享本地数据或多媒体数据的场景越来越普遍，进行多设备连接和分享的操作也越来越便捷。屏幕镜像、多屏互动、流媒体推送等是目前比较主流的无线投屏方式，主要由Airplay、Miracast、WiDi、DLNA等技术或协议支持。支持流模式，即支持仅流式传输视频的内容而不是镜像复制整个显示界面的协议如Airplay和DLNA，其中，Airplay是苹果公司在iOS中的隔空播放技术，DLNA的全称是数字生活网络联盟(DIGITAL LIVING NETWORK ALLIANCE)，实现个人电脑、消费电器、移动设备在内的无线网络和有线网络的互联互通，并提供在一个设备上获取内容并在另一台设备上播放内容的方法。
具体地，Airplay和DLNA可以将智能手机上访问视频应用(例如爱奇艺，优酷，Youtube等)的流媒体推送到大屏幕，例如智能电视上，来为用户提供流畅的流媒体推送体验。因此，支持Airplay或DLNA的设备，在同一局域网中能够发现支持同一协议的其他设备并发起视频投屏。同时，有一些投屏应用也可以通过上述协议或私有协议来实现两个智能设备之间的流媒体推送，例如智能手机和智能电视上都预装该投屏应用时，用户可以通过扫描智能电视上的二维码来进行投屏，通过投屏应用进行投屏并不要求两个设备在同一个局域网内。与此同时，流媒体应用的发展进一步推动了用户对各种流媒体的娱乐性和参与感的需求。例如，用户在爱奇艺上观看电视剧或综艺节目时，可以体验“弹幕”的功能，弹幕是指是在网络上观看视频时弹出的评论性字幕，用户在观看视频时可以查看该视频的弹幕，也可以同步发表评论形成弹幕。不同于传统的对视频的评论，弹幕只会在视频中特定的一个时间点出现，因此在相同时刻发送的弹幕基本上具有相同的主题，在参与评论时就会为用户提供一种与其他用户实时互动的体验感受。
通过上述投屏协议，例如Airplay和DLNA,或者投屏应用来进行投屏的方法，并不能在流式传输视频的同时，推送与该视频相应的弹幕，因此，需要提供一种可以同时投屏视频和弹幕的流式投屏方法。
发明内容
本申请提供一种投屏方法、用于投屏的电子设备和终端，该投屏方法实现了文本信息的投屏，提升了用户的投屏体验。
第一方面，本申请提供了一种投屏方法，用于具有显示屏和扬声器的源设备，其特征在于，所述方法包括：从服务器接收第一数据并解码，并通过所述显示屏显示已解码的所述第一数据，和/或通过所述扬声器输出已解码的所述第一数据；从所述服务器接收文本信息并解析，并通过所述显示屏显示已解析的所述文本信息，其中，所述文本信息与所述第一数据相关联；检测到对目标设备进行投屏的第一操作；以及响应于所述第一操作，向所述目标设备发送所述第一数据的下载地址以及所述文本信息的下载地址。
根据第一方面，在第一方面的第一实现方式中，所述第一数据是视频流，所述文本信息为弹幕或字幕，其中，通过所述显示屏显示已解析的所述文本信息包括：在显示屏上显示的所述视频流上叠加显示所述弹幕或字幕，其中，叠加显示的所述弹幕或字幕与所述视频流在时间上相关联。通过该投屏方法，可以将视频流和弹幕/字幕的下载地址发送给目标设备，从而实现视频流和弹幕/字幕的同时投屏。
根据第一方面，在第一方面的第二实现方式中，所述方法还包括：向所述目标设备发送所述文本信息的显示参数，其中所述显示参数包括所述文本信息显示的字体、字号、颜色、透明度、动画速度、显示区域中的至少一个。通过向目标设备发送文本信息的显示参数，可以将用户在源设备端设置的参数应用于目标设备端。
根据第一方面，在第一方面的第三实现方式中，所述方法还包括：向所述目标设备发送第一文本信息，所述第一文本信息与所述第一数据相关联。设备向目标设备投屏后，用户还可以通过与源设备发送文本信息，从而增强用户的互动体验。
根据第一方面，在第一方面的第四实现方式中，所述方法还包括：检测到对目标设备终止投屏的第二操作；以及从目标设备接收所述文本信息的显示参数，其中所述显示参数包括所述文本信息显示的字体、字号、颜色、透明度、动画速度、显示区域中的至少一个。在终止投屏时，可以将用户在目标设备上设置的参数应用于源设备端，从而延续用户在目标设备上显示所述文本信息的设置。
第二方面，本申请提供了一种投屏方法，用于具有第一显示屏和第一扬声器的源设备，其特征在于，所述方法包括：从服务器接收第一数据并解码，并通过所述第一显示屏显示已解码的所述第一数据，和/或通过所述第一扬声器输出已解码的所述第一数据；响应于对所述第一数据进行投屏的第一操作，向目标设备发送所述第一数据的下载地址；以及响应于对文本信息进行投屏的第二操作，向所述目标设备发送所述文本信息的下载地址，其中，所述文本信息与所述第一数据相关联。
根据第二方面，在第二方面的第一实现方式中，所述第一数据是视频流，所述文本信息为弹幕或字幕，所述方法还包括：通过所述第一数据的下载地址，所述目标设备下载所述视频流的第一部分，并通过所述目标设备的第二显示屏显示所述视频流的第一部分；通过所述文本信息的下载地址，所述目标设备下载所述弹幕或字幕，并在所述第二显示屏上叠加显示所述弹幕或字幕的第一部分，其中，叠加显示的所述弹幕或字幕的第一部分与所述视频流的第一部分在时间上相关联。通过该投屏方法，可以将视频流和弹幕/字幕的下载地址发送给目标设备，从而实现视频流和弹幕/字幕的同时投屏。
根据第二方面，在第二方面的第二实现方式中，所述方法还包括：向所述目标设备发送所述文本信息的显示参数，其中所述显示参数包括所述文本信息显示的字体、字号、颜色、透明度、动画速度、显示区域中的至少一个。通过向目标设备发送文本信息的显示参数，可以将用户在源设备端设置的参数应用于目标设备端。
根据第二方面，在第二方面的第三实现方式中，所述方法还包括：向所述目标设备发送第一文本信息，所述第一文本信息与所述第一数据相关联。设备向目标设备投屏后，用户还可以通过与源设备发送文本信息，从而增强用户的互动体验。
根据第二方面，在第二方面的第四实现方式中，所述方法还包括：检测到对目标设备终止投屏的第三操作；以及从目标设备接收所述文本信息的显示参数，其中所述显示参数包括所述文本信息显示的字体、字号、颜色、透明度、动画速度、显示区域中的至少一个。在终止投屏时，可以将用户在目标设备上设置的参数应用于源设备端，从而延续用户在目标设备上显示所述文本信息的设置。
第三方面，本申请提供了一种投屏方法，用于在具有显示屏和扬声器的目标设备上投屏，其特征在于，所述方法包括：接收到第一数据的下载地址；通过所述第一数据的下载地址，从服务器请求下发所述第一数据，并通过所述显示屏显示所述第一数据，和/或通过所述扬声器输出所述第一数据；接收到文本信息的下载地址，其中，所述文本信息与所述第一数据相关联；以及通过所述文本信息的下载地址，从所述服务器请求下发所述文本信息，并通过所述显示屏显示所述文本信息。
根据第三方面，在第三方面的第一实现方式中，所述第一数据是视频流，所述文本信息为弹幕或字幕，其中，通过所述显示屏显示所述文本信息包括：在所述显示屏上显示的所述视频流上叠加显示所述弹幕或字幕，其中，叠加显示的所述弹幕或字幕与显示的所述视频流在时间上相关联。通过该投屏方法，可以从源设备接收视频流和弹幕/字幕的下载地址，从而实现视频流和弹幕/字幕的同时投屏。
根据第三方面，在第三方面的第二实现方式中，所述方法还包括：接收到所述文本信息的显示参数，并根据所述显示参数在所述显示屏上显示所述文本信息，其中，所述显示参数包括所述文本信息显示的字体、字号、颜色、透明度、动画 速度、显示区域中的至少一个。通过从源设备接收文本信息的显示参数，可以将用户在源设备端设置的参数应用于目标设备端。
根据第三方面，在第三方面的第三实现方式中，所述方法还包括：检测到发送第一文本信息的第一操作；以及响应于所述第一操作，在所述显示屏上叠加显示所述第一文本信息，其中，所述第一文本信息与所述第一数据相关联。投屏后，用户还可以通过目标设备发送文本信息，从而增强用户的互动体验。
根据第三方面，在第三方面的第四实现方式中，所述方法还包括：检测到设置所述文本信息的显示参数的第二操作；以及响应于所述第二操作，根据设置后的显示参数在所述显示屏上显示所述文本信息，其中，所述显示参数包括所述文本信息显示的字体、字号、颜色、透明度、动画速度、显示区域中的至少一个。投屏后，用户还可以通过目标设备设置文本信息的显示参数，从而增强用户的互动体验。
根据第三方面的第四实现方式，在第二方面的第五实现方式中，所述方法还包括：接收到源设备终止投屏的请求；以及向源设备发送所述设置后的显示参数。在终止投屏时，可以将用户在目标设备上设置的参数发送给源设备端，从而可以在源设备上延续用户在目标设备上显示所述文本信息的设置。
第四方面，本申请提供了一种终端，包括显示屏、存储器、处理器，其中，所述存储器储存有一个或多个程序，其特征在于，当所述处理器在执行所述一个或多个程序时，使得所述终端实现上述任一种实现方式中的所述的方法。实现上述任一种实现方式中的所述的方法这样，该终端可以将实现弹幕或字幕的投屏，并叠加显示于投屏视频上，能够提升用户的投屏体验。
第五方面，本申请提供了一种计算机可读存储介质，包括指令，其特征在于，当所述指令在电子设备上运行时，使得所述电子设备执行如以上任意实现方式中所述的方法。
第六方面，本申请提供了一种用于投屏的电子设备，具有显示屏，所述电子设备包括：通信模块，用于从服务器接收第一数据和文本信息，其中，所述文本信息与所述第一数据相关联；解码和解析单元，用于将所述第一数据解码，以及将所述文本信息解析；显示单元，用于显示已解码的所述第一数据，以及显示已解析的所述文本信息，其中所述文本信息叠加显示于所述第一数据上；以及检测单元，用于检测对目标设备进行投屏的第一操作；其中，响应于检测到的所述第一操作，所述通信模块向所述目标设备发送所述第一数据的下载地址以及所述文本信息的下载地址。该电子设备实现了文本信息的投屏，可以提高用户的投屏体验。
根据第六方面，在第六方面的第一实现方式中，所述第一数据是视频流，所述文本信息为弹幕或字幕，其中，所述弹幕或字幕与所述视频流在时间上相关联。
根据第六方面，在第六方面的第二实现方式中，所述电子设备还包括：文本控制单元，用于在检测到发送第一文本信息的第二操作时，通知所述通信模块向所述目标设备发送第一文本信息，所述第一文本信息与所述第一数据相关联。通 过文本控制单元发送用户编辑的文本信息，能够为用户提供一种实时互动的体验。
第七方面，本申请提供了一种用于投屏的电子设备，所述电子设备包括：通信模块，用于在接收到源设备的投屏请求时，从源设备进一步接收第一数据的下载地址和文本信息的下载地址，其中，所述文本信息与所述第一数据相关联，并且通过所述第一数据的下载地址和所述文本信息的下载地址，从服务器请求并接收所述第一数据和所述文本信息；以及显示单元，用于显示所述第一数据，并在所述第一数据上叠加显示所述文本信息。该电子设备可以同时投屏显示第一数据和与其关联的文本信息，从而可以提高用户的投屏体验。
根据第七方面，在第七方面的第一实现方式中，所述第一数据是视频流，所述文本信息为弹幕或字幕，其中，所述显示单元在所述视频流上叠加显示所述弹幕或字幕，且叠加显示的所述弹幕或字幕与所述视频流在时间上相关联。
根据第七方面，在第七方面的第二实现方式中，文本控制单元，用于在检测到发送第一文本信息的第一操作时，通知所述显示单元，以在所述第一数据上叠加显示所述第一文本信息，其中，所述第一文本信息与所述第一数据相关联。通过文本控制单元发送用户编辑的文本信息，能够为用户提供一种实时互动的体验。
附图说明
图1A为本申请实施例提供的一种电子设备的结构示意图；
图1B为本申请实施例提供的一种电子设备的软件结构框图；
图2为本申请实施例提供的一种网络架构示意图；
图3为本申请实施例提供的一种投屏方法的场景示意图；
图4为本申请实施例提供的一种智能终端的模块架构示意图；
图5为本申请实施例提供的一种投屏方法的消息序列图；
图6为本申请实施例提供的一种投屏方法的消息序列图；
图7为本申请实施例提供的一种投屏方法的消息序列图；
图8为本申请实施例提供的一种投屏方法的消息序列图；
图9为本申请实施例提供的一种投屏方法流程图；
图10为本申请实施例提供的一种投屏方法流程图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例进行说明。
需要说明的是，本申请实施例中的“第一”、“第二”等描述，是用于区分不同的消息、设备、模块、应用等，不代表先后顺序，也不限定“第一”和“第二”是不同的类型。“第一”、“第二”等描述也不限定数量，例如“第一应用”可以为一个“第一应用”，也可以为多个“第一应用”。
本申请实施例中的术语“A和/或B”,仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如单独存在A,同时存在A和B,单独存在B这三种情况。另外，本申请实施例中字符"/",一般表示前后关联对象是一种"或"的 关系。
本申请实施例提供的方法可以应用于图1A所示的电子设备100中。图1A示出了电子设备100的结构示意图。
电子设备100，例如智能终端，可以包括处理器110，外部存储器接口120，内部存储器121，通用串行总线(universal serial bus，USB)接口130，充电管理模块140，电源管理模块141，电池142，天线1，天线2，移动通信模块150，无线通信模块160，音频模块170，扬声器170A，受话器170B，麦克风170C，耳机接口170D，传感器模块180，按键190，马达191，指示器192，摄像头193，显示屏194，以及用户标识模块(subscriber identification module，SIM)卡接口195等。其中传感器模块180可以包括压力传感器180A，陀螺仪传感器180B，气压传感器180C，磁传感器180D，加速度传感器180E，距离传感器180F，接近光传感器180G，指纹传感器180H，温度传感器180J，触摸传感器180K，环境光传感器180L，骨传导传感器180M等。
可以理解的是，本发明实施例示意的结构并不构成对电子设备100的具体限定。在本申请另一些实施例中，电子设备100可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件，软件或软件和硬件的组合实现。
处理器110可以包括一个或多个处理单元，例如：处理器110可以包括应用处理器(application processor，AP)，调制解调处理器，图形处理器(graphics processing unit，GPU)，图像信号处理器(image signal processor，ISP)，控制器，视频编解码器，数字信号处理器(digital signal processor，DSP)，基带处理器，和/或神经网络处理器(neural-network processing unit，NPU)等。其中，不同的处理单元可以是独立的器件，也可以集成在一个或多个处理器中。
控制器可以根据指令操作码和时序信号，产生操作控制信号，完成取指令和执行指令的控制。
处理器110中还可以设置存储器，用于存储指令和数据。在一些实施例中，处理器110中的存储器为高速缓冲存储器。该存储器可以保存处理器110刚用过或循环使用的指令或数据。如果处理器110需要再次使用该指令或数据，可从所述存储器中直接调用。避免了重复存取，减少了处理器110的等待时间，因而提高了系统的效率。
在一些实施例中，处理器110可以包括一个或多个接口。接口可以包括集成电路(inter-integrated circuit，I2C)接口，集成电路内置音频(inter-integrated circuit sound，I2S)接口，脉冲编码调制(pulse code modulation，PCM)接口，通用异步收发传输器(universal asynchronous receiver/transmitter，UART)接口，移动产业处理器接口(mobile industry processor interface，MIPI)，通用输入输出(general-purpose input/output，GPIO)接口，用户标识模块(subscriber identity module，SIM)接口，和/或通用串行总线(universal serial bus，USB)接口等。
I2C接口是一种双向同步串行总线，包括一根串行数据线(serial data line，SDA)和一根串行时钟线(derail clock line，SCL)。在一些实施例中，处理器110可以包含多组I2C总线。处理器110可以通过不同的I2C总线接口分别耦合触摸传感器180K，充电器，闪光灯，摄像头193等。例如：处理器110可以通过I2C接口耦合触摸传感器180K，使处理器110与触摸传感器180K通过I2C总线接口通信，实现电子设备100的触摸功能。
I2S接口可以用于音频通信。PCM接口也可以用于音频通信，将模拟信号抽样，量化和编码。UART接口是一种通用串行数据总线，用于异步通信。
MIPI接口可以被用于连接处理器110与显示屏194，摄像头193等外围器件。MIPI接口包括摄像头串行接口(camera serial interface，CSI)，显示屏串行接口(display serial interface，DSI)等。在一些实施例中，处理器110和摄像头193通过CSI接口通信，实现电子设备100的拍摄功能。处理器110和显示屏194通过DSI接口通信，实现电子设备100的显示功能。
GPIO接口可以通过软件配置。GPIO接口可以被配置为控制信号，也可被配置为数据信号。在一些实施例中，GPIO接口可以用于连接处理器110与摄像头193，显示屏194，无线通信模块160，音频模块170，传感器模块180等。GPIO接口还可以被配置为I2C接口，I2S接口，UART接口，MIPI接口等。
USB接口130是符合USB标准规范的接口，具体可以是Mini USB接口，Micro USB接口，USB Type C接口等。可以理解的是，本发明实施例示意的各模块间的接口连接关系，只是示意性说明，并不构成对电子设备100的结构限定。在本申请另一些实施例中，电子设备100也可以采用上述实施例中不同的接口连接方式，或多种接口连接方式的组合。
充电管理模块140用于从充电器接收充电输入。电源管理模块141用于连接电池142，充电管理模块140与处理器110。电子设备100的无线通信功能可以通过天线1，天线2，移动通信模块150，无线通信模块160，调制解调处理器以及基带处理器等实现。
天线1和天线2用于发射和接收电磁波信号。电子设备100中的每个天线可用于覆盖单个或多个通信频带。不同的天线还可以复用，以提高天线的利用率。例如：可以将天线1复用为无线局域网的分集天线。在另外一些实施例中，天线可以和调谐开关结合使用。
移动通信模块150可以提供应用在电子设备100上的包括2G/3G/4G/5G等无线通信的解决方案。移动通信模块150可以包括至少一个滤波器，开关，功率放大器，低噪声放大器(low noise amplifier，LNA)等。移动通信模块150可以由天线1接收电磁波，并对接收的电磁波进行滤波，放大等处理，传送至调制解调处理器进行解调。移动通信模块150还可以对经调制解调处理器调制后的信号放大，经天线1转为电磁波辐射出去。在一些实施例中，移动通信模块150的至少部分功能模块可以被设置于处理器110中。在一些实施例中，移动通信模块150的至少部分功能模块可以与处理器110的至少部分模块被设置在同一个器件中。
调制解调处理器可以包括调制器和解调器。其中，调制器用于将待发送的低频基带信号调制成中高频信号。解调器用于将接收的电磁波信号解调为低频基带信号。随后解调器将解调得到的低频基带信号传送至基带处理器处理。低频基带信号经基带处理器处理后，被传递给应用处理器。应用处理器通过音频设备(不限于扬声器170A，受话器170B等)输出声音信号，或通过显示屏194显示图像或视频。在一些实施例中，调制解调处理器可以是独立的器件。在另一些实施例中，调制解调处理器可以独立于处理器110，与移动通信模块150或其他功能模块设置在同一个器件中。
无线通信模块160可以提供应用在电子设备100上的包括无线局域网(wireless local area networks，WLAN)(如无线保真(wireless fidelity，Wi-Fi)网络)，蓝牙(bluetooth，BT)，全球导航卫星系统(global navigation satellite system，GNSS)，调频(frequency modulation，FM)，近距离无线通信技术(near field communication，NFC)，红外技术(infrared，IR)等无线通信的解决方案。无线通信模块160可以是集成至少一个通信处理模块的一个或多个器件。无线通信模块160经由天线2接收电磁波，将电磁波信号调频以及滤波处理，将处理后的信号发送到处理器110。无线通信模块160还可以从处理器110接收待发送的信号，对其进行调频，放大，经天线2转为电磁波辐射出去。
在一些实施例中，电子设备100的天线1和移动通信模块150耦合，天线2和无线通信模块160耦合，使得电子设备100可以通过无线通信技术与网络以及其他设备通信。所述无线通信技术可以包括全球移动通讯系统(global system for mobile communications，GSM)，通用分组无线服务(general packet radio service，GPRS)，码分多址接入(code division multiple access，CDMA)，宽带码分多址(wideband code division multiple access，WCDMA)，时分码分多址(time-division code division multiple access，TD-SCDMA)，长期演进(long term evolution，LTE)，BT，GNSS，WLAN，NFC，FM，和/或IR技术等。所述GNSS可以包括全球卫星定位系统(global positioning system，GPS)，全球导航卫星系统(global navigation satellite system，GLONASS)，北斗卫星导航系统(beidou navigation satellite system，BDS)，准天顶卫星系统(quasi-zenith satellite system，QZSS)和/或星基增强系统(satellite based augmentation systems，SBAS)。
电子设备100通过GPU，显示屏194，以及应用处理器等实现显示功能。GPU为图像处理的微处理器，连接显示屏194和应用处理器。GPU用于执行数学和几何计算，用于图形渲染。处理器110可包括一个或多个GPU，其执行程序指令以生成或改变显示信息。
显示屏194用于显示图像，视频等。显示屏194包括显示面板。显示面板可以采用液晶显示屏(liquid crystal display，LCD)，有机发光二极管(organic light-emitting diode，OLED)，有源矩阵有机发光二极体或主动矩阵有机发光二极体(active-matrix organic light emitting diode的，AMOLED)，柔性发光 二极管(flex light-emitting diode，FLED)，Miniled，MicroLed，Micro-oLed，量子点发光二极管(quantum dot light emitting diodes，QLED)等。在一些实施例中，电子设备100可以包括1个或N个显示屏194，N为大于1的正整数。
电子设备100可以通过ISP，摄像头193，视频编解码器，GPU，显示屏194以及应用处理器等实现拍摄功能。
ISP用于处理摄像头193反馈的数据。摄像头193用于捕获静态图像或视频。物体通过镜头生成光学图像投射到感光元件。
数字信号处理器用于处理数字信号，除了可以处理数字图像信号，还可以处理其他数字信号。例如，当电子设备100在频点选择时，数字信号处理器用于对频点能量进行傅里叶变换等。
视频编解码器用于对数字视频压缩或解压缩。电子设备100可以支持一种或多种视频编解码器。这样，电子设备100可以播放或录制多种编码格式的视频，例如：动态图像专家组(moving picture experts group，MPEG)1，MPEG2，MPEG3，MPEG4等。
NPU为神经网络(neural-network，NN)计算处理器，通过借鉴生物神经网络结构，例如借鉴人脑神经元之间传递模式，对输入信息快速处理，还可以不断的自学习。
外部存储器接口120可以用于连接外部存储卡，例如Micro SD卡，实现扩展电子设备100的存储能力。外部存储卡通过外部存储器接口120与处理器110通信，实现数据存储功能。例如将音乐，视频等文件保存在外部存储卡中。
内部存储器121可以用于存储计算机可执行程序代码，所述可执行程序代码包括指令。内部存储器121可以包括存储程序区和存储数据区。其中，存储程序区可存储操作系统，至少一个功能所需的应用程序(比如声音播放功能，图像播放功能等)等。存储数据区可存储电子设备100使用过程中所创建的数据(比如音频数据，电话本等)等。此外，内部存储器121可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件，闪存器件，通用闪存存储器(universal flash storage，UFS)等。处理器110通过运行存储在内部存储器121的指令，和/或存储在设置于处理器中的存储器的指令，执行电子设备100的各种功能应用以及数据处理。
在本申请的一些实施例中，内部存储器还用于存储翻译应用和缓存该翻译应用运行过程中所产生的所有图片，用户退出该翻译应用后上述缓存的所有图片可以自动删除。
电子设备100可以通过音频模块170，扬声器170A，受话器170B，麦克风170C，耳机接口170D，以及应用处理器等实现音频功能。例如音乐播放，录音等。
音频模块170用于将数字音频信息转换成模拟音频信号输出，也用于将模拟音频输入转换为数字音频信号。音频模块170还可以用于对音频信号编码和解码。在一些实施例中，音频模块170可以设置于处理器110中，或将音频模块170 的部分功能模块设置于处理器110中。
扬声器170A，也称“喇叭”，用于将音频电信号转换为声音信号。电子设备100可以通过扬声器170A收听音乐，或收听免提通话。
受话器170B，也称“听筒”，用于将音频电信号转换成声音信号。当电子设备100接听电话或语音信息时，可以通过将受话器170B靠近人耳接听语音。
麦克风170C，也称“话筒”，“传声器”，用于将声音信号转换为电信号。
耳机接口170D用于连接有线耳机。耳机接口170D可以是USB接口130，也可以是3.5mm的开放移动终端平台(open mobile terminal platform，OMTP)标准接口，美国蜂窝电信工业协会(cellular telecommunications industry association of the USA，CTIA)标准接口。
压力传感器180A用于感受压力信号，可以将压力信号转换成电信号。在一些实施例中，压力传感器180A可以设置于显示屏194。压力传感器180A的种类很多，如电阻式压力传感器，电感式压力传感器，电容式压力传感器等。电容式压力传感器可以是包括至少两个具有导电材料的平行板。当有力作用于压力传感器180A，电极之间的电容改变。电子设备100根据电容的变化确定压力的强度。当有触摸操作作用于显示屏194，电子设备100根据压力传感器180A检测所述触摸操作强度。电子设备100也可以根据压力传感器180A的检测信号计算触摸的位置。在一些实施例中，作用于相同触摸位置，但不同触摸操作强度的触摸操作，可以对应不同的操作指令。例如：当有触摸操作强度小于第一压力阈值的触摸操作作用于短消息应用图标时，执行查看短消息的指令。当有触摸操作强度大于或等于第一压力阈值的触摸操作作用于短消息应用图标时，执行新建短消息的指令。
陀螺仪传感器180B可以用于确定电子设备100的运动姿态。气压传感器180C用于测量气压。在一些实施例中，电子设备100通过气压传感器180C测得的气压值计算海拔高度，辅助定位和导航。磁传感器180D包括霍尔传感器。加速度传感器180E可检测电子设备100在各个方向上(一般为三轴)加速度的大小。距离传感器180F，用于测量距离。电子设备100可以通过红外或激光测量距离。在一些实施例中，拍摄场景，电子设备100可以利用距离传感器180F测距以实现快速对焦。接近光传感器180G可以包括例如发光二极管(LED)和光检测器，例如光电二极管。环境光传感器180L用于感知环境光亮度。指纹传感器180H用于采集指纹。电子设备100可以利用采集的指纹特性实现指纹解锁，访问应用锁，指纹拍照，指纹接听来电等。温度传感器180J用于检测温度。
触摸传感器180K，也称“触控器件”。触摸传感器180K可以设置于显示屏194，由触摸传感器180K与显示屏194组成触摸屏，也称“触控屏”。触摸传感器180K用于检测作用于其上或附近的触摸操作。触摸传感器可以将检测到的触摸操作传递给应用处理器，以确定触摸事件类型。可以通过显示屏194提供与触摸操作相关的视觉输出。在另一些实施例中，触摸传感器180K也可以设置于电子设备100的表面，与显示屏194所处的位置不同。
骨传导传感器180M可以获取振动信号。按键190包括开机键，音量键等。按键190可以是机械按键。也可以是触摸式按键。电子设备100可以接收按键输入，产生与电子设备100的用户设置以及功能控制有关的键信号输入。马达191可以产生振动提示。指示器192可以是指示灯，可以用于指示充电状态，电量变化，也可以用于指示消息，未接来电，通知等。
SIM卡接口195用于连接SIM卡。SIM卡可以通过插入SIM卡接口195，或从SIM卡接口195拔出，实现和电子设备100的接触和分离。电子设备100可以支持1个或N个SIM卡接口，N为大于1的正整数。SIM卡接口195可以支持Nano SIM卡，Micro SIM卡，SIM卡等。同一个SIM卡接口195可以同时插入多张卡。所述多张卡的类型可以相同，也可以不同。SIM卡接口195也可以兼容不同类型的SIM卡。SIM卡接口195也可以兼容外部存储卡。电子设备100通过SIM卡和网络交互，实现通话以及数据通信等功能。在一些实施例中，电子设备100采用eSIM，即：嵌入式SIM卡。eSIM卡可以嵌在电子设备100中，不能和电子设备100分离。
电子设备100的软件系统可以采用分层架构，事件驱动架构，微核架构，微服务架构，或云架构。本发明实施例以分层架构的Android系统为例，示例性说明电子设备100的软件结构。
图1B是本发明实施例图1A所示电子设备100的软件结构框图。
分层架构将软件分成若干个层，每一层都有清晰的角色和分工。层与层之间通过软件接口通信。在一些实施例中，将Android系统分为四层，从上至下分别为应用程序层，应用程序框架层，安卓运行时(Android runtime)和系统库，以及内核层。
应用程序层可以包括一系列应用程序包。
如图1B所示，应用程序包可以包括相机，图库，日历，通话，地图，导航，WLAN，蓝牙，音乐，视频，短信息等应用程序。
应用程序框架层为应用程序层的应用程序提供应用编程接口(application programming interface，API)和编程框架。应用程序框架层包括一些预先定义的函数。
如图1B所示，应用程序框架层可以包括窗口管理器，内容提供器，视图系统，电话管理器，资源管理器，通知管理器等。
窗口管理器用于管理窗口程序。窗口管理器可以获取显示屏大小，判断是否有状态栏，锁定屏幕，截取屏幕等。
内容提供器用来存放和获取数据，并使这些数据可以被应用程序访问。所述数据可以包括视频，图像，音频，拨打和接听的电话，浏览历史和书签，电话簿等。
视图系统包括可视控件，例如显示文字的控件，显示图片的控件等。视图系统可用于构建应用程序。显示界面可以由一个或多个视图组成的。例如，包括短信通知图标的显示界面，可以包括显示文字的视图以及显示图片的视图。
电话管理器用于提供电子设备100的通信功能。例如通话状态的管理(包括接通，挂断等)。
资源管理器为应用程序提供各种资源，比如本地化字符串，图标，图片，布局文件，视频文件等等。
通知管理器使应用程序可以在状态栏中显示通知信息，可以用于传达告知类型的消息，可以短暂停留后自动消失，无需用户交互。比如通知管理器被用于告知下载完成，消息提醒等。通知管理器还可以是以图表或者滚动条文本形式出现在系统顶部状态栏的通知，例如后台运行的应用程序的通知，还可以是以对话窗口形式出现在屏幕上的通知。例如在状态栏提示文本信息，发出提示音，电子设备振动，指示灯闪烁等。
安卓运行时(即Android Runtime)包括核心库和虚拟机。Android runtime负责安卓系统的调度和管理。
核心库包含两部分：一部分是java语言需要调用的功能函数，另一部分是安卓的核心库。
应用程序层和应用程序框架层运行在虚拟机中。虚拟机将应用程序层和应用程序框架层的java文件执行为二进制文件。虚拟机用于执行对象生命周期的管理，堆栈管理，线程管理，安全和异常的管理，以及垃圾回收等功能。
系统库可以包括多个功能模块。例如：表面管理器(surface manager)，媒体库(Media Libraries)，三维图形处理库(例如：OpenGL ES)，二维图形引擎(例如：SGL)等。
表面管理器用于对显示子系统进行管理，并且为多个应用程序提供了2D和3D图层的融合。
媒体库支持多种常用的音频，视频格式回放和录制，以及静态图像文件等。媒体库可以支持多种音视频编码格式，例如:MPEG4，H.264，MP3，AAC，AMR，JPG，PNG等。
三维图形处理库用于实现三维图形绘图，图像渲染，合成，和图层处理等。
二维图形引擎是2D绘图的绘图引擎。
内核层是硬件和软件之间的层。内核层至少包含显示驱动，摄像头驱动，音频驱动，传感器驱动。
下面结合捕获拍照场景，示例性说明电子设备100软件以及硬件的工作流程。
当图1A中的触摸传感器180K接收到触摸操作，相应的硬件中断被发给内核层。内核层将触摸操作加工成原始输入事件(包括触摸坐标，触摸操作的时间戳等信息)。原始输入事件被存储在内核层。应用程序框架层从内核层获取原始输入事件，识别该输入事件所对应的控件。以该触摸操作是触摸单击操作，该单击操作所对应的控件为相机应用图标的控件为例，相机应用调用应用框架层的接口，启动相机应用，进而通过调用内核层启动摄像头驱动，通过图1A所示的摄像头193捕获静态图像或视频。
在一些实施例中，上述提到的显示可以通过显示屏显示，显示屏具有显示的 功能，显示屏可以具有触摸功能，也可以不具有触摸功能。在触摸显示屏上的操作可以通过虚拟按键实现，可以点击触摸屏实现。在非触摸显示屏上的操作可以通过物理按键实现。
图2示出了本申请实施例提供的网络架构示意图200。在一实施例中，上述电子设备100，例如智能手机和智能电视等的智能终端，可以用于结合所述网络架构200工作。如图所示，至少提供一个网络202。在本网络架构200上下文中，所述网络202可以采用任何形式，包括但不限于电信网络、局域网(local area network，简称LAN)、无线网络以及因特网、对等网络、有线网络等广域网(wide area network，简称WAN)。虽然仅示出了一个网络，但应理解的是，可以提供两个或更多类似或不同的网络202。
多个设备与所述网络202连接。例如，服务器计算机212和终端用户计算机208可以连接到所述网络202用于通信，此类终端用户计算机208可以包括台式计算机、膝上型计算机和/或任何其他类型的逻辑。此外，各种其他设备也可以连接到所述网络202，包括个人数字助理(personal digital assistant，简称PDA)设备210、移动电话设备206、电视机204等。又例如，移动电话设备206(例如智能手机)和电视机204(例如智能电视)可以连接到所述网络202(例如局域网)进行通信以共享内容；智能手机206或智能电视204和服务器212还可以连接到所述网络202(例如广域网)进行通信以供用户在终端设备上访问服务器数据，例如访问或下载视频流等。
本申请下列各实施例提供了一种投屏方法，以使智能终端之间，例如智能手机和智能电视之间，在互联互通时能够更加方便快捷的推送并显示例如是流媒体数据，例如在智能手机向智能电视投屏时，在智能电视的大屏端能同时显示视频，以及弹幕信息或外挂字幕信息中的至少一种。
图3示出了本申请提供的一种投屏方法的场景示意图，例如流数据的投屏方法。如图3a所示，第一电子设备，例如第一智能终端310，正在第一应用程序上观看视频。其中，第一智能终端310可以是手机、平板电脑、智能穿戴、智能家居设备等等，在本申请的一个优选实施例中，第一智能终端310是用户使用频率相对较高而屏幕相对较小的设备。第一应用程序可以是任何视频网站客户端，例如爱奇艺、优酷、腾讯等，也可以是支持播放任何视频网站中的视频源的应用程序，例如微信、微博等。参见图3a，当用户在智能手机或平板电脑等第一智能终端310上观看视频时，在第一智能终端310的第一应用程序中，除了显示视频画面311，还可以显示操作界面，图3a中显示了与该视频内容相关的部分播控界面，包括但不限于投屏键302、弹幕键304、弹幕设置键306、以及弹幕发送框308。其中，操作界面可以在视频画面311上显示，也可以与视频画面311显示于不同的显示区域中，本申请不对此做任何限制。
在一个实施例中，用户在第一应用程序上选择视频播放后，随视频画面默认可以显示弹幕或弹幕信息313，或者外挂字幕信息(图中未显示)。在另一个实施例中，用户播放视频后，点击弹幕键304以显示弹幕313，和/或者通过菜单 界面(图中未显示)来选择显示外挂字幕。当第一智能终端310的第一应用程序上播放弹幕313的动画以及视频画面时，用户可以通过弹幕设置键306和弹幕发送框308，分别设置弹幕的显示参数和发送新的弹幕信息。
当用户点击第一应用程序上的投屏键302时，第一智能终端310检测到该投屏操作，并开始搜寻可投屏设备。搜寻的设备结果可以是列表的形式呈现于智能终端310上，以供用户选择，并基于用户的选择，智能终端310向被选设备发送投屏请求以进行投屏。在其他情况下，基于用户的选择，智能终端310向被选设备通信以发送投屏请求。
在本申请的一个实施例中，响应于用户点击投屏键302，或者响应于用户选择投屏设备，智能终端310还可以显示如图3b或3c所示的菜单栏或弹窗。例如，当智能终端310上的第一应用程序播放视频并叠加显示弹幕313时，用户点击投屏键302并选择投屏的目标设备，此时智能终端310上显示如3b所示的菜单栏或弹窗，以供用户选择301“视频投屏”或者选择303“视频和弹幕投屏”，从而根据用户的实际需求进行视频投屏或者视频与弹幕一起投屏。又例如，当智能终端310上的第一应用程序播放的视频已经投屏至目标设备时，用户在智能终端310上点击弹幕键304，此时智能终端310上显示如3c所示的菜单栏或弹窗305，以供用户选择是否将该弹幕投屏，若用户选择307“是”，则目标设备将在视频上叠加显示弹幕，若用户选择309“否”，则目标设备显示的视频上不会叠加显示弹幕，而在智能终端310上可以显示弹幕313。
请参见图3d，在图3d所示的实施例中，用户选择第二电子设备，例如第二智能终端320，进行投屏，第二智能终端320可以具有比第一智能终端310更大的显示屏。在本申请的一个实施例中，第二智能终端320显示投屏内容之前，第一智能终端310判断当前的视频播放内容中是否显示弹幕和/或外挂字幕,如果判断的结果为是，则在给第二智能终端320推送视频时，同时推送弹幕和/或外挂字幕。如图3d所示，开始投屏时，第二智能终端320的显示屏上同时显示视频画面321和弹幕323。在本申请的另一实施例中，第二智能终端320已经接收到第一智能终端310的投屏请求并显示视频画面321后，第一智能终端310的第一应用程序接收到用户点击弹幕键304的操作后，该视屏画面321的弹幕信息将被投屏到第二智能终端320的显示屏上，即显示弹幕或弹幕信息323。
在本申请的一个实施例中，在第一智能终端310上的视频画面311，以及弹幕313和/或外挂字幕，被投屏到第二智能终端320上后，用户依然可以通过第一智能终端310上的播放控件对第二智能终端320上显示的视频画面321进行操控，包括但不限于切换视频、快进、快退、音量控制等，用户还可以通过第一智能终端310对第二智能终端320上显示的弹幕323进行操控，包括但不限于调整弹幕参数、发送新弹幕等等。此外，用户也可以直接在第二智能终端320上对视频画面321和弹幕323进行操控，包括但不限于上述操控的操作。
视频投屏后，由于第二智能终端320上可以同时显示弹幕和/或外挂字幕，可以继续为用户提供一边观看视频一边进行弹幕互动的场景，和/或在观看视频 的同时查看相应字幕的场景。因此，本申请为用户提供了一种方便快捷的弹幕和/或字幕投屏显示的方案，满足了用户观看视频同时的娱乐性，可以大大提高了用户的投屏体验。
本申请中的第一智能终端310可以是便携智能设备，第二智能终端320可以是智能电视，当便携智能设备310上显示的视频和弹幕如3d所示投屏到智能电视320上之后，便携智能设备310还可以如图3e所示退出该第一应用程序或退出该视频显示画面，以进行其它操作。因此，本申请的实施例中的第一智能终端310实现了将视频和弹幕投屏，且可自由灵活选择投屏方案。
本申请中提到的“弹幕”指的是视频网站客户端所提供的，可以与视频同时显示的关于视频的评论性内容，通常叠加显示于视频画面的上方，任何观看该视频的用户都可以添加自己的弹幕信息以进行互动，该添加的弹幕信息会显示于弹幕显示区域。本申请中提到的通过弹幕设置键进行设置，可以调整弹幕的“显示参数”，该显示参数可以包括但不限于以下弹幕参数：字体、字号、颜色、透明度、动画速度、显示区域、屏蔽用户ID(账户名)等等。本申请中提到的“外挂字幕”是区别于视频的内嵌字幕的一种字幕，外挂字幕的数据与视频数据分离，而没有集成在一起，需要单独导入以显示在视频画面中，通常叠加显示于视频画面的下方。
在本申请的实施例中，第一智能终端310进行“投屏”是指将第一智能终端310的显示屏上显示的内容的至少一部分，以及与该显示内容相关的数据的至少一部分，包括但不限于视频、音频、文本等，投射到另一个电子设备上，例如第二智能终端320。当一个智能终端需要投屏时，其搜寻的“可投屏设备”可以是在同一局域网(包括二层局域网，连接到同一路由器上)内支持投屏的电子设备，也可以是不在同一局域网内，但是与第一智能终端310共享电子账号并通过其他网络(例如有线或无线网络)互联互通的电子设备。
在本申请的一个具体实施例中，第一智能终端搜寻到的可投屏设备是同一局域网内支持同一投屏协议的设备，其中，投屏协议可以是无线投屏协议，例如Airplay、DLNA等技术或协议。
图4示出了智能终端的模块架构示意图，其中图4a还示出了第一智能终端401、第二智能终端402、以及第一应用程序的视频服务器410的架构和通信连接的示意图。
请参见图4a，第一智能终端401包括通信模块411、数据判断单元412、视频解码单元413、文本解析单元414、显示模块415、视频控制单元416、和文本控制单元417，其中显示模块415还可以包括视频显示单元418和文本显示单元419。为了方便说明本申请的实施例，图4a中仅示出第一智能终端401中的与视频播放和文本显示相关的模块，应可理解，第一智能终端401还可以包括更多的功能模块，如图1A所示，此处不再赘述。本申请实施例中的第一智能终端401可以是智能手机，通过第一应用程序，与视频服务器410进行第一通信431，从而获取视频服务器410中的视频数据和文本数据，并显示于第一智能终端401 的显示模块415。
第二智能终端402包括通信模块421、数据判断单元422、视频解码单元423、文本解析单元424、显示模块425，其中显示模块425还可以包括视频显示单元428和文本显示单元429。本申请实施例中的第二智能终端402可以是智能电视，通过投屏协议与第一智能终端401，如智能手机，进行互通。在本申请一个实施例中，智能电视与智能手机之间进行第二通信432。通过第一智能终端401提供的数据，第二智能终端402进而与视频服务器410进行第三通信433，来获取视频服务器410中的视频数据和文本数据，并显示于第二智能终端402的显示模块425，以进行投屏。应可理解，第一通信431、第二通信432、第三通信433可以是任何形式的通信，例如有线或无线的通信方法等，本申请不对此进行限制。例如第一智能终端401和第二智能终端402在一个局域网中，连接到一个路由器上，上述第一通信431、第二通信432、第三通信433可以经由路由器实现。在本申请的一个实施例中，第二智能终端402还可以包括视频控制单元426和文本控制单元427(图中均以虚线框表示)，以控制被投屏的视频的播放和相应的文本显示。为了方便说明本申请的实施例，图4中仅示出第二智能终端402中的与视频播放、文本显示相关的模块，应可理解，第二智能终端402还可以包括更多的功能模块，如图1A所示，此处不再赘述。
请参见图4b，文本解析单元414/424可以包含弹幕解析单元4141和/或字幕解析单元4142；文本显示单元419/429可以包含弹幕显示单元4191和/或字幕显示单元4192；文本控制单元417/427可以包含弹幕控制单元4171和/或字幕控制单元4172，其中，文本控制单元417/427还可以包含弹幕发送单元4173。
第一和第二智能终端401、402的通信模块411、421用于分别与视频服务器410进行第一通信431和第三通信433，从而相互连通以请求和下发流媒体及相关数据。
在本申请的一个实施例中，通过第一通信431，第一智能终端401从视频服务器410接收视频数据和文本数据；通过第二通信432，第二智能终端402从第一智能终端401接收视频下载地址和文本下载地址的数据；通过第三通信433，第二智能终端402从视频服务器410接收视频数据和文本数据。
第一和第二智能终端401、402的数据判断单元412、422用于判断从视频服务器410接收到的数据的类型，并根据判断的结果将不同的数据传送给不同的单元，例如将视频数据传送给视频解码单元413、423，将文本数据传送给文本解析单元414、424。举例说明，当数据判断单元412、422判断接收的是弹幕数据时，该弹幕数据将被传送给图4b所示的弹幕解析单元4141；当数据判断单元412、422判断接收的是字幕数据时，该字幕数据将传送给图4b所示的字幕解析单元4142。
视频解码单元413、423将收到的视频数据解码并传送给视频显示单元418、428以显示于智能终端的显示屏上。文本解析单元414、424将收到的文本数据解析并传送给文本显示单元419、429以叠加显示于显示屏上播放的视频画面上。 举例说明，弹幕解析单元4141将弹幕数据解析后得到弹幕信息，并发送给弹幕显示单元4191，弹幕显示单元4191对解析的弹幕信息进行渲染后显示于智能终端的显示屏上，并通常地叠加到视频画面的顶部区域显示；字幕解析单元4142将字幕数据解析后，发送给字幕显示单元4192，字幕显示单元4192对解析的字幕数据进行渲染后显示于智能终端的显示屏上，并通常地叠加到视频画面的底部区域显示。
视频控制单元416、426用于控制在显示屏上显示的视频，包括但不限于控制视频的暂停、播放、快进、拖放、切换等。文本控制单元417、427用于控制在显示屏上显示的文本。举例说明，弹幕控制单元4171用于控制正在播放的视频上的弹幕信息，包括但不限于控制弹幕信息的播放速度、文本字体、颜色、大小、显示区域等等；字幕控制单元4172用于控制正在播放的视频的字幕数据，包括但不限于控制字幕的字体、颜色、大小、显示区域、切换字幕等等。文本控制单元417、427还可以包括弹幕发送单元4173，用于在智能终端上发送弹幕信息，弹幕发送单元4173接收到用户输入的弹幕信息(例如用户在图3a所示的弹幕发送框308内输入的信息)，将该弹幕信息发送至视频服务器410，以在视频服务器端更新该视频对应的弹幕数据包，并以流媒体的形式下发到对应的客户端。在本申请的一个实施例中，弹幕发送单元4173将该弹幕信息发送至另一智能终端并显示在其显示屏上，和/或发送至本地的弹幕显示单元419、429以显示该弹幕，从而可以实时的在显示屏上显示用户新发送的新弹幕，为用户提供更好的弹幕发送体验。
本申请实施例虽仅以视频投屏为例，详细描述了智能终端的模块架构以及弹幕等文本信息的投屏方案，应可理解，在本申请的其他实施例中，可以对其他数据进行投屏，例如音频数据。当投屏的音频数据上有例如弹幕和/或歌词等文本信息时，可以根据上述实施例同时投屏该弹幕和/或歌词信息，应可理解，在此实施例中，投屏的音频数据将在智能终端本地的音频解码单元进行解析，并在音频输出单元播放。在音频或其他数据的投屏过程中，同时投屏弹幕和/或歌词等文本信息的投屏方案应和上述实施例类似，因此此处不再赘述。
下面将以弹幕信息为例，结合图5到图8示出的时序图，举例说明本申请实施例在投屏时、投屏中、以及结束投屏时，对弹幕信息的显示以及控制过程。应可理解，对字幕信息的显示以及控制过程是类似的。
对图5至图8的描述，将结合图4所示的架构图以及图3所示场景图来进行。
下文将以弹幕显示和弹幕发送为例，说明本申请的智能终端与视频服务器之间的数据请求与下发。
图5到图8为本申请实施例提供的投屏方法的消息序列图，例如流数据的投屏方法。图5示出源设备和投屏设备之间进行投屏的时序图500。
在图5所示的实施例中，源设备是终端设备501，投屏设备为终端设备502。在511中，终端设备501上打开应用程序1并在选中视频后，在513中向视频服务器510请求视频数据，响应于该请求，在514中视频服务器510向终端设备 501下发视频数据例如视频流数据，因此在515中，终端设备501在如图4所述的，通过数据判断单元412判断收到的数据为视频数据时，将该视频数据传输给视频解码单元413进行解码，视频数据解码后，在终端设备501的视频显示单元418上显示视频，即在显示屏上显示视频画面。在一个具体的实施例中，视频服务器510响应于终端设备501获取视频的请求，给终端设备501下发授权的视频下载地址，以使终端设备501下载视频。其中，视频下载地址可以是URL(Uniform Resource Locator,统一资源定位符)地址。
当终端设备501检测到打开弹幕的操作时(例如点击图3所示的弹幕按键304)，在517中，向视频服务器510请求弹幕数据，响应于该请求，在518中，视频服务器510向终端设备501下发弹幕数据，因此在519中，终端设备501如图4所述的，通过判断单元412判断收到的数据为弹幕数据时，将该弹幕数据传输给文本解析单元414进行解析，弹幕数据解析后，在终端设备501的文本显示单元419上显示弹幕信息，即在显示屏上叠加到视频画面中显示弹幕信息，如图3a所示通常显示于视频画面的顶部。图中未示出字幕的显示，当收到的文本数据为字幕数据时，解析后得到的字幕信息将叠加到视频画面中，并通常在底部显示。在一个具体的实施例中，视频服务器510响应于终端设备501获取弹幕的请求，给终端设备501下发授权的弹幕下载地址(例如，URL)，以使终端设备501下载弹幕数据。在本申请的实施例中，弹幕可以是动画播放于显示屏上，其可以从终端设备501显示屏的右侧向左侧滚入或滑入，也可以是以其它动画效果向显示屏的其它方向移动显示。应可理解，弹幕的动画效果不应作为对本申请的限制。
当源设备501检测到点击投屏按钮302的操作，在521中，源设备501选中投屏设备502发送投屏请求以进行投屏。在525中，源设备501判断应用程序1中是否正在显示弹幕信息，如果判断的结果为是，则在526中向视频服务器510请求授权的流数据下载地址时，同时请求视频数据下载地址以及弹幕数据下载地址。响应于源设备501的请求，在526中，视频服务器510向源设备501下发授权的视频及弹幕数据下载地址。在528中，源设备501将收到的授权下载地址发送给投屏设备502。在本申请的另一实施例中，可以跳过步骤526和527。在此实施例中，在源设备501向视频服务器510请求视频和弹幕(即，步骤513和517)后，视频服务器510下发授权的视频数据下载地址和授权的弹幕数据下载地址，因此，源设备501可以不经过步骤526和527，而直接执行步骤528。具体地，在终端设备501向终端设备502发送的投屏请求中可以增加该授权的视频数据以及弹幕数据下载地址。特别地，在一个实施例中，用户在终端设备501上观看爱奇艺的视频时，进行投屏的操作时，终端设备501判断同时显示了字幕和外挂弹幕，则终端设备501向终端设备502发送的投屏请求中增加爱奇艺授权的该视频下载地址，以及弹幕和外挂字幕的下载地址，
例如：
视频下载地址，"videoSource":"http://video.iqiyi.com/source？p＝9088&tk＝skdhfsdf"
弹幕下载地址，"bulletComments":"http://comment.iqiyi.com/c？id＝skdhfsdf"
字幕下载地址，"subtitle":"http://sub.iqiyi.com/source？id＝89898"
以DLNA协议为例，终端设备501，例如智能手机在DLNA中作为Control Point，终端设备502，例如智能电视作为Media Renderer，视频服务器510，例如爱奇艺服务器作为Media Server。在本申请的一个实施例中，智能手机Control Point选择好播放的视频后，向智能电视Media Renderer推送视频源的下载地址的同时，如判断爱奇艺APP(application,应用程序)的当前视频中还叠加显示了相应的弹幕信息和/或外挂字幕信息，则智能手机Control Point向智能电视Media Renderer推送弹幕源的下载地址和/或外挂字幕源下载地址。智能电视Media Renderer根据接收到的视频源、弹幕源、和/或外挂字幕源的下载地址，从爱奇艺服务器Media Server获取视频源、弹幕源、和/或外挂字幕源文件，爱奇艺服务器Media Server向智能电视Media Renderer传输这些文件的方式可以是，例如HTTP/HTTPS。接收到视频源、弹幕源、和/或外挂字幕源文件后，智能电视Media Renderer对这些文件进行渲染，并在显示屏上显示视频画面，以及叠加显示弹幕和/或外挂字幕。其中，智能手机Control Point通过DLNA协议向智能电视Media Renderer推送的信息可以新增以下字段：
dlna-playsingle://uuid:742b972e-3e50-4fa9-bc49-8d1ac91587e5？sid＝urn:upnp-org:serviceId:ContentDirectory
dlna-bulletcomments:http://comment.iqiyi.com/c？id＝skdhfsdf
dlna-subtitle:http://sub.iqiyi.com/source？id＝89898
在531和535中，投屏设备502根据接收到的视频下载地址和弹幕下载地址分别向视频服务器510请求视频数据和弹幕数据，并在532和536中分别向投屏设备502下发视频数据和弹幕数据。因此，在533和537中，投屏设备502如图4所述的，通过数据判断单元422将判断为视频数据的部分传输给视频解码单元423进行解码，并将判断为弹幕数据的部分传输给文本解析单元424进行解析，解析得到的视频和弹幕信息，分别通过视频显示单元428和文本显示单元429在终端设备502的显示屏上加载和显示。应可理解，由于弹幕的显示是为了使用户在一边观看视频时一边有和其他用户进行互动评论的效果，因此，弹幕数据与视频的播放时间有关，即一部分弹幕数据仅仅与视频中的某一段有关联且应该显示于该段视频播放时的画面上。并且，在533中，投屏设备502上显示的视频应是接续用户在源设备上点击投屏按钮后切断的视频画面或切断的视频画面前后画面，同时在537上，投屏设备502上显示的弹幕信息应是与该切断的视频画面或其前后画面有关联的弹幕信息。举例说明，当源设备501在应用程序1上播放视频Video1到03′30″时，用户点击投屏按钮，则投屏设备502上进行投屏播放时，视频应是从03′30″或其前后，例如从03′28″开始的画面，在视频画 面上显示的弹幕应是与03′30″或其前后的视频画面相关联的弹幕信息。
在一个具体地实施例中，投屏设备502接收到的经过解析的弹幕文件的一部分举例如下：
<d p＝"51.593,5,25,16711680,1408852480,0,7fa769b4,576008622">弹幕1</d>
<d p＝"10.286,1,25,16777215,1408852600,0,a3af4d0d,576011065">弹幕2</d>
<d p＝"12.65,1,25,16777215,1408852761,0,24570b5a,576014281">弹幕3</d>
<d p＝"19.033,1,25,16777215,1408852789,0,cb20d1c7,576014847">弹幕4</d>
<d p＝"66.991,1,25,16777215,1408852886,0,a78e484d,576016806">弹幕5</d>
p字段里面的内容，例如上述弹幕文件的第一行：
p＝"51.593,5,25,16711680,1408852480,0,7fa769b4,576008622"
其中，通过逗号分割的多个数据是和这一条弹幕“激动激动好激动”的显示参数相关的数据，该数据为示例性的，可解释如下：
第一个参数“51.593”是在视频中该条弹幕出现的时间，以秒数为单位；
第二个参数“5”是弹幕的动画模式，其中，1到3代表三种不同形式的滚动弹幕，4代表底端弹幕，5代表顶端弹幕，6代表逆向弹幕，7代表精准定位，8代表高级弹幕，等等；
第三个参数“25”弹幕显示的字号大小，其中，12代表非常小，16代表特小，18代表小，25代表中，36代表大，45代表很大，64代表特别大，等等；
第四个参数“16711680”是弹幕显示的字体颜色，本实施例使用HTML颜色的十位数；
第五个参数“1408852480”是Unix格式的时间戳；
第六个参数“0”是弹幕池，其中，0代表普通池，1代表字幕池，2代表特殊池(特殊池为高级弹幕专用)，等等；
第七个参数“7fa769b4”是该条弹幕的发送者的账号ID，可以用于“屏蔽此弹幕的发送者”的功能；
第八个参数“576008622”是该条弹幕在弹幕数据库中代表排列顺序的rowID，可以用于“历史弹幕”的功能。
投屏设备502接收到例如上述的弹幕文件后，在本地的播放器上进行渲染，再叠加显示于显示屏播放的视频画面中。其中，以DLNA为例，在智能电视的播放器中对弹幕进行渲染的相关代码的一部分将在下文中举例示出，应可理解，这些代码仅仅作为示例作用，不应作为对本发明的显示。
自定义ViewGroup-XCDanmuView，继承RelativeLayout来实现，也可以是继承其他布局类，示例性的部分代码如下：
Figure PCTCN2021098499-appb-000001
Figure PCTCN2021098499-appb-000002
初始化弹幕的item View，这里以TextView，ImageView为例，然后通过addView添加到自定义的danmuView中，示例性的部分代码如下：
Figure PCTCN2021098499-appb-000003
Figure PCTCN2021098499-appb-000004
通过addView添加到XCDanmuView中，以实现从屏幕外移动进来的效果，其中弹幕位置是坐标，以从右向左移动方向来说，新产生的danMuView隐藏到layout最右边，addView后将该danMuView的位置设置到右边的屏幕外，示例性的部分代码如下：
Figure PCTCN2021098499-appb-000005
随机从左侧或右侧出来弹幕itemView，移动采用属性动画来实现平移，从屏幕的一端移动到另一端，当动画结束后，就将该view从XCDanmuView中remove掉。并重新创建一个弹幕itemView，并addView到XCDanmuView中，并开始动画移动，示例性的部分代码如下：
Figure PCTCN2021098499-appb-000006
Figure PCTCN2021098499-appb-000007
在本申请的一个实施例中，可选地，在529中，源设备501可以将当前的弹幕显示参数发送到投屏设备502，从而当投屏设备502显示弹幕时相应地匹配其在源设备上显示的弹幕参数，例如按最大字体以蓝色显示四排弹幕信息，等等。
在本申请实施例提供的流式投屏的技术方案中，当源设备501发起投屏请求时，主动判断投屏视频中是否显示弹幕和/或外挂字幕，并基于判断的结果为是，主动向投屏设备502发送弹幕数据和/或外挂字幕数据的授权下载地址，使得投屏设备502从视频服务器510请求流数据，如此，本申请的实施例实现了同时将视频和弹幕(和/或外挂字幕)数据投屏的流式投屏方案。同时，由于投屏设备502的视频数据和弹幕数据都是由视频服务器510下发，二者通过有线或无线通信，能够保证投屏中视频播放以及弹幕显示的流畅性，提高用户的投屏体验。
在源设备501和投屏设备502之间进行投屏后，源设备501可以在539中继续控制该投屏状态，也可以退出当前投屏的视频界面，在应用程序1中浏览其它视频导览，或者退出应用程序1以浏览其它应用的界面，又或者进入息屏状态，而不会对投屏设备502上的视频和弹幕显示造成干扰。
在投屏后，用户还可以通过源设备501上显示的用户交互界面来控制、调整弹幕的显示、以及发送弹幕信息等，例如通过图3a中的弹幕键304来终止或开启弹幕在投屏设备上的显示，通过弹幕设置键306来调整弹幕在投屏设备上的显示参数，或者通过弹幕发送框308来发送弹幕以显示到投屏设备上。
在本申请的一个实施例中，当用户从源设备501或投屏设备502上关闭弹幕，例如点弹幕键304以关闭弹幕显示时，示例性的部分代码如下：
Figure PCTCN2021098499-appb-000008
Figure PCTCN2021098499-appb-000009
而当用户又从源设备501或投屏设备502上重新开启弹幕，例如点弹幕键304以开启弹幕显示时，部分伪代码可以如下：
Figure PCTCN2021098499-appb-000010
图6示出投屏过程中调整弹幕参数的时序图600。在601中，用户通过终端设备501，即源设备上的用户交互画面进行设置，并通过文本控制单元417来调整弹幕的显示参数。当源设备501检测到用户交互界面的弹幕设置键306上的操作时，弹出弹幕设置窗口(图中未示出)，可以包括但不限于设置弹幕的：透明度(0-100％)、字号(小号-标准-大号-特大号)、播放速度(慢速-正常-2x-3x)、显示区域(10-50％)、颜色、动画效果，等等。当源设备501检测到弹幕参数的设置完成时，在603中，源设备501发送更新的弹幕参数到投屏设备502上。在605中，响应于接收到的弹幕参数，投屏设备502的文本控制单元427根据该弹幕参数调整文本显示单元429上显示的弹幕，调整后的弹幕显示对应于用户在源设备501上做的设置或者对应于文本控制单元427对一些参数做相应适配之后的显示设置。
在611中，用户还可以在投屏设备502上设置弹幕的显示参数，例如通过智能电视的遥控器来操控显示屏上的用户交互画面进行设置，并通过文本显示单元429来调整弹幕的显示参数。在本申请的一个实施例中，还包括步骤613，投屏设备502将更新的弹幕显示参数发送给源设备501，以在源设备501上更新用户对弹幕显示参数的调整/变更。
在投屏过程中，用户可以自由的选择是否在源设备或投屏设备上对弹幕信息的显示进行个性化设置，即使在源设备上进行弹幕显示的参数设置，也可以相应的适配于当前在投屏设备上的投屏画面中，使得设置的操作十分便捷、灵活。相应地，用户在投屏设备上进行弹幕信息的显示设置的变更也可以更新到源设备上，使得用户切换回源设备观看视频时即可看到已相应地适配了该变更的弹幕，无需再次进行设置。
图7示出投屏过程中用户通过两个设备发送弹幕的时序图700。在703中，用户在终端设备501上的用户交互界面上通过弹幕发送框308来编辑待发送的弹幕信息，并通过文本控制单元417中的弹幕发送单元来发送该弹幕信息，当终端 设备501检测到用户输入弹幕信息并点击发送时，终端设备501将该弹幕信息封装为弹幕数据并将该弹幕数据发送至终端设备502。响应于接收到该新添加的弹幕数据，终端设备502的文本解析单元424解析该弹幕数据得到新的弹幕信息，在707中，该解析的弹幕信息通过文本显示单元429加载/显示于当前显示的视频画面中，该新添加的弹幕信息可以是以滑入的动画效果从显示屏的右上侧滑入显示屏。
在本申请的一个实施例中，在701中，终端设备501还可以将该弹幕数据发送至视频服务器510，且在705中，视频服务器510响应于该收到的弹幕数据，更新与其对应的弹幕数据包，并在706中，将更新的弹幕数据包发送给终端设备502，从而使得终端设备502在步骤707中显示该新添加的弹幕。
在本申请的另一实施例中，若终端设备501在投屏后仍然在其显示屏上显示弹幕信息时，用户在其用户交互界面上新发送的弹幕信息也会以滑入的动画效果显示于终端设备501的显示屏上。
继续参考图7，在711中，用户还可以在终端设备502上的用户交互界面编辑弹幕信息，并通过文本控制单元427中的弹幕发送单元来发送该弹幕信息，具体操作可以例如通过智能电视的遥控器在显示屏上的用户交互界面中输入文本信息并点击发送，当终端设备502检测用户点击发送该弹幕信息时，将其封装为弹幕数据并发送至视频服务器510。在715中，响应于该收到的弹幕数据，视频服务器510更新与其对应的弹幕数据包，并在716中，将更新的弹幕数据包发送给终端设备502。在717中，终端设备502通过文本控制单元427来控制文本显示单元429，从而使该新的弹幕信息以滑入动画效果显示于终端设备502的显示屏上。
在本申请的一个实施例中，步骤715和716可以省去，用户在终端设备502上发送的新弹幕，会直接显示于终端设备502的显示屏上，而不是在收到视频服务器510更新后的弹幕数据后才显现。
在本申请的另一实施例中，若终端设备501在投屏后仍然在其显示屏上显示弹幕信息时，终端设备502也可以在712中，将包含该新弹幕信息的弹幕数据发送给终端设备501，终端设备501通过文本解析单元414解析该弹幕数据后，在714中，通过文本显示单元417将用户新发送的弹幕信息以滑入的动画效果显示于终端设备501的显示屏上。
其中，关于用户点击发送弹幕，以及新弹幕在源设备501，和/或投屏设备502上以动画效果滑入，示例性的部分代码如下：
Figure PCTCN2021098499-appb-000011
Figure PCTCN2021098499-appb-000012
由于用户在观看视频时，通过弹幕显示以及弹幕发送能够有类似实时互动的体验，而在视频投屏的过程中，用户依然可以从源设备端或投屏设备端发送新的弹幕，并在观看视频时查看到刚刚发送的新弹幕，继续地参与“实时互动”。通常投屏设备具有比源设备更大的显示屏幕，使得用户可以在大屏上观看视频并且“实时互动”，增强了互动时的体验。
图8示出两个设备之间的投屏终止时，视频画面和弹幕信息继续在源设备501的显示屏上显示的时序图800。在802中，终端设备501检测到用户终止投屏的操作，例如点击如图3所示的投屏按键302以结束投屏，终端设备501发送终止请求至终端设备502，响应于该终止请求，在803中，终端设备502通过视频控制单元426和文本控制单元427控制显示模块425以停止显示当前的视频画面以及弹幕信息，并在804中，终端设备502与视频服务器510之间终止数据的收发。在810中，终端设备501向视频服务器510请求当前中断的视频数据和弹幕数据，响应于该请求，在812中，视频服务器510向终端设备501下发当前中断的视频画面或其前后画面开始的视频数据和弹幕数据，并在814中，在终端设备501上通过视频解码单元413和文本解析单元414传输解码后的视频和弹幕， 并在显示模块415上显示。
在本申请的一个实施例中，如图6所示，在611中，用户通过投屏设备502来设置弹幕的显示参数，由于此时源设备501上没有显示任何视频画面和/或弹幕信息，因此，在投屏设备502上进行的参数变更没有进一步发送到源设备501上。在这个实施例中，当用户终止投屏时，如图8所示，在805中，投屏设备502判断在投屏过程中是否改变了弹幕显示参数，如果判断结果为是，则在807中，向源设备501发送当前更新的弹幕显示参数。基于从投屏设备502收到的弹幕显示参数，源设备501在显示屏上显示弹幕时将对应于该参数并做相应的适配。
在本申请另一实施例中，用户在投屏设备上对当前观看的视频的弹幕进行显示设置后，可以不执行图6所示的613，即该弹幕显示设置的变更可以不更新到源设备501上；并且，当用户终止投屏时，也可以不执行图8所示的805和807，即该弹幕显示设置的变更依然不更新到源设备501上。在这个实施例中，用户可以在源设备501上保留一组对弹幕显示的设置ConfigA，而在投屏设备502上保留另一组对弹幕显示的设置ConfigB，且用户于任何时候在源设备501或投屏设备502上更改弹幕的显示设置时，只会分别更改ConfigA或ConfigB中的参数，而不会影响另一个参数，从而满足用户在不同的设备上对弹幕显示参数的不同需求。
图9为本申请实施例提供的一种投屏的方法流程图900，例如流数据的投屏方法。图9所示为终端设备501，即源设备上执行的流数据投屏的方法流程图900。
在步骤901中，源设备501在第一应用程序，如爱奇艺，优酷等APP上播放视频。源设备501上播放的视频是接收自爱奇艺或优酷等的服务器所下发的视频流。服务器将源设备501请求播放的视频流下载地址发送给源设备501。基于该下载地址，源设备501可以下载相应的视频流数据，并在本地通过视频解码单元413进行解码后播放，播放的视频画面通过显示模块415中的视频显示单元418，显示于源设备501的显示屏上。
在步骤903中，源设备501检测用户在第一应用程序界面上的投屏操作，例如，当源设备501检测到用户点击视频画面上的投屏键302，则弹出在当前环境下检测到的可投屏设备列表的选项卡。其中，当前环境指的是例如当前源设备501所在的局域网。在本申请另一实施例中，源设备501可以检测出当前系统账户登录的其他智能设备，并将这些设备作为可投屏设备。在步骤903中，若源设备501未检测到该投屏操作，则继续播放当前的视频。
在本申请的一个实施例中，第一应用程序可以响应于用户上次退出该应用前已勾选了启动弹幕，而在此次播放视频时默认弹幕是开启的。则在此实施例中，源设备501当前显示的视频画面中还可以叠加显示与该视频相关的弹幕信息。
在本申请的另一个实施例中，第一应用程序的服务器可能提供与视频相关的外挂字幕，并且响应于用户选择播放该视频时，自动下发该外挂字幕的数据。则在此实施例中，源设备501收到当前显示的视频画面中还可以叠加显示与该视频相关的外挂字幕信息。
以下将以弹幕为例，进一步的描述本实施例提供的流数据投屏方法。
在步骤905中，源设备501检测到用户选择终端设备502进行投屏，终端设备502可以是和源设备处于同一局域网并支持相同投屏协议的设备，例如智能电视。
在步骤907中，源设备501在该第一应用程序下检测当前播放的视频中是否显示了弹幕，即检测当前是否开启弹幕的功能。当检测到当前开启弹幕功能，则执行步骤909。
在步骤909中，源设备501向投屏设备502发送经第一应用程序的视频服务器授权的弹幕数据的下载地址和视频数据的下载地址，例如弹幕下载URL和视频下载URL。
投屏设备502接收到源设备501发送的数据下载地址后，通过访问地址与视频服务器获取视频流和弹幕数据。投屏设备502的视频解码单元423和文本解析单元424分别解码和解析了获取的视频流和弹幕数据，并通过显示模块425的视频显示单元428和文本显示单元429分别显示解码和解析得到的视频和弹幕信息，其中弹幕信息载入并叠加到视频画面上，并通知显示于视频画面的顶部。
图10为本申请实施例提供的一种投屏的方法流程图1000，例如流数据的投屏方法。图10所示为终端设备502，即投屏设备上执行的流数据投屏的方法流程图1000。
在步骤1001中，投屏设备502接收到来自终端设备501，即源设备的投屏请求。
在步骤1003中，投屏设备502从源设备501接收到视频数据的下载地址和弹幕数据的下载地址。
具体地，通过投屏设备502上的第一判断单元分别判断出属于视频数据下载地址的部分和属于弹幕数据下载地址的部分，以分别请求视频流数据和弹幕数据。
其中，视频数据和弹幕数据的下载地址是第一应用程序的视频服务器510下发给源设备501的经授权的下载地址，该下载地址可以是在源设备501向视频服务器510请求视频时由视频服务器510下发给源设备501的，也可以是当源设备501准备向投屏设备502投屏时，向视频服务器请求的下载地址，即，在源设备501检测到用户点击投屏按键302后，在向投屏设备502发出投屏邀请的同时向视频服务器510发出请求，以请求下发授权的视频数据下载地址、和弹幕数据下载地址。
在步骤1005中，响应于收到的视频数据下载地址和弹幕数据下载地址，投屏设备502访问该地址以请求视频流数据和弹幕数据。
具体地，视频服务器510判断投屏设备502为授权设备后，则向投屏设备502下发所请求的视频流和弹幕数据包。
在步骤1007中，投屏设备502接收到视频服务器下发的视频流和弹幕数据包。
具体地，投屏设备502上的数据判断单元判断接收到的数据类型，并分别将 视频数据发给视频解码单元，将弹幕数据发给文本解析单元，以分别进行解析。
在步骤1009中，投屏设备502的显示屏播放视频，同时将弹幕叠加在视频画面上显示。
具体地，投屏设备502的视频解码单元423将接收到的视频流进行解码并传输给显示单元425的视频显示单元428，以将解码得到的视频画面显示于投屏设备502的显示屏上。同时，投屏设备502的文本解析单元424将接收到的弹幕数据进行解析并传输给显示单元425的文本显示单元429，以将解析得到的弹幕信息叠加到视频画面中，并显示于投屏设备502的显示屏上。
应可理解，虽然上述实施例以弹幕投屏、弹幕投屏后的显示和弹幕投屏后的控制为例进行描述，本申请的实施例也可以实现字幕投屏、以及字幕投屏后的显示与控制，应可理解，字幕的下载、解析、和显示的过程与弹幕相似，字幕的设置/参数调节也与弹幕相似，此处不再赘述字幕数据的投屏过程，因此在本申请中不再对字幕的投屏方案进行详细的描述。
通过本申请提供的流数据投屏方法，智能设备能够根据用户的投屏需求，在向可投屏设备进行视频投屏的同时，还将与该视频相关的弹幕和/或字幕等文本信息进行投屏，并且可选择地在源设备和投屏设备上通过用户交互界面来设置和控制文本数据的显示和更新。这些文本数据的投屏显示和更新操作有利于增强用户在观看视频时的整体感受、以及观看视频时的互动体验。与以往的投屏方案比较而言，本发明实施例提供了非常方便快捷的对与视频匹配的弹幕和字幕进行投屏的方案，还提供了灵活的弹幕发送方案、以及灵活调整弹幕/字幕的显示参数的方案等等，从而大大地提高了用户观看视频时的体验和互动感受。
本申请提供一种包含指令的计算机程序产品，当所述计算机程序产品在终端(例如上述终端设备501或502)上运行时，使得所述终端执行本申请实施例提供的一种流数据投屏方法中的步骤。
本申请提供一种计算机可读存储介质，包括指令，当所述指令在终端上运行时，使得所述终端执行本申请实施例提供的一种流数据投屏方法中的步骤。
所属领域的技术人员可以清楚地了解到本申请实施例可以用硬件实现，或硬件与软件的方式实现。当使用硬件与软件实现，可以将上述功能存储在计算机可读介质中。基于这样的理解，本申请实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)或处理器执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：快闪存储器、移动硬盘、只读存储器、随机存取存储器、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本申请实施例的具体实施方式，但本申请实施例的保护范围并不局限于此，任何在本申请实施例揭露的技术范围内的变化或替换，都应涵盖在本申请实施例的保护范围之内。因此，本申请实施例的保护范围应以所述权利 要求的保护范围为准。

Claims (18)

  1. 一种投屏方法，用于具有显示屏和扬声器的源设备，其特征在于，所述方法包括：
    从服务器接收第一数据并解码，并通过所述显示屏显示已解码的所述第一数据，和/或通过所述扬声器输出已解码的所述第一数据；
    从所述服务器接收文本信息并解析，并通过所述显示屏显示已解析的所述文本信息，其中，所述文本信息与所述第一数据相关联；
    检测到对目标设备进行投屏的第一操作；以及
    响应于所述第一操作，向所述目标设备发送所述第一数据的下载地址以及所述文本信息的下载地址。
  2. 如权利要求1所述的投屏方法，其特征在于，所述第一数据是视频流，所述文本信息为弹幕或字幕，其中，通过所述显示屏显示已解析的所述文本信息包括：
    在显示屏上显示的所述视频流上叠加显示所述弹幕或字幕，其中，叠加显示的所述弹幕或字幕与所述视频流在时间上相关联。
  3. 如权利要求1所述的投屏方法，其特征在于，所述方法还包括：
    向所述目标设备发送所述文本信息的显示参数，其中所述显示参数包括所述文本信息显示的字体、字号、颜色、透明度、动画速度、显示区域中的至少一个。
  4. 如权利要求1所述的投屏方法，其特征在于，所述方法还包括：
    向所述目标设备发送第一文本信息，所述第一文本信息与所述第一数据相关联。
  5. 如权利要求1所述的投屏方法，其特征在于，所述方法还包括：
    检测到对目标设备终止投屏的第二操作；以及
    从目标设备接收所述文本信息的显示参数，其中所述显示参数包括所述文本信息显示的字体、字号、颜色、透明度、动画速度、显示区域中的至少一个。
  6. 一种投屏方法，用于具有第一显示屏和第一扬声器的源设备，其特征在于，所述方法包括：
    从服务器接收第一数据并解码，并通过所述第一显示屏显示已解码的所述第一数据，和/或通过所述第一扬声器输出已解码的所述第一数据；
    响应于对所述第一数据进行投屏的第一操作，向目标设备发送所述第一数据的下载地址；以及
    响应于对文本信息进行投屏的第二操作，向所述目标设备发送所述文本信息 的下载地址，其中，所述文本信息与所述第一数据相关联。
  7. 如权利要求6所述的投屏方法，其特征在于，所述第一数据是视频流，所述文本信息为弹幕或字幕，所述方法还包括：
    通过所述第一数据的下载地址，所述目标设备下载所述视频流的第一部分，并通过所述目标设备的第二显示屏显示所述视频流的第一部分；
    通过所述文本信息的下载地址，所述目标设备下载所述弹幕或字幕，并在所述第二显示屏上叠加显示所述弹幕或字幕的第一部分，其中，叠加显示的所述弹幕或字幕的第一部分与所述视频流的第一部分在时间上相关联。
  8. 如权利要求6所述的投屏方法，其特征在于，所述方法还包括：
    向所述目标设备发送所述文本信息的显示参数，其中所述显示参数包括所述文本信息显示的字体、字号、颜色、透明度、动画速度、显示区域中的至少一个。
  9. 如权利要求6所述的投屏方法，其特征在于，所述方法还包括：
    向所述目标设备发送第一文本信息，所述第一文本信息与所述第一数据相关联。
  10. 如权利要求6所述的投屏方法，其特征在于，所述方法还包括：
    检测到对目标设备终止投屏的第三操作；以及
    从目标设备接收所述文本信息的显示参数，其中所述显示参数包括所述文本信息显示的字体、字号、颜色、透明度、动画速度、显示区域中的至少一个。
  11. 一种投屏方法，用于在具有显示屏和扬声器的目标设备上投屏，其特征在于，所述方法包括：
    接收到第一数据的下载地址；
    通过所述第一数据的下载地址，从服务器请求下发所述第一数据，并通过所述显示屏显示所述第一数据，和/或通过所述扬声器输出所述第一数据；
    接收到文本信息的下载地址，其中，所述文本信息与所述第一数据相关联；以及
    通过所述文本信息的下载地址，从所述服务器请求下发所述文本信息，并通过所述显示屏显示所述文本信息。
  12. 如权利要求11所述的投屏方法，其特征在于，所述第一数据是视频流，所述文本信息为弹幕或字幕，其中，通过所述显示屏显示所述文本信息包括：
    在所述显示屏上显示的所述视频流上叠加显示所述弹幕或字幕，其中，叠加显示的所述弹幕或字幕与显示的所述视频流在时间上相关联。
  13. 如权利要求11所述的投屏方法，其特征在于，所述方法还包括：
    接收到所述文本信息的显示参数，并根据所述显示参数在所述显示屏上显示所述文本信息，其中，所述显示参数包括所述文本信息显示的字体、字号、颜色、透明度、动画速度、显示区域中的至少一个。
  14. 如权利要求11所述的投屏方法，其特征在于，所述方法还包括：
    检测到发送第一文本信息的第一操作；以及
    响应于所述第一操作，在所述显示屏上叠加显示所述第一文本信息，其中，所述第一文本信息与所述第一数据相关联。
  15. 如权利要求11所述的投屏方法，其特征在于，所述方法还包括：
    检测到设置所述文本信息的显示参数的第二操作；以及
    响应于所述第二操作，根据设置后的显示参数在所述显示屏上显示所述文本信息，其中，所述显示参数包括所述文本信息显示的字体、字号、颜色、透明度、动画速度、显示区域中的至少一个。
  16. 如权利要求15所述的投屏方法，其特征在于，所述方法还包括：
    接收到源设备终止投屏的请求；以及
    向源设备发送所述设置后的显示参数。
  17. 一种终端，包括显示屏、存储器、处理器，其中，所述存储器储存有一个或多个程序，其特征在于，当所述处理器在执行所述一个或多个程序时，使得所述终端实现如权利要求1至16任一项所述的方法。
  18. 一种计算机可读存储介质，包括指令，其特征在于，当所述指令在电子设备上运行时，使得所述电子设备执行如权利要求1-16中任一项所述的方法。
PCT/CN2021/098499 2020-06-08 2021-06-05 一种投屏方法和终端 WO2021249318A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP21821880.8A EP4148551A4 (en) 2020-06-08 2021-06-05 SCREEN PROJECTION METHOD AND TERMINAL
US18/000,981 US20230217081A1 (en) 2020-06-08 2021-06-05 Screen Casting Method and Terminal

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010514417.5 2020-06-08
CN202010514417.5A CN113835649B (zh) 2020-06-08 2020-06-08 一种投屏方法和终端

Publications (1)

Publication Number Publication Date
WO2021249318A1 true WO2021249318A1 (zh) 2021-12-16

Family

ID=78845308

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/098499 WO2021249318A1 (zh) 2020-06-08 2021-06-05 一种投屏方法和终端

Country Status (4)

Country Link
US (1) US20230217081A1 (zh)
EP (1) EP4148551A4 (zh)
CN (1) CN113835649B (zh)
WO (1) WO2021249318A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114786060A (zh) * 2022-02-28 2022-07-22 长沙朗源电子科技有限公司 基于安卓平台无线投屏实现帧率控制的方法及其装置
CN114827690A (zh) * 2022-03-30 2022-07-29 北京奇艺世纪科技有限公司 一种网络资源显示方法、装置及系统
CN115525453A (zh) * 2022-01-19 2022-12-27 荣耀终端有限公司 多屏协同中断的处理方法及电子设备

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115145507A (zh) * 2021-03-15 2022-10-04 华为技术有限公司 基于多设备的在线互动方法、芯片、电子设备及存储介质
CN114565070B (zh) * 2022-02-23 2023-11-10 Tcl通讯科技（成都）有限公司 一种标签绑定方法、装置及移动终端
CN118057794A (zh) * 2022-11-18 2024-05-21 广州视源电子科技股份有限公司 数据传输方法、电子设备和存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101179347A (zh) * 2007-12-21 2008-05-14 深圳市迅雷网络技术有限公司 一种提供文本文件信息的方法、系统及服务端
US20130042293A1 (en) * 2011-08-08 2013-02-14 Ite Tech. Inc. Method for transmitting extra information in digital broadcast contents and apparatus using the same
CN108063952A (zh) * 2017-12-14 2018-05-22 暴风集团股份有限公司 视频处理方法、装置及终端
CN108156480A (zh) * 2017-12-27 2018-06-12 腾讯科技（深圳）有限公司 一种视频字幕生成的方法、相关装置及系统
CN108810594A (zh) * 2018-05-04 2018-11-13 新华三技术有限公司 一种远程投屏方法、装置及系统

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150100993A1 (en) * 2012-03-13 2015-04-09 Lg Electronics Inc. Seamless playback method using bookmark, and apparatus and system therefor
US11076205B2 (en) * 2014-03-07 2021-07-27 Comcast Cable Communications, Llc Retrieving supplemental content
CN110119241B (zh) * 2019-04-26 2021-11-02 广州视源电子科技股份有限公司 一种投屏内容评价方法、装置、用户终端和接收终端

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101179347A (zh) * 2007-12-21 2008-05-14 深圳市迅雷网络技术有限公司 一种提供文本文件信息的方法、系统及服务端
US20130042293A1 (en) * 2011-08-08 2013-02-14 Ite Tech. Inc. Method for transmitting extra information in digital broadcast contents and apparatus using the same
CN108063952A (zh) * 2017-12-14 2018-05-22 暴风集团股份有限公司 视频处理方法、装置及终端
CN108156480A (zh) * 2017-12-27 2018-06-12 腾讯科技（深圳）有限公司 一种视频字幕生成的方法、相关装置及系统
CN108810594A (zh) * 2018-05-04 2018-11-13 新华三技术有限公司 一种远程投屏方法、装置及系统

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4148551A4

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115525453A (zh) * 2022-01-19 2022-12-27 荣耀终端有限公司 多屏协同中断的处理方法及电子设备
CN114786060A (zh) * 2022-02-28 2022-07-22 长沙朗源电子科技有限公司 基于安卓平台无线投屏实现帧率控制的方法及其装置
CN114827690A (zh) * 2022-03-30 2022-07-29 北京奇艺世纪科技有限公司 一种网络资源显示方法、装置及系统
CN114827690B (zh) * 2022-03-30 2023-07-25 北京奇艺世纪科技有限公司 一种网络资源显示方法、装置及系统

Also Published As

Publication number Publication date
CN113835649A (zh) 2021-12-24
EP4148551A4 (en) 2024-01-17
CN113835649B (zh) 2024-09-10
US20230217081A1 (en) 2023-07-06
EP4148551A1 (en) 2023-03-15

Similar Documents

Publication Publication Date Title
WO2020221039A1 (zh) 投屏方法、电子设备以及系统
WO2021249318A1 (zh) 一种投屏方法和终端
WO2020244495A1 (zh) 一种投屏显示方法及电子设备
EP4044609A1 (en) Cross-device content projection method and electronic device
WO2021164445A1 (zh) 一种通知处理方法、电子设备和系统
WO2022257977A1 (zh) 电子设备的投屏方法和电子设备
CN112394895B (zh) 画面跨设备显示方法与装置、电子设备
CN113497909B (zh) 一种设备交互的方法和电子设备
WO2020155014A1 (zh) 智能家居设备分享系统、方法及电子设备
JP7416519B2 (ja) マルチ端末マルチメディアデータ通信方法及びシステム
CN109819306B (zh) 一种媒体文件裁剪的方法、电子设备和服务器
WO2022105445A1 (zh) 基于浏览器的应用投屏方法及相关装置
WO2022037407A1 (zh) 一种回复消息的方法、电子设备和系统
WO2022017393A1 (zh) 显示交互系统、显示方法及设备
WO2022135527A1 (zh) 一种视频录制方法及电子设备
WO2022078295A1 (zh) 一种设备推荐方法及电子设备
CN112527222A (zh) 一种信息处理方法及电子设备
WO2022135157A1 (zh) 页面显示的方法、装置、电子设备以及可读存储介质
CN112527174A (zh) 一种信息处理方法及电子设备
WO2021143391A1 (zh) 基于视频通话的共享屏幕方法及移动设备
WO2024045801A1 (zh) 用于截屏的方法、电子设备、介质以及程序产品
WO2023030056A1 (zh) 一种通知消息的处理方法及设备
WO2023005711A1 (zh) 一种服务的推荐方法及电子设备
CN114079691A (zh) 一种设备识别方法及相关装置
WO2022206763A1 (zh) 一种显示方法、电子设备和系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21821880

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021821880

Country of ref document: EP

Effective date: 20221205

NENP Non-entry into the national phase

Ref country code: DE