WO2021185318A1 - 多媒体设备及投屏播放方法 - Google Patents

多媒体设备及投屏播放方法 Download PDF

Info

Publication number
WO2021185318A1
WO2021185318A1 PCT/CN2021/081561 CN2021081561W WO2021185318A1 WO 2021185318 A1 WO2021185318 A1 WO 2021185318A1 CN 2021081561 W CN2021081561 W CN 2021081561W WO 2021185318 A1 WO2021185318 A1 WO 2021185318A1
Authority
WO
WIPO (PCT)
Prior art keywords
terminal
video
buffer area
terminals
target terminal
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2021/081561
Other languages
English (en)
French (fr)
Inventor
宋子全
庞秀娟
朱雪生
肖成创
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hisense Visual Technology Co Ltd
Original Assignee
Hisense Visual Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN202010199814.8A external-priority patent/CN111427527B/zh
Priority claimed from CN202010519510.5A external-priority patent/CN111711843A/zh
Priority claimed from CN202010522106.3A external-priority patent/CN111726663A/zh
Application filed by Hisense Visual Technology Co Ltd filed Critical Hisense Visual Technology Co Ltd
Publication of WO2021185318A1 publication Critical patent/WO2021185318A1/zh
Priority to US17/804,257 priority Critical patent/US12273586B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/439—Processing of audio elementary streams
    • H04N21/4392—Processing of audio elementary streams involving audio buffer management
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H04N21/43615—Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44004—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving video buffer management, e.g. video decoder buffer or video display buffer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/643—Communication protocols
    • H04N21/64322—IP

Definitions

  • This application relates to the technical field of smart terminals, and in particular to a multimedia device and a projection screen playback method.
  • multi-channel mirroring In support of multi-channel mirroring, that is, it can support the simultaneous mirroring of the screen images of multiple terminals to the multimedia device.
  • the multiple terminals When multiple terminals are connected to the multimedia device, the multiple terminals will all send their own audio data to the multimedia device.
  • the multimedia device When a multimedia device receives audio data from multiple terminals, the multimedia device applies for the corresponding buffer (buffer) to store the corresponding audio data, that is, the multimedia device is connected to several terminals, and the multimedia device applies for several buffer areas.
  • the multimedia device plays For sound, read data from the selected corresponding audio storage buffer area and send it to the audio playback interface.
  • Each audio data corresponds to an audio voice unit.
  • one of the corresponding audio and voice units is not set to the mute state, and the sound is played normally , The sound of other audio and voice units corresponding to the other channels is set to mute.
  • This application provides a multimedia device and a projection screen playback method
  • a television provided by an embodiment of the present application includes: a processor, a display screen, and a voice unit;
  • the display screen is used to display video data in at least one video buffer area
  • the voice unit is used to play audio data in the audio buffer area
  • the processor is configured to determine a target mobile phone from the multiple mobile phones in the process of simultaneous screen projection with multiple mobile phones;
  • the audio data sent by the target mobile phone is stored in the audio buffer area, and the audio data sent by other mobile phones are discarded; the video data sent by multiple mobile phones is stored in the video buffer area corresponding to each mobile phone.
  • a projection screen playback method provided by an embodiment of the present application is applied to a TV, and the method includes:
  • FIG. 1 is a schematic diagram of the memory of a multimedia device in the background art during screen projection
  • FIG. 2 is a schematic diagram of the memory of a multimedia device during a screen projection process according to an embodiment of the present application
  • FIG. 3 is a schematic diagram of a connection between a TV and a mobile phone during a screen projection process according to an embodiment of the present application
  • FIG. 4 is a schematic diagram of a connection between a projector and a computer during a screen projection process according to an embodiment of the present application
  • FIG. 5 is a schematic structural diagram of a multimedia device provided by an embodiment of the present application.
  • FIG. 6 is a flowchart of a method for screencasting and playing according to an embodiment of the present application.
  • FIG. 7 is a flow chart of a method for projecting and playing on a TV provided by an embodiment of the present application.
  • FIG. 8 is a flowchart of a method for projecting and playing on a projector according to an embodiment of the present application.
  • FIG. 9 is a schematic diagram of interaction between a user and a multimedia device when the user selects a target terminal according to an embodiment of the present application.
  • FIG. 10 is a schematic diagram of a remote control for controlling a TV provided by an embodiment of the present application.
  • FIG. 11 is another schematic diagram of the interaction between the user and the multimedia device when the user selects a target terminal according to an embodiment of the present application
  • FIG. 12 is a working flow chart of a method for screencasting and playing in which a user selects a target terminal according to an embodiment of the present application
  • FIG. 13 is a work flow chart of a method for screencasting and playing in which a terminal randomly selects a terminal from a plurality of terminals as a target terminal according to an embodiment of the present application;
  • FIG. 14 is a work flow chart of a method for screencasting and playback in which a terminal selects the first connected terminal from a plurality of terminals as the target terminal according to an embodiment of the present application;
  • FIG. 15 is a work flow chart of a method for screencasting and playing in which a terminal selects the last connected terminal from a plurality of terminals as the target terminal according to an embodiment of the present application;
  • FIG. 16 is a work flow chart of a method for screencasting and playback of a switching target terminal provided by an embodiment of the present application
  • FIG. 17 is a schematic diagram of an operation scenario between a display device and a control device provided by an embodiment of the application.
  • FIG. 18 is a schematic diagram of a processing procedure for establishing a connection between a mobile phone and a smart TV provided by an embodiment of the application;
  • FIG. 19 is a schematic diagram of a user interface that provides users with untrust, trust and connect, and shield selection buttons provided by an embodiment of this application;
  • FIG. 20 is a schematic diagram of the management interface of the black and white lists presented to the user if the black and white lists on the TV terminal are empty according to an embodiment of this application;
  • FIG. 21 is a schematic diagram of the management interface of the black and white lists that are presented to the user if the user needs to add a common device according to an embodiment of the application;
  • FIG. 22 is a schematic diagram of the management interface of the black and white lists that are presented to the user if the user needs to remove the commonly used equipment provided by the embodiment of this application;
  • FIG. 23 is a schematic diagram of a prompt interface presented to the user when the device identifier is removed from a commonly used device according to an embodiment of this application;
  • FIG. 24 is a schematic diagram of the management interface of the black and white lists that are presented to the user if the user needs to remove the shielding device according to an embodiment of this application;
  • FIG. 25 is a schematic diagram of a prompt interface presented to the user when the identification of the device is removed from the shielding device according to an embodiment of the application;
  • FIG. 26 is a schematic diagram of the management interface of the black and white lists that the user can also add a shielding device provided by this embodiment of the application;
  • FIG. 27 is a schematic structural diagram of a display device provided by an embodiment of this application.
  • FIG. 28 is a schematic flowchart of a control method for establishing a connection with a display device according to an embodiment of the application
  • FIG. 29 is a schematic structural diagram of a screen projection system provided by an embodiment of the application.
  • FIG. 30 is a flowchart of a screen projection method provided by an embodiment of the application.
  • FIG. 31 is a schematic diagram of an end-to-end discovery device provided by an embodiment of this application.
  • FIG. 32 is a signaling diagram of a screen projection method provided by an embodiment of this application.
  • FIG. 33 is a schematic structural diagram of a screen projection device provided by an embodiment of the application.
  • FIG. 34 is a schematic structural diagram of a screen projection device provided by an embodiment of the application.
  • each terminal sends to the multimedia device
  • the multimedia device will apply for an audio buffer area and a video buffer area.
  • four A terminal will apply for 8 buffer areas, four audio buffer areas and four video buffer areas in the memory of the multimedia device, and store the corresponding audio data in each audio buffer area, and store the corresponding video in each video buffer area. data.
  • the embodiments of the present application provide a television, a multimedia device, and a screencast playback method.
  • the audio data of the target terminal is stored in the audio buffer area, and the audio data of other terminals are discarded. Without affecting the normal playback, the memory of the multimedia device is saved.
  • the memory of the multimedia device in the embodiment of the present application includes an audio buffer area, four video buffer areas, and an audio buffer area stores the audio of the target terminal. Data, the video data of the corresponding four terminals are stored in the four video buffer areas.
  • the method provided by this application can greatly save the memory of the multimedia device, especially when a terminal that is more connected to a multimedia device is used for screen projection.
  • the multimedia device 300 shown in FIG. 3 is only an example, and the multimedia device 300 may have more or fewer components than those shown in FIG. 3, two or more components may be combined, or Can have different component configurations.
  • the various components shown in the figure may be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and/or application specific integrated circuits.
  • Fig. 3 exemplarily shows a block diagram of the hardware configuration of the multimedia device 300 according to an exemplary embodiment.
  • the multimedia device 300 includes: a radio frequency (RF) circuit 310, a memory 320, a display unit 330, a voice unit 340, a wireless fidelity (Wireless Fidelity, Wi-Fi) module 350, a processor 360, Bluetooth module 370, power supply 380 and other components.
  • RF radio frequency
  • the processor 360 is the control center of the multimedia device 300, which uses various interfaces and lines to connect various parts of the entire terminal, and executes by running or executing software programs stored in the memory 320, and calling data stored in the memory 320.
  • the processor 360 may include one or more processing units; the processor 360 may also integrate an application processor and a baseband processor, where the application processor mainly processes the operating system, user interface, and application programs, etc.
  • the baseband The processor mainly deals with wireless communication. It can be understood that the aforementioned baseband processor may not be integrated into the processor 360.
  • the processor 360 in this application can run an operating system, an application program, a user interface display and touch response, as well as the screencast playback method described in the embodiment of this application.
  • the processor 360 is coupled to the display unit 330.
  • the display unit 330 mentioned in this application is for different multimedia devices, and the structure of the display unit 330 is also different.
  • the display unit 330 may be used to receive input digital or character information, and generate signal input related to the user settings and function control of the multimedia device 300.
  • the display unit 330 may include the information provided in the multimedia device 300.
  • the front touch screen 332 can collect user touch operations on or near it, such as clicking a button, dragging a scroll box, and so on.
  • FIG. 3 shows the structure of the display unit 330 taking the multimedia device as a TV as an example.
  • the display unit 330 may also be used to display information input by the user or information provided to the user, as well as a graphical user interface (GUI) of various menus of the multimedia device 300.
  • the display unit 330 may include a display screen 331 provided on the front of the multimedia device 300.
  • the display screen 331 may be configured in the form of a liquid crystal display, a light emitting diode, or electronic ink.
  • the display unit 330 may be used to display various graphical user interfaces described in this application. Among them, the display screen formed by using electronic ink is usually called an ink screen.
  • the touch screen 332 can be overlaid on the display screen 331, or the touch screen 332 can be integrated with the display screen 331 to realize the input and output functions of the multimedia device 300. After integration, it can be referred to as a touch display screen.
  • the screen made of electronic ink is Ink screen.
  • the display unit 330 may display application programs and corresponding operation steps.
  • the display unit 330 is an imaging system.
  • the imaging system includes a light source, an image display element, and a lens.
  • the image display element is irradiated by the light source to produce an image, and then the video data is projected onto the wall or projection cloth through the lens.
  • the voice unit 340 includes a speaker 341 and a microphone 342 which can provide an audio interface between the user and the multimedia device 300.
  • the voice unit 340 can transmit the electrical signal converted from the received audio data to the speaker 341, and the speaker 341 converts it into a sound signal for output.
  • the multimedia device 300 may also be equipped with a volume button for adjusting the volume of the sound signal.
  • the microphone 342 converts the collected sound signals into electrical signals, which are received by the voice unit 340 and converted into audio data, and then output the audio data to the memory 320 for further processing, such as controlling the multimedia device by voice. In this application, the microphone 342 can acquire the user's voice.
  • the memory 320 can be used to store software programs and data.
  • the processor 360 executes various functions and data processing of the multimedia device 300 by running software programs or data stored in the memory 320.
  • the memory 320 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.
  • the memory 320 stores an operating system that enables the multimedia device 300 to run. In the present application, the memory 320 may store an operating system and various application programs, and may also store codes for executing the projection screen playback method described in the embodiments of the present application.
  • the RF circuit 310 can be used for receiving and sending signals in the process of sending and receiving information or talking. It can receive the downlink data of the base station and then send it to the processor 360 for processing; it can send the uplink data to the base station.
  • the RF circuit includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and other devices.
  • Wi-Fi is a short-distance wireless transmission technology.
  • the multimedia device 300 can help users send and receive emails, browse web pages, and access streaming media through the Wi-Fi module 350.
  • the Wi-Fi module 350 provides users with wireless broadband Internet access.
  • the bluetooth module 370 is used to exchange information with other bluetooth devices with bluetooth modules through the bluetooth protocol.
  • the multimedia device 300 may establish a Bluetooth connection with a wearable electronic device (such as a smart watch) that also has a Bluetooth module through the Bluetooth module 370, so as to perform data interaction.
  • the multimedia device 300 also includes a power source 380 (such as a battery) for supplying power to various components.
  • the power supply can be logically connected to the processor 360 through the power management system, so that functions such as charging, discharging, and power consumption can be managed through the power management system.
  • the multimedia device 300 may also be configured with a power button, which is used to power on and off the terminal, and lock the screen.
  • the processor 360 is configured to determine a target terminal from the multiple terminals during simultaneous screen projection with multiple terminals; store the audio data sent by the target terminal in the audio buffer area, and discard other The audio data sent by the terminal; and the video data sent by multiple terminals are stored in the video buffer area corresponding to each terminal.
  • the display unit 330 is configured to display video data in at least one video buffer area.
  • the voice unit 340 is used to play audio data in the audio buffer area.
  • the multimedia equipment is a television
  • the terminal is a mobile phone as an example.
  • the TV 400 selects a target from the connected 4 mobile phones.
  • the TV stores the audio data sent by the mobile phone 42 in the audio storage area, and discards the audio data sent by the mobile phone 41, the mobile phone 43, and the mobile phone 44.
  • the TV will send the video data from 4 mobile phones.
  • mobile phone 41 Stored in the video storage area corresponding to each mobile phone, mobile phone 41 sends video 411, mobile phone 42 sends video 422, mobile phone 43 sends video 433, mobile phone 44 sends video 444, TV 400 reads data from the audio storage area and plays the audio of mobile phone 42 , Read the data from the video storage and display the video data of at least one mobile phone on the display screen, for example, display the video 411 and the video 422.
  • the multimedia equipment is a projector
  • the terminal is a computer.
  • the processor 360 has the following functions:
  • the processor 360 is specifically configured to: use the terminal selected by the user from the multiple terminals as the target terminal; or
  • the Nth connected terminal is selected from the multiple terminals as the target terminal, where N is a positive integer.
  • the display unit 330 For the mode that the terminal selected by the user from the multiple terminals is used as the target terminal, the display unit 330 needs to interact with the processor 360 to implement it. In detail, the display unit 330 is also used to receive the user's display on the multimedia device. For the operation selection video instruction, the processor 360 is specifically configured to respond to the user's operation selection video instruction on the multimedia device, and use the terminal corresponding to the video selected by the selection video instruction as the target terminal.
  • the processor 360 has the following functions:
  • the processor 360 is further configured to: in the process of simultaneous screen projection with multiple terminals, detect the connection status of the multiple terminals; if it is detected that the target terminal is disconnected, delete the audio buffer area The audio data of the target terminal and the memory space of the video buffer area corresponding to the target terminal are released, and a new target terminal is determined from the remaining terminals; the audio data sent by the new target terminal is stored in the audio buffer area.
  • the processor 360 is further configured to release the memory space of the audio buffer area if it is detected that all terminals are disconnected.
  • S600 In the process of simultaneous screen projection with multiple terminals, the target terminal is determined from the multiple terminals.
  • S601 Store the audio data sent by the target terminal in the audio buffer area, and discard the audio data sent by other terminals; store the video data sent by multiple terminals in the video buffer area corresponding to each terminal.
  • S602 Play the audio data in the audio buffer area, and display the video data in at least one video buffer area.
  • the audio data in the audio storage area when the audio data in the audio storage area is played, the audio data in the audio buffer area is read and played, while the video data in at least one video buffer area is read for display. This saves the memory of the multimedia device while ensuring that the videos of multiple terminals connected to the multimedia device are normally screened and one channel of audio data is played.
  • the multimedia device can be a TV, a projector, and so on.
  • the multimedia device is a TV and the terminal is a mobile phone, as shown in Figure 7, the specific screencasting process is as follows:
  • S700 In the process of screen projection with multiple mobile phones at the same time, determine the target mobile phone from multiple mobile phones;
  • S701 Store the audio data sent by the target mobile phone in the audio buffer area, and discard the audio data sent by other mobile phones; store the video data sent by multiple mobile phones in the video buffer area corresponding to each mobile phone;
  • S702 Play the audio data in the audio buffer area, and display the video data in at least one video buffer area on the display screen.
  • the multimedia device is a projector and the terminal is a computer, as shown in Figure 8, the specific projection and playback process is:
  • S800 In the process of screen projection with multiple computers at the same time, determine the target computer from multiple computers;
  • S801 Store the audio data sent by the target computer in the audio buffer area, and discard the audio data sent by other computers; store the video data sent by multiple computers in the video buffer area corresponding to each computer;
  • S802 Playing the audio data in the audio buffer area, and projecting and displaying the video data in at least one video buffer area.
  • the method for determining the target terminal from multiple terminals may include the following:
  • Method 1 Use the terminal selected by the user from multiple terminals as the target terminal.
  • This method provides a user-defined method, and provides a terminal that plays the sound through the user's independent choice.
  • the multimedia device responds to the video selection instruction operated by the user on the multimedia device, and uses the terminal corresponding to the video selected by the video selection instruction as the target terminal.
  • video 1 is a video formed by video data sent by terminal 1
  • video 2 is a video formed by video data sent by terminal 2
  • video 3 is a video formed by video data sent by terminal 3
  • video 4 is video data sent by terminal 4. The formed video.
  • the page By default, it is displayed in the order of connecting to the multimedia device.
  • the video of the first terminal connected to the multimedia device and the video of the second terminal connected to the multimedia device are displayed on the first page, for example, as shown in the figure As shown in 9, video 1 and video 2, at the same time, the page also includes a "next page” button, when the user clicks "next page", video 3 and video 4 can be displayed.
  • the multimedia device When the user clicks on the selected video, for example, selects video 3, the multimedia device responds to the user's operation on the multimedia device to select the video instruction, highlighting the play area of the selected video 3, and at the same time selects the terminal corresponding to the video 3 selected by the video instruction 3 As the target terminal, after the target terminal is selected, the multimedia device stores the audio data of the terminal 3, and the multimedia device reads the audio data of the terminal 3 for playback.
  • the remote control contains a ring control button.
  • the circle in the middle surrounded by the ring control button is the OK button.
  • the ring control button includes up, down, left, and right buttons in four different directions.
  • the upper button It is set at the top of the ring, using "+”
  • the lower button is located at the bottom of the ring, using "-”
  • the left button is set on the left of the ring, using "-”
  • the right button is set on the ring On the right, it is indicated by "+”.
  • the left button and the right button are page turning functions. For example, the user presses the right button to go to the next page, the left button to return to the previous page, the upper button is the previous video, and the lower button is the next video.
  • the user selects he can turn the page by manipulating the left and right buttons of the remote control to view the video in the terminal connected to the multimedia device.
  • the user turns the page by manipulating the left and right buttons of the remote control to view the video in the terminal connected to the multimedia device.
  • the user selects the video through the upper button and the lower button.
  • the multimedia device will highlight the playback area of the selected video 3 after receiving the user’s selection of the video through the up button or the down button, and then after the user clicks the OK button, the terminal 3 corresponding to the video 3
  • the multimedia device stores the audio data of the terminal 3, and the multimedia device reads the audio data of the terminal 3 for playback.
  • buttons There may be no physical buttons, or the number of physical buttons can be set as needed, and the button arrangement can also be set in different positions as needed.
  • the user independently selects the voting method for the target terminal, including:
  • S1200 In the process of simultaneous screen projection with multiple terminals, apply for memory space from the memory of the multimedia device as the video buffer area corresponding to the multiple terminals;
  • S1210 Store the video data sent by multiple terminals in the video buffer area corresponding to each terminal, and discard the audio data sent by multiple terminals at the same time;
  • S1220 Display video data in multiple video buffer areas
  • S1250 Store the audio data sent by the target terminal in the audio buffer area, and discard the audio data sent by other terminals;
  • the process of automatically applying for the audio buffer area is triggered to apply for the memory space of the audio buffer area from the memory, so as to meet the purpose of storing only the target terminal in the audio buffer. Save the memory of multimedia devices.
  • Method 2 Select one terminal from multiple terminals as the target terminal.
  • the multimedia device can randomly select a terminal from a plurality of terminals as the target terminal. That is, when a multimedia device is connected to four terminals, one terminal can be randomly selected from the four terminals as the target terminal. Since multiple terminals are connected to the multimedia device, they will send audio data and video data to the multimedia device. Therefore, the multimedia device It is necessary to confirm that all terminals are connected before the target terminal can be selected. Based on this, the multimedia device automatically selects the target terminal's projection screen playback method, as shown in Figure 13, including:
  • S1300 Start timing when it is detected that the first terminal is connected to the multimedia device
  • S1310 Discard the audio data sent by the terminal connected to the multimedia device, apply for memory space from the memory of the multimedia device as a video buffer area corresponding to the terminal connected to the multimedia device, and store the video data sent by the terminal connected to the multimedia device;
  • S1330 Determine whether the detection duration reaches the preset time, if yes, execute S1340, otherwise, execute S1310;
  • S1340 randomly select a terminal from a plurality of terminals connected to the multimedia device as the target terminal;
  • S1360 Store the audio data sent by the target terminal in the audio buffer area, and discard the audio data sent by other terminals;
  • the multimedia device can select a terminal corresponding to a preset video type from multiple terminals as the target terminal.
  • the multimedia device when detecting that the terminal is connected to the multimedia device, applies for the video buffer area corresponding to the terminal, stores the video data sent by the terminal, detects the video type in the video, and determines if the video type meets the preset video type
  • the terminal is the target terminal.
  • the multimedia device applies for an audio buffer area in the memory, and stores the audio data sent by the target terminal in the audio buffer area. At the same time, it does not judge whether the video connected to the multimedia device meets the preset video category. Directly apply for the video buffer area corresponding to the terminal to store the video data sent by the terminal.
  • the multimedia device discards the audio data sent by the terminal, and continues to detect whether the video type of the next terminal connected to the multimedia device meets the preset video type.
  • the audio data sent by the target terminal is stored in the audio buffer area.
  • the video connected to the multimedia device is not judged whether it meets the preset video category, and directly applies for the video buffer area corresponding to the terminal to store the video data sent by the terminal.
  • Manner 3 Select the Nth connected terminal from multiple terminals as the target terminal, where N is a positive integer.
  • the Nth connected terminal can be the first or the last. That is, select the first connected terminal from multiple terminals as the target terminal, or select the last connected terminal from multiple terminals as the target terminal.
  • the multimedia device automatically selects the first terminal connected to the multimedia device as the target terminal's screencast playback method, as shown in FIG. 14, including:
  • S1400 After the first terminal is connected to the multimedia device, apply for two memory spaces from the memory of the multimedia device as the audio buffer area and the video buffer area respectively;
  • S1410 Store the audio data sent by the first terminal in the audio buffer area, and store the video data sent by the first terminal in the corresponding video buffer area;
  • S1440 Discard audio data sent by other terminals.
  • the audio data sent by the first terminal is stored in the audio buffer area, and the video data sent by the first terminal is stored in the corresponding video buffer In the area, for the terminals connected to the multimedia device except the first terminal, only the video data is stored, and the sent audio data is discarded.
  • the multimedia device automatically selects the last terminal connected to the multimedia device as the target terminal's screencast playback method. Take the connection of three terminals to the multimedia device as an example, as shown in Figure 15, including:
  • S1510 Store the audio data sent by the first terminal in the audio buffer area, and store the video data sent by the first terminal in the corresponding video buffer area;
  • S1520 After the second terminal is connected to the multimedia device, delete the audio data sent by the first terminal in the audio buffer area, and store the audio data sent by the second terminal in the audio buffer area;
  • S1550 After the third terminal is connected to the multimedia device, delete the audio data sent by the second terminal in the audio buffer area, and store the audio data sent by the third terminal in the audio buffer area;
  • S1570 Store the video data sent by the third terminal in the corresponding video buffer area.
  • this application since this application specifies that the last terminal connected to the multimedia device is the target terminal, after the first terminal is connected to the multimedia device, since there is only one terminal in the multimedia device, this terminal is the first connected terminal The terminal is also the last connected terminal. Therefore, this application will store the audio data of the first terminal in the audio buffer area of the application.
  • the multimedia device When the second terminal is connected to the multimedia device, the multimedia device includes two terminals, the second The terminal is the last terminal connected to the multimedia device, so it is necessary to store the audio data sent by the second terminal in the audio buffer area, and delete the audio data stored in the audio buffer area of the first terminal.
  • the audio data sent by the first terminal is discarded, and the video data sent by the first terminal can continue to be stored in the corresponding video buffer area.
  • the multimedia device when the third terminal is connected to the multimedia device, the multimedia device includes three terminals.
  • the third terminal is the last terminal connected to the multimedia device, so the audio sent by the third terminal needs to be stored in the audio buffer area.
  • the audio data stored in the audio buffer area of the second terminal is deleted at the same time.
  • the audio data sent by the first terminal and the second terminal are discarded, and the audio data sent by the first terminal is discarded.
  • the video data can continue to be stored in the corresponding video buffer area, and the video data sent by the second terminal can continue to be stored in the corresponding video buffer area.
  • the multimedia device can first automatically select a terminal as the target. Terminal, and then switch the target terminal to the terminal selected by the user.
  • the specific process is shown in combination with Figure 16:
  • S1600 During simultaneous screen projection with multiple terminals, select the Nth connected terminal from the multiple terminals as the target terminal;
  • S1610 Store the audio data sent by the target terminal in the audio buffer area, and discard the audio data sent by other terminals; store the video data sent by multiple terminals in the video buffer area corresponding to each terminal;
  • S1620 Playing the audio data in the audio buffer area, and displaying the video data in at least one video buffer area;
  • S1650 Play the audio data sent by the terminal corresponding to the video selected by the video selection instruction.
  • the multimedia device will deal with the data stored in the terminal and the memory space allocated to the terminal to store data. If it is not released, the memory space is still not included in the available memory space.
  • the embodiment of the present application also provides a processing method after the terminal is disconnected from the multimedia device, including:
  • the released memory space is the available space.
  • the memory space of the corresponding video buffer area is released, so that data other than the video data sent by the target terminal can use the memory space of the video buffer area, thereby improving the utilization of the memory .
  • the multimedia device will always play the audio stored in the audio buffer area, which will cause the screencast video and the played audio to be different. Therefore, the application needs to re-determine a new target terminal from the remaining terminals, and change the audio data sent by the new target terminal from the operation of discarding to the operation of storing the audio data in the audio buffer area.
  • the memory space of the video buffer area corresponding to the non-target terminal is released. In this way, data other than the video data sent by the non-target terminal uses the memory space of the video buffer area, thereby improving the utilization rate of the memory.
  • the audio data of the target terminal is still stored in the audio buffer area, so it has no effect on the audio buffer area, and only the memory space of the video buffer area corresponding to the non-target terminal needs to be released.
  • the memory space of the audio buffer area will be released for the purpose of improving the utilization of the memory, so that other data can use the memory space of the audio buffer area.
  • the embodiments of the present application provide a display device and a control method for establishing a connection with the display device.
  • a display device such as a mobile phone, a computer, a tablet computer, and other terminal devices
  • a television device such as a smart TV
  • the identifications of the terminal devices through all connection methods are classified and saved on the TV, divided into whitelists and blacklists, and can be managed. Only mobile devices in the whitelist allow them to perform Connect and respond to its control commands, and the devices in the blacklist are directly shielded in the connection phase, and the user will not perceive it. If the device is not in the whitelist or blacklist, a dialog box will pop up every time the user connects to block, distrust, trust and connect, thereby greatly improving the security of the multi-screen interactive function and avoiding illegal terminals and smart devices.
  • the TV establishes a connection, so as to prevent the TV from being threatened, and to prevent the user from unknowingly, the TV at home is illegally connected and manipulated by other people, or illegal content is cast on the TV, which affects the normal use of the TV.
  • the methods and devices described in the embodiments of the present application are based on the same application concept. Since the principles of the methods and devices for solving problems are similar, the implementation of the methods and devices can be referred to each other, and the repetitions will not be repeated.
  • FIG. 17 exemplarily shows a schematic diagram of an operation scenario between a display device and a control device in an embodiment of the present application. As shown in FIG. 17, the user can operate the display device 200 by controlling the device 100.
  • control device 100 may be a remote control 100A, including infrared protocol communication or Bluetooth protocol communication, and other short-distance communication methods, etc., to control the display device 200 through wireless or other wired methods.
  • the user can control the display device 200 by inputting user instructions through keys on the remote control, voice input, control panel input, etc.
  • the user can control the display device 200 by inputting corresponding control commands through the volume plus and minus keys, channel control keys, up/down/left/right movement keys, voice input keys, menu keys, and power on/off keys on the remote control. Function.
  • the control device 100 may also be a smart device, such as a mobile terminal 100B, a tablet computer, a computer, a notebook computer, and the like.
  • a smart device such as a mobile terminal 100B, a tablet computer, a computer, a notebook computer, and the like.
  • an application program running on a smart device is used to control the display device 200.
  • the application can provide users with various controls through an intuitive user interface (UI) on the screen associated with the smart device.
  • UI intuitive user interface
  • the mobile terminal 100B may install a software application with the display device 200, realize connection communication through a network communication protocol, and realize the purpose of one-to-one control operation and data communication.
  • the mobile terminal 100B can establish a control instruction protocol with the display device 200, synchronize the remote control keyboard to the mobile terminal 100B, and realize the function of controlling the display device 200 by controlling the user interface on the mobile terminal 100B.
  • the audio and video content displayed on the mobile terminal 100B can also be transmitted to the display device 200 to realize the synchronous display function.
  • the display device 200 also performs data communication with the server 200A through multiple communication methods.
  • the display device 200 may be allowed to communicate through a local area network (LAN), a wireless local area network (WLAN), and other networks.
  • the server 200A can provide various contents and interactions to the display device 200.
  • the display device 200 can receive software program updates or access a remotely stored digital media library by sending and receiving information and interacting with an electronic program guide (EPG).
  • EPG electronic program guide
  • the server 200A may be one group or multiple groups, and it may be one type or multiple types of servers.
  • the server 200A provides other network service contents such as video-on-demand and advertising services.
  • the display device 200 may be a liquid crystal display, an OLED display, or a projection display device.
  • the specific display device type, size, resolution, etc. are not limited, and those skilled in the art can understand that the display device 200 can make some changes in performance and configuration as required.
  • the display device 200 may additionally provide a smart network TV function with a computer support function. Examples include Internet TV, Smart TV, Internet Protocol TV (IPTV), and so on.
  • IPTV Internet Protocol TV
  • the specific implementation process of the technical solution provided by the embodiment of the present application includes, for example:
  • Step S201a The TV terminal receives a connection request sent by an external device, the connection request carries the identifier of the external device, and the TV terminal detects whether the identifier is in the blacklist or whitelist preset by the TV terminal, if both are If not, execute step S202;
  • step S201b is executed, that is, it is determined whether the external device is requested to connect through the TV protocol, if not, then step S202 is executed; if so, Step S203 is executed. If the identifier of the external device is in the blacklist preset by the TV, step S205 is executed.
  • protocol types of the TV-side protocol include:
  • Projection protocol such as DLNA, Miracast, Airplay, etc.
  • Step S202 A dialog box pops up on the TV side to provide the user with selection buttons such as distrust, trust and connect, and shield, for example, the user interface shown in FIG. 19.
  • step S204 is executed
  • step S205 is executed
  • step S206 is executed.
  • Step S203 If the TV protocol is a connection protocol, the connection between the external device and the TV is directly established; if the TV protocol is a projection protocol, the screen projection content of the external device is displayed on the TV.
  • Step S204 If there is no identification of the external device in the white list, the identification of the external device is added to the white list preset by the television, and the external device is further connected to the television.
  • the protocol type (projection protocol or private protocol) of the TV protocol used to establish the connection is also added to the white list preset by the TV terminal.
  • the protocol type of the television protocol (projection protocol or private protocol) used by the external device to establish a connection with the television is also added to the In the white list preset on the TV side;
  • the television protocol used to establish the connection between the external device and the television terminal is a connection protocol
  • the connection between the external device and the television terminal is directly established, if the external device establishes a connection with the television terminal
  • the TV-side protocol used for the connection is a screen-casting protocol, and the screen-cast content of the external device is displayed on the TV terminal.
  • step S205 the TV terminal does not perform any processing, or rejects the connection request of the external device, and further, it can also send an alarm message.
  • Step S206 Add the protocol type of the television protocol used by the external device to establish the connection with the television, and the identifier of the external device to the blacklist.
  • the user can also open the black and white list management interface in the TV settings, and the user can call up the black and white lists for maintenance and management, such as deleting the device identification and the TV protocol that have been added to the black and white lists. If the device requests to connect to the TV again next time, repeat the above step 1. Specifically, for example:
  • the management interface of the black and white lists presented to the user is shown in Fig. 20, where commonly used devices are white lists and blocked devices are black lists.
  • the black and white list management interface presented to the user is shown in Figure 21.
  • a new device identification can be added, such as device 1, and of course other related content can be added, such as TV protocol, etc.
  • the black and white list management interface presented to the user is shown in Figure 22.
  • the device identification can be removed.
  • the prompt interface is shown in Figure 23.
  • the user is asked to choose whether to remove the device.
  • users can also remove other related content, such as TV-side protocols.
  • the management interface of the black and white list presented to the user is shown in Figure 24.
  • the device identification can be removed.
  • the prompt interface for the user is shown in Figure 25.
  • the user is asked to confirm whether to cancel the shielding of Device Five.
  • users can also remove other related content, such as TV-side protocols.
  • the user can also add a shielded device, and the management interface of the black and white list presented to the user is shown in Figure 26.
  • the user can add a new device identifier to the shielded device list. For example, when the user selects The shielding button shown and when the user selects the shielding button shown in FIG. 23 will present the interface shown in FIG. 26, and the identification of the shielding device that the user has confirmed to be added is added to the shielding device list.
  • a display device provided by an embodiment of the present application includes:
  • the display 11 is configured to display a user interface and screen projection content of an external device connected to the display device;
  • the controller 12 that communicates with the display is configured to execute the following processes to implement the display 11 to display a user interface, screen projection content of an external device connected to the display device, etc.:
  • the identifier of the external device When receiving the connection request of the external device, obtain the identifier of the external device from the connection request; for example, the identifier of the external device carried in the connection request of the received external device is "device one";
  • the list includes identifiers of external devices that can establish a connection with the display device
  • the blacklist includes identifiers of external devices that cannot establish a connection with the display device.
  • the display device when receiving a connection request from an external device, judges whether the identifier of the external device carried in the connection request is included in the blacklist or the whitelist, and determines whether to establish according to the judgment result
  • the connection between the display device and the external device improves the security of the multi-screen interactive function and prevents illegal terminals from establishing a connection with the smart TV.
  • the controller 12 is further configured to:
  • Control the display to present a user interface containing buttons for untrust, trust and connect, and shield selection for example, the interface shown in Figure 19, where "XX mobile phone” is the identification of the external device requesting to establish a connection); wherein,
  • the identification of the external device is added to the white list, and if the protocol used to establish the connection between the external device and the display device is a connection protocol, then all the external devices are directly established.
  • the protocol used to establish the connection between the external device and the display device is a projection protocol, control the display 11 to display the projection content of the external device;
  • control the display device not to establish a connection with the external device for example, perform the above-mentioned process of step S205;
  • the identifier of the external device is added to the blacklist.
  • the controller 12 when the identifier of the external device is added to the white list, the controller 12 also adds the protocol used to establish a connection between the external device and the display device to all In the white list.
  • the controller 12 while adding the identifier of the external device to the blacklist, the controller 12 also adds the protocol used to establish the connection between the external device and the display device to all Said in the blacklist.
  • the identifier of the external device if the identifier of the external device is in the whitelist, and the external device requests to establish a connection through the protocol of the display device, then:
  • the controller controls to directly establish a connection between the external device and the display device.
  • the protocol used by the display device to establish a connection is a projection protocol, and the controller controls the display to display the projection content of the external device.
  • the controller 12 is further configured to:
  • Control the display to present a user interface containing buttons for untrust, trust and connect, and shield selection for example, the interface shown in Figure 19, where "XX mobile phone” is the identification of the external device requesting to establish a connection); wherein,
  • the identification of the external device is added to the white list, and if the protocol used to establish the connection between the external device and the display device is a connection protocol, then all the external devices are directly established.
  • the protocol used to establish the connection between the external device and the display device is a projection protocol, controlling the display to display the projection content of the external device;
  • the identifier of the external device is added to the blacklist.
  • the controller 12 controls the display device not to establish a connection with the external device.
  • controller 12 is further configured to:
  • Control the display to present the management interface of the black and white list of the display device (such as the interface shown in Figure 20 to Figure 26), so that the user can edit and manage the black list and/or white list through the management interface, For example, operations such as deleting and adding an external device identifier in the blacklist and/or whitelist, etc., refer to the foregoing content for details, and will not be repeated here.
  • a method for controlling connection establishment with a display device includes:
  • S102 Determine whether the identifier of the external device carried in the connection request is included in a preset blacklist or whitelist, and determine whether to establish a connection between the external device and the display device according to the judgment result, wherein
  • the whitelist includes identifiers of external devices that can establish a connection with the display device
  • the blacklist includes identifiers of external devices that cannot establish a connection with the display device.
  • the technical solutions provided by the embodiments of this application can categorize and save the identifications of terminal devices through all connection modes on the TV, divided into whitelists and blacklists, and can be managed. Only mobile phones in the whitelist can be managed. Terminals such as devices allow it to connect with the TV and respond to its control instructions, while the devices in the blacklist are directly shielded in the connection stage, and the user will not perceive it. If a device is not in the whitelist or blacklist, a dialog box will pop up every time the user connects, allowing the user to choose to block, untrust, or trust and connect. Thus, the security of the multi-screen interactive function can be greatly improved.
  • the screen projection method provided by the embodiment of the present application may be applicable to the screen projection system shown in FIG. 29.
  • the screen projection system includes: a first terminal device 291, an access network device 292, and a second terminal device 293.
  • the first terminal device 291 and the second terminal device 293 may be devices such as mobile phones, pads, computers, and televisions
  • the access network device 292 may be a router.
  • the first terminal device 291 may be a mobile phone
  • the access network device 292 may be a router
  • the second terminal device 293 may be a TV.
  • the router can be used as an access point (Access Point, AP) to establish a local area network, and both the mobile phone and the TV are located in the local area network.
  • AP Access Point
  • a user opens a certain video resource on iQiyi, Youku, Tencent Video and other video software, and wants to cast the video resource to the TV for viewing, he can initiate a screencast request by clicking the screencast button on the video software At this time, the user clicking the screen projection button is equivalent to initiating a device discovery request.
  • the device discovery request will first reach the router, and the router will search for other devices within the above-mentioned local area network and feed back the search results to the user.
  • the device discovery rate will be low, and if the router is set up with AP isolation, it will also cause the device to be unable to be discovered, which will result in the TV not being searched, that is, the TV cannot be found. As a result, DLNA screen projection cannot be realized, which affects user experience.
  • the screen projection method provided in the embodiments of the present application aims to solve the above technical problems in the prior art.
  • FIG. 30 is a flowchart of a screen projection method provided by an embodiment of the application.
  • the embodiments of the present application provide a screen projection method in response to the above technical problems in the prior art, and the specific steps of the method are as follows:
  • Step 901 Send a first discovery request.
  • the first discovery request adopts the DLNA communication protocol.
  • the execution subject of this embodiment may be a terminal device, such as a mobile phone.
  • the terminal device serves as the control point, and the terminal device can send the first discovery request in the form of sending a multicast search message.
  • the mobile phone can be based on the user's operation information on the mobile phone received by the mobile phone to send the first discovery request. For example, the user opens the video software and clicks the screen projection button on the video software to initiate the first discovery request. After the discovery request, the first discovery request is sent to the router in the local area network where the mobile phone is located.
  • the operation changes the motion state of the terminal device, thereby initiating the first discovery request.
  • Step 902 If it is determined that the target device is not found according to the response result of the first discovery request, send a second discovery request.
  • the second discovery request adopts an end-to-end communication protocol, such as a P2P (Peer to Peer) protocol.
  • the second discovery request includes a WIFI direct connection request.
  • the router will send the first discovery request to other devices in the LAN.
  • the other devices After receiving the first discovery request, the other devices will reply with a confirmation message to Terminal equipment, and reply to the terminal equipment such information as the description file of the Digital Media Renderer (DMR). If the terminal device does not receive any confirmation message, or the terminal device determines that the target device is not found according to the received confirmation message and the reply message of the DMR description file, it will automatically switch to the P2P protocol, that is, start the P2P protocol to initiate the second Discovery request.
  • DMR Digital Media Renderer
  • terminal device and the target device in this embodiment must be connected to the same WIFI network.
  • sending the second discovery request includes: the confirmation message of the multicast search message is not received within a preset time period , Send the second discovery request. For example, after a user sends a multicast search message by clicking the screencast button on a mobile phone, the router will send the multicast search message to other devices in the local area network, and other devices will reply with a confirmation after receiving the multicast search message The received message (confirmation message) is sent to the mobile phone. If the mobile phone does not receive any confirmation message after waiting for T1 time, it is considered that the discovery has failed.
  • sending the second discovery request includes: in the case that the confirmation message of the multicast search message is not received, again Sending a first discovery request; sending a second discovery request when the number of times that the confirmation message of the multicast search message is not received exceeds the preset number of times.
  • the router will send the multicast search message to other devices in the LAN, and other devices will receive the multicast search message After that, it will reply with a confirmation message (confirmation message) to the mobile phone. This process usually takes T2.
  • the mobile phone waits for T2 and does not receive any confirmation message, it will send the multicast search message again and record the sending The number of multicast search messages. If the multicast search message is sent three times in a row and no confirmation message is received, it is considered that the discovery has failed.
  • Step 903 In a case where the target device is discovered through the second discovery request, obtain the IP address of the target device.
  • obtaining the IP address of the target device includes:
  • Step a1 Receive a reply message of the second discovery request, where the reply message includes the mac address and/or device name of the discovered device.
  • Step a2 Determine the mac address and/or device name of the target device based on the reply message.
  • Step a3 Send a P2P connection to the target device according to the mac address and/or device name of the target device.
  • Step a4 In the case where it is determined that the terminal device is the server and the target device is the client, receive the TCP connection request sent by the target device, and the TCP connection request carries the IP address of the target device.
  • Step a5 parse the TCP connection request to obtain the IP address of the target device.
  • obtaining the IP address of the target device includes: the terminal device initiates WIFI Direct connection request, the terminal device connected to the same WIFI network as the terminal device and the distance between the terminal device and the terminal device is less than the preset distance (hereinafter referred to as other terminal device) after receiving the WIFI direct connection request, reply its own device name and The MAC information is given to the terminal device; the terminal device initiates a connection establishment request to other terminal devices corresponding to the device name and MAC information according to the device name and MAC information. After receiving the connection establishment request, the other terminal device communicates with the terminal device that initiated the connection establishment request.
  • the content of the negotiation includes determining the client (group client) and the server (group owner). After that, the server creates a TCP server, and the client sends a TCP connection request to the server. After the server receives the TCP connection request, it parses The IP address of the client is output, and both the IP address of the client and the IP address of the server are returned to the client through TCP communication.
  • the mobile phone when the TV and the mobile phone are connected to the same WIFI network, the mobile phone initiates a WIFI direct connection request, and when the TV receives the WIFI direct connection request, it will reply the TV’s device name and MAC information to the mobile phone, and the mobile phone will respond according to the TV’s equipment
  • the name and MAC information initiates a connection establishment request to the TV.
  • the TV After the TV receives the connection establishment request, it negotiates with the mobile phone. After the negotiation result is that the TV is the client and the mobile phone is the server, the mobile phone creates a TCPserver and receives the TV to the mobile phone.
  • the mobile phone For the sent TCP connection request, after the mobile phone receives the TCP connection request, it parses out the TV's IP address from the TCP connection request, and then sends the TV's IP address and the mobile phone's IP address to the TV via TCP.
  • the method of this embodiment further includes:
  • Step a6 In the case where it is determined that the terminal device is the client and the target device is the server, the IP address of the target device is obtained.
  • both the TV and the mobile phone know the IP address of the TV, and both the mobile phone and the TV can directly obtain the IP address of the TV.
  • the method of this embodiment further includes: sending the IP address of the target device to the target device, so that the target device initializes the DLNA communication protocol according to the IP address of the target device .
  • the mobile phone when the mobile phone is the server and the TV is the client, the mobile phone sends the TV’s IP address to the TV, and the TV uses the TV’s own IP address to initialize the DLNA protocol so that it can subsequently establish a data transmission channel with the mobile phone in the form of unicast. .
  • the TV can directly obtain the TV's own IP address, and then initialize the DLNA protocol with its own IP address, so as to subsequently establish a data transmission channel with the mobile phone in the form of unicast.
  • Step 904 Establish a data transmission channel with the target device corresponding to the IP address through the DLNA communication protocol.
  • establishing a data transmission channel with the target device corresponding to the IP address through the DLNA communication protocol includes: sending a unicast message to the target device corresponding to the IP address; receiving the address information of the description file returned by the target device; The address information of the description file is obtained from the address corresponding to the address information of the description file.
  • the description file includes the description information of the target device; according to the description information, the target device is saved in the device list.
  • the mobile phone can initiate a DLNA connection to the TV through unicast to establish a data transmission channel. Specifically, the mobile phone sends a unicast message to the TV corresponding to the IP address. After the TV receives the unicast message, it returns the DMR description file address information, and then the mobile phone requests the TV to obtain the DMR description file according to the DMR description file address information. Reply the DMR description file to the mobile phone, and the mobile phone can obtain more detailed device description information, such as the Universally Unique Identifier (UUID) of the TV and the DLNA device type information. After that, the mobile phone will obtain all the information of the TV. Save to the device list. At this point, the data transmission channel is established.
  • UUID Universally Unique Identifier
  • Step 905 Transmit multimedia data to the target device through the data transmission channel, so as to display and/or play the multimedia data through the target device.
  • transmitting multimedia data to the target device through a data transmission channel so as to display and/or play the multimedia data through the target device includes: determining according to the received selection operation information of the multimedia data The multimedia data to be pushed; the push link is formed based on the multimedia data to be pushed; the push link is sent to the target device so that the target device parses the file address according to the push link, and obtains the corresponding file according to the file address, and the obtained file Send to the player for playback.
  • the user can select multimedia data such as pictures and videos on the mobile phone and click to push it to the TV.
  • multimedia data such as pictures and videos on the mobile phone
  • click to push it to the TV After the TV receives the pushed content, it parses out the file address, and then obtains the corresponding file according to the file address. Send the obtained file to the player for playback.
  • the first discovery request is sent, and the first discovery request uses the DLNA communication protocol; in the case of determining that the target device is not found according to the response result of the first discovery request, the second discovery request is sent, and the second discovery request adopts the end End-to-end communication protocol; when the target device is discovered through the second discovery request, obtain the IP address of the target device; establish a data transmission channel through the DLNA communication protocol and the target device corresponding to the IP address; transmit multimedia to the target device through the data transmission channel Data to display and/or play multimedia data through the target device.
  • the target device if the target device is not found, switch to the end-to-end communication protocol to discover the target device.
  • the end-to-end communication protocol is a decentralized direct communication method, no intermediate relay is required. Therefore, for the situation where the router packet loss leads to a low device discovery rate, or the router is set with AP isolation and other situations that cause the device to fail, it is also possible to bypass the router to discover the target device. Further, after obtaining the IP address of the target device through end-to-end communication, establish a data transmission channel with the target device through the DLNA protocol, and then transmit the multimedia data to be projected to the target device through the data transmission channel, and then pass the target device The player and/or display of the device perform display and/or playback, and therefore, DLNA projection can be realized.
  • FIG. 32 is a flowchart of a screen projection method provided by an example of an embodiment of this application. As shown in FIG. 312, the screen projection method provided by the example of this embodiment specifically includes the following steps:
  • Step 4001 After the TV is connected to the WIFI, it initializes the DLNA protocol, and sends a multicast join request to the multicast address, such as the address 239.255.255.250, to join the multicast.
  • Step 4002 After the mobile phone is connected to the WIFI, it sends a multicast search message to the multicast address.
  • the mobile phone sends a multicast search message to the address 239.255.255.250.
  • Step 4003 The mobile phone judges whether the confirmation message of the multicast search message is received.
  • the mobile phone receives the confirmation message of the multicast search message, it means that the TV has received the multicast search message and returned the confirmation message to the mobile phone. At the same time, the TV will also return the DMR description file address to the mobile phone.
  • Step 4004 When receiving the confirmation message of the multicast search message, the mobile phone sends a DMR description file acquisition request to the TV according to the DMR description file address.
  • Step 4005 After receiving the request for obtaining the DMR description file, the TV returns the DMR description file to the mobile phone.
  • the content of the DMR description file includes at least the device name, IP address, MAC information, UUID, and DLNA device type.
  • Step 4006 After receiving the DMR description file, the mobile phone parses the DMR description file, and saves the content of the parsed DMR description file to the device list.
  • Step 4007 The mobile phone sends a push link to the TV according to the push content selected by the user on the mobile phone.
  • Step 4008 After receiving the push link, the TV parses the push link, and calls the player to play the push content in the push link.
  • Step 4009 In the case that the mobile phone does not receive the confirmation message of the multicast search message, initiate a P2P discovery request.
  • step 202 For the implementation process of this step, please refer to the introduction of step 202 above, which will not be repeated here.
  • Step 4010 After receiving the P2P discovery request, the TV returns the MAC information and/or device name of the TV to the mobile phone.
  • Step 4011 the mobile phone saves the received MAC information and/or device name of the TV.
  • Step 4012 the mobile phone initiates a P2P connection request to the TV according to the MAC information and/or device name of the TV.
  • Step 4013 After the TV receives the P2P connection request, it negotiates with the mobile phone to determine the client and server.
  • Step 4014 In the case where the TV is the client and the mobile phone is the server, the TV sends a TCP connection request to the mobile phone.
  • the mobile phone when the TV is the client and the mobile phone is the server, the mobile phone creates a server thread tcpserver, the TV actively connects to the tcpserver, and then the mobile phone receives the TCP connection request from the TV.
  • Step 4015 After receiving the TCP connection request, the mobile phone parses out the IP address of the TV, and returns the IP address of the TV and the IP address of the mobile phone to the TV through TCP.
  • Step 4016 The TV initializes the DLNA protocol through the TV's own IP address.
  • the TV initializes the DLNA protocol through the TV's own IP address to prepare for the subsequent establishment of a connection between the mobile phone and the TV through unicast.
  • Step 4017 The mobile phone sends a unicast message to the TV IP address.
  • Step 4018 After receiving the unicast message, the TV returns the address information of the description file to the mobile phone.
  • Step 4019 The mobile phone sends a description file acquisition request to the TV's description file address according to the description file address information.
  • Step 4020 After receiving the description file acquisition request, the TV returns the description file to the mobile phone.
  • Step 4021 after receiving the description file, the mobile phone parses the description file to obtain the description file information of the TV, and saves the description file information of the TV to the device list.
  • the description file information of the TV includes at least UUID and DLNA device type.
  • Step 4022 the mobile phone selects the corresponding push content according to the received selection information of the push content, and sends the DLNA push link to the TV.
  • Step 4023 After receiving the DLNA push link, the TV parses the DLNA push link, and calls the player to play the push content.
  • the mobile phone can also be the client and the TV is the server.
  • both the TV and the mobile know the TV’s IP address.
  • the TV obtains the TV's own IP address, and directly uses the TV IP to initialize the DLNA protocol, and then the mobile phone directly sends a unicast message to the TV's IP address, and executes the above steps 4018-4023.
  • FIG. 33 is a schematic structural diagram of a screen projection device provided by an embodiment of the application.
  • the screen projection device may specifically be the terminal device in the foregoing embodiment.
  • the screen projection device provided by the embodiment of the present application can execute the processing flow provided in the embodiment of the screen projection method. As shown in FIG.
  • the screen projection device 50 includes: a first sending module 51, a second sending module 52, an acquisition module 53, and a connection Module 54 and push module 55; among them, the first sending module 51 is used to send a first discovery request, and the first discovery request adopts the DLNA communication protocol; the second sending module 52 is used to send data according to the first discovery request If it is determined that the target device is not found in the response result, a second discovery request is sent, the second discovery request adopts an end-to-end communication protocol; the acquisition module 53 is configured to discover the target device through the second discovery request In the case of obtaining the IP address of the target device; the connection module 54 is configured to establish a data transmission channel with the target device corresponding to the IP address through the DLNA communication protocol; the push module 55 is configured to transmit the data through the The channel transmits multimedia data to the target device, so as to display and/or play the multimedia data through the target device.
  • the first sending module 51 is used to send a first discovery request, and the first discovery request adopts the DLNA communication protocol
  • the obtaining module 53 when the obtaining module 53 obtains the IP address of the target device when the target device is discovered through the second discovery request, it specifically includes: receiving a reply to the second discovery request
  • the reply message includes the mac address and/or device name of the discovered device; the mac address and/or device name of the target device is determined based on the reply message; the mac address and/or device name of the target device is determined based on the reply message , Send a P2P connection to the target device; in the case where it is determined that the terminal device is a server and the target device is a client, receive a TCP connection request sent by the target device, and the TCP connection request carries something The IP address of the target device; parse the TCP connection request to obtain the IP address of the target device.
  • the obtaining module 53 is further configured to obtain its own IP address when it is determined that the terminal device is a client and the target device is a server.
  • connection module 54 when the connection module 54 establishes a data transmission channel with the target device corresponding to the IP address through the DLNA communication protocol, it specifically includes: sending a unicast message to the target device corresponding to the IP address; The address information of the description file returned by the target device; according to the address information of the description file, the description file is obtained from the address corresponding to the address information of the description file, and the description file includes the description information of the target device ; According to the description information, save the target device to the device list.
  • sending the second discovery request when it sends the first discovery request, it specifically includes: sending a multicast search message; the second sending module 52 is in accordance with the response result of the first discovery request When it is determined that the target device is not found, sending the second discovery request specifically includes: sending the second discovery request without receiving the confirmation message of the multicast search message within a preset time period.
  • the first sending module 51 when the first sending module 51 sends the first discovery request, it specifically includes: sending a multicast search message; the second sending module 52 is in accordance with the response result of the first discovery request When it is determined that the target device is not found, sending the second discovery request specifically includes: sending the first discovery request again when the confirmation message of the multicast search message is not received; In the case that the number of confirmation messages of the multicast search message exceeds the preset number of times, the second discovery request is sent.
  • the second sending module 52 is further configured to send the IP address of the target device to the target device, so that the target device can communicate with the DLNA according to the IP address of the target device.
  • the protocol is initialized.
  • the screen projection device of the embodiment shown in FIG. 33 can be used to implement the technical solutions of the foregoing method embodiments, and its implementation principles and technical effects are similar, and will not be repeated here.
  • FIG. 34 is a schematic structural diagram of a screen projection device provided by an embodiment of the application.
  • the screen projection device may specifically be the terminal device in the foregoing embodiment.
  • the screen projection device provided by the embodiment of the present application can execute the processing flow provided in the embodiment of the screen projection method. As shown in FIG.
  • the screen projection device 60 includes: a memory 61, a processor 62, a computer program, and a communication interface 63;
  • the program is stored in the memory 61 and is configured to be performed by the processor 62 to perform the following operations: send a first discovery request, the first discovery request adopts the DLNA communication protocol; In the case of the target device, send a second discovery request, the second discovery request adopts an end-to-end communication protocol; in the case of discovering the target device through the second discovery request, obtain the IP address of the target device Establish a data transmission channel through the DLNA communication protocol and the target device corresponding to the IP address; transmit multimedia data to the target device through the data transmission channel, so as to display and display the multimedia data through the target device / Or play.
  • the processor 62 when the processor 62 obtains the IP address of the target device when the target device is discovered through the second discovery request, it specifically includes: receiving the second discovery request
  • the reply message includes the mac address and/or device name of the discovered device; the mac address and/or device name of the target device is determined based on the reply message; the mac address and/or device name of the target device is determined based on the reply message Device name, send a P2P connection to the target device; in the case where it is determined that the terminal device is a server and the target device is a client, receive a TCP connection request sent by the target device, and the TCP connection request carries There is an IP address of the target device; the TCP connection request is parsed to obtain the IP address of the target device.
  • the processor 62 is further configured to: when it is determined that the terminal device is a client and the target device is a server, the target device obtains its own IP address.
  • the processor 62 establishing a data transmission channel with the target device corresponding to the IP address through the DLNA communication protocol includes: sending a unicast message to the target device corresponding to the IP address; receiving The address information of the description file returned by the target device; according to the address information of the description file, the description file is obtained from an address corresponding to the address information of the description file, the description file including the description of the target device Information; according to the description information, save the target device to the device list.
  • sending the second discovery request specifically includes: sending the second discovery request without receiving the confirmation message of the multicast search message within a preset time period.
  • the processor 62 when the processor 62 sends the first discovery request, it specifically includes: sending a multicast search message; the processor 62 determines that no target is found according to the response result of the first discovery request.
  • the processor 62 when sending the second discovery request, it specifically includes: sending the first discovery request again when the confirmation message of the multicast search message is not received; and when the multicast search message is not received If the number of confirmation messages of the message exceeds the preset number of times, the second discovery request is sent.
  • the processor 62 is further configured to: send the IP address of the target device to the target device, so that the target device communicates the DLNA according to the IP address of the target device
  • the protocol is initialized.
  • the screen projection device of the embodiment shown in FIG. 34 can be used to execute the technical solutions of the foregoing method embodiments, and its implementation principles and technical effects are similar, and will not be repeated here.
  • an embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored, and the computer program is executed by a processor to implement the screen projection method described in the foregoing embodiment.
  • a storage medium including instructions such as a memory including instructions, which can be executed by a processor of a multimedia device to complete the above-mentioned projection and playback method.
  • the storage medium may be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage Equipment, etc.
  • the embodiments of the present application also provide a computer program product, which when the computer program product runs on an electronic device, causes the electronic device to execute the method for implementing any one of the above-mentioned projections in the embodiments of the present application.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

本申请是关于一种多媒体设备及投屏播放方法，包括：在与多个终端同时进行投屏过程中，从多个终端中确定目标终端；将目标终端发送的音频数据存放在音频缓存区中，以及丢弃其他终端发送的音频数据；并将多个终端发送视频数据存放在各终端对应的视频缓存区中；播放音频缓存区内的音频数据，以及将至少一个视频缓存区内的视频数据进行显示。

Description

多媒体设备及投屏播放方法
本申请要求在2020年3月20日提交中国专利局、申请号为202010199814.8申请名称为“投屏方法、装置、设备及计算机可读存储介质”的优先权；其全部内容通过引用结合在本申请中；本申请要求在2020年6月9日提交中国专利局、申请号为202010519510.5申请名称为“多媒体设备及投屏播放方法”的优先权；其全部内容通过引用结合在本申请中；本申请要求在2020年6月10日提交中国专利局、申请号为202010522106.3申请名称为“显示设备以及与显示设备建立连接的控制方法”的优先权；其全部内容通过引用结合在本申请中。
技术领域
本申请涉及智能终端技术领域，尤其涉及一种多媒体设备及投屏播放方法。
背景技术
在支持多路镜像投屏，即可以支持同时把多个终端的屏幕画面镜像到多媒体设备上，当多个终端连接多媒体设备时，多个终端均会把自己的音频数据发送给多媒体设备。
当多媒体设备接收到的多个终端的音频数据，多媒体设备申请对应的缓存buffer(缓存区)存放对应的音频数据，即多媒体设备连接几个终端，多媒体设备申请几个缓存区，当多媒体设备播放声音时，从选择的对应的音频存放缓存区中读取数据送到音频播放接口，每个音频数据对应创建音频语音单元，同时，将其中一路对应的音频语音单元不设置静音状态，正常播放声音，其它几路对应的音频语音单元声音设置为静音。
发明内容
本申请提供一种多媒体设备及投屏播放方法，
第一方面，本申请实施例提供的一种电视，包括：处理器、显示屏以及语音单元；
所述显示屏，用于显示至少一个视频缓存区内的视频数据；
所述语音单元，用于播放所述音频缓存区内的音频数据；
所述处理器，用于在与多个手机同时进行投屏过程中，从所述多个手机中确定目标手机；
将所述目标手机发送的音频数据存放在音频缓存区中，以及丢弃其他手机发送的音频数据；并将多个手机发送视频数据存放在各手机对应的视频缓存区中。
第二方面，本申请实施例提供的一种投屏播放方法，应用于电视，所述方法包括：
在与多个手机同时进行投屏过程中，从所述多个手机中确定目标手机；
将所述目标手机发送的音频数据存放在音频缓存区中，以及丢弃其他手机发送的音频数据；并将多个手机发送视频数据存放在各手机对应的视频缓存区中；
播放所述音频缓存区内的音频数据，以及将至少一个视频缓存区内的视频数据在显示屏上显示。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本申请的实施例， 并与说明书一起用于解释本申请的原理，并不构成对本申请的不当限定。
图1是背景技术中的一种多媒体设备在投屏过程中的内存的示意图；
图2是本申请实施例提供的一种多媒体设备在投屏过程中的内存的示意图；
图3是本申请实施例提供的一种电视与手机在投屏过程中的连接示意图；
图4是本申请实施例提供的一种投影仪与电脑在投屏过程中的连接示意图；
图5是本申请实施例提供的一种多媒体设备的结构示意图；
图6是本申请实施例提供的一种投屏播放方法的流程图；
图7是本申请实施例提供的一种在电视上投屏播放的方法流程图；
图8是本申请实施例提供的一种在投影仪上投屏播放的方法流程图；
图9是本申请实施例提供的一种用户选择目标终端时用户与多媒体设备交互的示意图；
图10是本申请实施例提供的一种控制电视的遥控器的示意图；
图11是本申请实施例提供的另一种用户选择目标终端时用户与多媒体设备交互的示意图；
图12是本申请实施例提供的一种用户选择目标终端的投屏播放方法的工作流程图；
图13是本申请实施例提供的一种终端从多个终端中随机选择一个终端作为目标终端的投屏播放方法的工作流程图；
图14是本申请实施例提供的一种终端从多个终端中选择第一个连接的终端作为目标终端的投屏播放方法的工作流程图；
图15是本申请实施例提供的一种终端从多个终端中选择最后一个连接的终端作为目标终端的投屏播放方法的工作流程图；
图16是本申请实施例提供的一种切换目标终端的投屏播放方法的工作流程图；
图17中为本申请实施例提供的显示设备与控制装置之间操作场景的示意图；
图18中为本申请实施例提供的实现手机与智能电视建立连接的处理过程示意图；
图19中为本申请实施例提供的向用户提供不信任、信任并连接、屏蔽选择按钮的用户界面示意图；
图20中为本申请实施例提供的若电视端的黑、白名单为空，则呈现给用户的黑、白名单的管理界面示意图；
图21中为本申请实施例提供的若用户需要添加常用设备，则呈现给用户的黑、白名单的管理界面示意图；
图22中为本申请实施例提供的若用户需要移除常用设备，则呈现给用户的黑、白名单的管理界面示意图；
图23中为本申请实施例提供的在常用设备中移除设备的标识时呈现给用户的提示界面示意图；
图24中为本申请实施例提供的若用户需要移除屏蔽设备，则呈现给用户的黑、白名单的管理界面示意图；
图25中为本申请实施例提供的在屏蔽设备中移除设备的标识时呈现给用户的提示界面示意图；
图26中为本申请实施例提供的用户也可以添加屏蔽设备，则呈现给用户的黑、白名单的管理界面示意图；
图27为本申请实施例提供的一种显示设备的结构示意图；
图28为本申请实施例提供的一种与显示设备建立连接的控制方法的流程示意图；
图29为本申请实施例提供的投屏系统的结构示意图；
图30为本申请实施例提供的投屏方法流程图；
图31为本申请实施例提供的端到端发现设备的示意图；
图32为本申请实施例提供的投屏方法的信令图；
图33为本申请实施例提供的投屏装置的结构示意图；
图34为本申请实施例提供的投屏设备的结构示意图。
具体实施方式
为了使本领域普通人员更好地理解本申请的技术方案，下面将结合附图，对本申请实施例中的技术方案进行清楚、完整地描述。
目前，在进行投屏过程中，以四个终端连接多媒体设备为例，每个终端发送给多媒体设备时，多媒体设备均会申请一个音频缓存区和一个视频缓存区，结合图1所示，四个终端会在多媒体设备的内存中申请8个缓存区，四个音频缓存区和四个视频缓存区，在每个音频缓存区中存储对应的音频数据，每个视频缓存区中存储对应的视频数据。
基于此，本申请实施例提供一种电视、多媒体设备以及投屏播放方法，通过选择一个目标终端，将目标终端的音频数据存储在音频缓存区中，其他终端的音频数据均丢弃掉，这样做在保证不影响正常播放的情况下，节省了多媒体设备的内存。
以四个终端连接多媒体设备为例，结合图2所示，本申请实施例中的多媒体设备的内存中，包括一个音频缓存区，四个视频缓存区，一个音频缓存区内存储目标终端的音频数据，四个视频缓存区中存储对应的四个终端的视频数据。
综合图1和图2所示，本申请提供的方式，特别是与多媒体设备连接比较多的终端进行投屏播放时，能够大大的节省多媒体设备的内存。
以下结合附图对多媒体设备进行详细介绍。
应该理解的是，图3所示多媒体设备300仅是一个范例，并且多媒体设备300可以具有比图3中所示的更多的或者更少的部件，可以组合两个或多个的部件，或者可以具有不同的部件配置。图中所示出的各种部件可以在包括一个或多个信号处理和/或专用集成电路在内的硬件、软件、或硬件和软件的组合中实现。
图3中示例性示出了根据示例性实施例中多媒体设备300的硬件配置框图。如图3所示，多媒体设备300包括：射频(radio frequency，RF)电路310、存储器320、显示单元330、语音单元340、无线保真(Wireless Fidelity，Wi-Fi)模块350、处理器360、蓝牙模块370、以及电源380等部件。
其中，处理器360是多媒体设备300的控制中心，利用各种接口和线路连接整个终端的各个部分，通过运行或执行存储在存储器320内的软件程序，以及调用存储在存储器320内的数据，执行多媒体设备300的各种功能和处理数据。在一些实施例中，处理器360可包括一个或多个处理单元；处理器360还可以集成应用处理器和基带处理器，其中，应用处理器主要处理操作系统、用户界面和应用程序等，基带处理器主要处理无线通信。可以理解的是，上述基带处理器也可以不集成到处理器360中。本申请中处理器360可以运行操作系统、应用程序、用户界面显示及触控响应，以及本申请实施例所述的投屏播放方法。 另外，处理器360与显示单元330耦接。
其中，本申请中提到的显示单元330针对不同的多媒体设备，显示单元330的结构也不同。
以多媒体设备为电视为例，显示单元330可用于接收输入的数字或字符信息，产生与多媒体设备300的用户设置以及功能控制有关的信号输入，具体地，显示单元330可以包括设置在多媒体设备300正面的触摸屏332，可收集用户在其上或附近的触摸操作，例如点击按钮，拖动滚动框等。图3中示出了以多媒体设备为电视为例的显示单元330的结构。
显示单元330还可用于显示由用户输入的信息或提供给用户的信息以及多媒体设备300的各种菜单的图形用户界面(graphical user interface，GUI)。具体地，显示单元330可以包括设置在多媒体设备300正面的显示屏331。其中，显示屏331可以采用液晶显示器、发光二极管、电子墨水等形式来配置。显示单元330可以用于显示本申请中所述的各种图形用户界面。其中，采用电子墨水形成的显示屏通常称为水墨屏。
其中，触摸屏332可以覆盖在显示屏331之上，也可以将触摸屏332与显示屏331集成而实现多媒体设备300的输入和输出功能，集成后可以简称触摸显示屏，其中利用电子墨水制作的屏幕为水墨屏。本申请中显示单元330可以显示应用程序以及对应的操作步骤。
以多媒体设备为投影仪为例，显示单元330为成像系统。成像系统包括光源、图像显示元件、镜头。在进行显示时，通过光源照射图像显示元件产生影像，然后通过镜头将视频数据投影到墙上或者投影布上。
其中，语音单元340包括：扬声器341、麦克风342可提供用户与多媒体设备300之间的音频接口。语音单元340可将接收到的音频数据转换后的电信号，传输到扬声器341，由扬声器341转换为声音信号输出。多媒体设备300还可配置音量按钮，用于调节声音信号的音量。另一方面，麦克风342将收集的声音信号转换为电信号，由语音单元340接收后转换为音频数据，再将音频数据输出至存储器320以便进一步处理，例如通过语音控制多媒体设备。本申请中麦克风342可以获取用户的语音。
存储器320可用于存储软件程序及数据。处理器360通过运行存储在存储器320的软件程序或数据，从而执行多媒体设备300的各种功能以及数据处理。存储器320可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他易失性固态存储器件。存储器320存储有使得多媒体设备300能运行的操作系统。本申请中存储器320可以存储操作系统及各种应用程序，还可以存储执行本申请实施例所述投屏播放方法的代码。
RF电路310可用于在收发信息或通话过程中信号的接收和发送，可以接收基站的下行数据后交给处理器360处理；可以将上行数据发送给基站。通常，RF电路包括但不限于天线、至少一个放大器、收发信机、耦合器、低噪声放大器、双工器等器件。
Wi-Fi属于短距离无线传输技术，多媒体设备300可以通过Wi-Fi模块350帮助用户收发电子邮件、浏览网页和访问流媒体等，Wi-Fi模块350为用户提供了无线的宽带互联网访问。
蓝牙模块370，用于通过蓝牙协议来与其他具有蓝牙模块的蓝牙设备进行信息交互。例如，多媒体设备300可以通过蓝牙模块370与同样具备蓝牙模块的可穿戴电子设备(例如智能手表)建立蓝牙连接，从而进行数据交互。
多媒体设备300还包括给各个部件供电的电源380(比如电池)。电源可以通过电源管 理系统与处理器360逻辑相连，从而通过电源管理系统实现管理充电、放电以及功耗等功能。多媒体设备300还可配置有电源按钮，用于终端的开机和关机，以及锁屏等功能。
在本申请中，上述介绍的元件可以具有以下功能：
所述处理器360，用于在与多个终端同时进行投屏过程中，从所述多个终端中确定目标终端；将所述目标终端发送的音频数据存放在音频缓存区中，以及丢弃其他终端发送的音频数据；并将多个终端发送视频数据存放在各终端对应的视频缓存区中。
所述显示单元330，用于显示至少一个视频缓存区内的视频数据。
所述语音单元340，用于播放所述音频缓存区内的音频数据。
其中，多媒体设备以电视、终端以手机为例。
结合图4所示，以4个手机连接电脑为例，手机41、手机42、手机43、手机44，上述四个手机均与电视400进行连接，电视400从连接的4个手机中选择一个目标手机，例如，选择目标手机为手机42，电视将手机42发送的音频数据存储在音频存储区中，以及丢弃手机41、手机43和手机44发送的音频数据，电视将4个手机发送的视频数据存放在各手机对应的视频存储区，手机41发送视频411，手机42发送视频422，手机43发送视频433，手机44发送视频444，电视400从音频存储区中读取数据进行播放手机42的音频，从视频存储中去读数据在显示屏上显示至少一个手机的视频数据，例如显示视频411和视频422。
其中，多媒体设备以投影仪、终端以电脑为例。
结合图5所示，以四个电脑为例，电脑51、电脑52、电脑53、电脑54，上述4个电脑均与投影仪500进行连接，投影仪500从连接的四个电脑中选择一个目标电脑，例如，选择目标电脑为电脑51，投影仪500将电脑51发送的音频数据存储在音频存储区中，以及丢弃电脑52、电脑53以及电脑54发送的音频数据，投影仪500将四个电脑发送的视频数据存放在各电脑对应的视频存储区，电脑51发送视频511，电脑52发送视频522，电脑53发送视频533，电脑54发送视频544，投影仪500从音频存储区中读取数据进行播放电脑51的音频，从视频存储中去读数据显示至少一个手机的视频数据，例如显示视频511和视频522。
对于确定目标终端，所述处理器360具有以下功能：
所述处理器360，具体用于：将用户从所述多个终端中选择的终端作为目标终端；或
从所述多个终端中选择一个终端作为目标终端；或
从所述多个终端中选择第N个连接的终端作为目标终端，其中N为正整数。
对于将用户从所述多个终端中选择的终端作为目标终端的方式，需要显示单元330与处理器360交互实现，详细来讲，所述显示单元330还用于接收用户在所述多媒体设备上操作的选择视频指令，所述处理器360，具体用于：响应用户在所述多媒体设备上操作的选择视频指令，将所述选择视频指令所选择的视频对应的终端作为目标终端。
对于多个终端与多媒体设备300断开时，处理器360具有以下功能：
所述处理器360，还用于：在与多个终端同时进行投屏过程中，检测所述多个终端的连接状态；若检测到所述目标终端断开连接，则删除所述音频缓存区的音频数据以及释放所述目标终端对应的视频缓存区的内存空间，并从剩余的终端中确定新的目标终端；将新的目标终端发送的音频数据存放在所述音频缓存区中。
所述处理器360，还用于：若检测到所有终端均断开连接，则释放所述音频缓存区的 内存空间。
以下结合附图对本申请实施例提供的一种投屏播放方法进行详细介绍。如图6所示，该方法具体包括以下步骤：
S600：在与多个终端同时进行投屏过程中，从多个终端中确定目标终端。
S601：将目标终端发送的音频数据存放在音频缓存区中，以及丢弃其他终端发送的音频数据；并将多个终端发送视频数据存放在各终端对应的视频缓存区中。
S602：播放音频缓存区内的音频数据，以及将至少一个视频缓存区内的视频数据进行显示。
其中，播放音频存储区内的音频数据时，读取音频缓存区内的音频数据，进行播放，同时读取至少一个视频缓存区内的视频数据，进行显示。这样在保证与多媒体设备连接的多个终端的视频进行正常投屏，且播放一路音频数据的情况下，节省了多媒体设备的内存。
需要说明的是，上述步骤执行的过程仅为示例性的，不能作为本申请的具体限制，可以根据具体情况进行调整。
由上可知，多媒体设备可以为电视和投影仪等。
当多媒体设备为电视、终端为手机时，结合图7所示，具体投屏播放过程为：
S700：在与多个手机同时进行投屏过程中，从多个手机中确定目标手机；
S701：将目标手机发送的音频数据存放在音频缓存区中，以及丢弃其他手机发送的音频数据；并将多个手机发送视频数据存放在各手机对应的视频缓存区中；
S702：播放音频缓存区内的音频数据，以及将至少一个视频缓存区内的视频数据在显示屏上显示。
当多媒体设备为投影仪、终端为电脑时，结合图8所示，具体投屏播放过程为：
S800：在与多个电脑同时进行投屏过程中，从多个电脑中确定目标电脑；
S801：将目标电脑发送的音频数据存放在音频缓存区中，以及丢弃其他电脑发送的音频数据；并将多个电脑发送视频数据存放在各电脑对应的视频缓存区中；
S802：播放音频缓存区内的音频数据，以及将至少一个视频缓存区内的视频数据进行投屏显示。
对于从多个终端中确定目标终端的方式可以包括以下：
方式1：将用户从多个终端中选择的终端作为目标终端。
该方式提供了一种用户自定义的方式，提供通过用户自主选择播放出声音的终端。具体来说，多媒体设备响应用户在多媒体设备上操作的选择视频指令，将选择视频指令所选择的视频对应的终端作为目标终端。
以四个终端连接多媒体设备为例，在未选择目标终端时，显示全部的视频，即四个终端在连接到多媒体设备时，会触发多媒体设备自动在内存中申请四个视频缓存区，每个视频缓存区存储对应存储四个终端的视频数据，则多媒体设备会读取四个视频缓存区内的视频数据进行显示。由于多媒体设备的显示屏幕的尺寸有限，所以，四个视频可能显示在不同的界面中，以一个界面显示两个视频为例，则显示四个视频，需要两个页面进行显示。其中，视频1为终端1发送的视频数据形成的视频，视频2为终端2发送的视频数据形成的视频，视频3为终端3发送的视频数据形成的视频，视频4为终端4发送的视频数据形成的视频。
在默认的情况下，按照连接到多媒体设备的顺序进行显示，在第一页面中显示第一个 连接到多媒体设备的终端的视频和第二个连接到多媒体设备的终端的视频，例如，结合图9所示，视频1和视频2，同时，在页面中还包括“下一页”的按钮，当用户点击“下一页”时，可以显示视频3和视频4。
当用户点击选择的视频时，例如选择视频3，多媒体设备响应用户在多媒体设备上操作的选择视频指令，将选择的视频3的播放区域突出，同时将选择视频指令所选择的视频3对应的终端3作为目标终端，选择目标终端后，多媒体设备存储终端3的音频数据，多媒体设备读取终端3的音频数据进行播放。
当然，对于本申请来说，用户还可以通过遥控器选择目标终端。结合图10所示，遥控器中包含环形控制按钮，环形控制按钮包围的中间的圆形为确定按钮，环形控制按钮包含上、下、左、右，四个不同的方向的按钮，上的按钮设置在环形的上方，采用“+”表示，下的按钮设置在环形的下方，采用“-”表示，左的按钮设置在环形的左方，采用“-”表示，右的按钮设置在环形的右方，采用“+”表示。
左的按钮、右的按钮为翻页功能，例如，用户按右的按钮为下一页，左的按钮为返回上一页，上的按钮为上一个视频，下的按钮为下一个视频，当用户在选择时，可以通过操控遥控器的左的按钮、右的按钮翻页查看连接到多媒体设备的终端中的视频，用户通过上的按钮、下的按钮选择视频，终端响应用户选择的视频，将视频对应的终端作为目标终端。
或者，结合图11所示，用户通过操控遥控器的左的按钮、右的按钮翻页查看连接到多媒体设备的终端中的视频，选择完成后，用户通过上的按钮、下的按钮选择视频后，例如，选择视频3，多媒体设备在接收到用户通过上的按钮或下的按钮选择视频后，将选择的视频3的播放区域突出，然后用户通过点击确定按钮后，将视频3对应的终端3作为目标终端，选择目标终端后，多媒体设备存储终端3的音频数据，多媒体设备读取终端3的音频数据进行播放。
需要说明的是，对于附图中多媒体的样式只是举例说明，本申请并不限定该样式，可以没有实体按键，或者实体按键数量可以根据需要设置，按键排布也可以根据需要设置在不同位置。
结合图12所示，用户自主选择目标终端的投票播放方法，包括：
S1200：在与多个终端同时进行投屏过程中，向多媒体设备的内存申请内存空间作为与多个终端对应的视频缓存区；
S1210：将多个终端发送的视频数据存放在各终端对应的视频缓存区中，同时丢弃多个终端发送的音频数据；
S1220：显示多个视频缓存区内的视频数据；
S1230：响应用户在多媒体设备上操作的选择视频指令，将选择视频指令所选择的视频对应的终端作为目标终端；
S1240：向多媒体设备的内存申请内存空间作为音频缓存区；
S1250：将目标终端发送的音频数据存放在音频缓存区中，以及丢弃其他终端发送的音频数据；
S1260：播放音频缓存区内的音频数据。
上述方式中，由于需要用户根据多媒体设备上显示的视频选择目标终端，所以，在用户未选择目标终端时，处于节省多媒体设备的内存考虑，不会申请音频缓存区，同时将与多媒体设备连接的所有终端发送的音频数据全部丢弃，在用户选择目标终端后，触发自动 申请音频缓存区的流程，向内存申请音频缓存区的内存空间，从而能够满足音频缓存去内只存储目标终端的目的，达到节省多媒体设备的内存。
方式2：从多个终端中选择一个终端作为目标终端。
该方式提供了一种自动选择方式，具体来说，多媒体设备可以从多个终端中随机选择一个终端作为目标终端。即当多媒体设备连接四个终端时，可以从四个终端中随机选择一个终端作为目标终端，由于多个终端与多媒体设备连接时，就会发送音频数据和视频数据给多媒体设备，所以，多媒体设备需要在确定所有的终端均连接完成后，才能选择目标终端，基于此，多媒体设备自动选择目标终端的投屏播放方法，结合图13所示，包括：
S1300：在检测到第一个终端连接到多媒体设备开始计时；
S1310：丢弃连接到多媒体设备的终端发送的音频数据，向多媒体设备的内存申请内存空间作为连接到多媒体设备的终端对应的视频缓存区，存储连接到多媒体设备的终端发送的视频数据；
S1320：显示连接到多媒体设备的终端对应的视频缓存区内的视频数据；
S1330：判断检测的持续时间是否达到预设时间，如果是，则执行S1340，否则，执行S1310；
S1340：从多个与多媒体设备连接的终端中随机选择一个终端作为目标终端；
S1350：向多媒体设备的内存申请内存空间作为音频缓存区；
S1360：将目标终端发送的音频数据存放在音频缓存区中，以及丢弃其他终端发送的音频数据；
S1370：播放音频缓存区内的音频数据。
另外，多媒体设备可以从多个终端中选择预设视频类型对应的终端作为目标终端。
详细来说，在检测到终端连接到多媒体设备时，多媒体设备申请终端对应的视频缓存区，存储终端发送的视频数据，检测视频中的视频类型，若视频类型满足预设视频类型时，则确定该终端为目标终端，多媒体设备在内存中申请音频缓存区，将目标终端发送的音频数据存储在音频缓存区内，同时对之后连接到多媒体设备的视频不进行判断是否满足预设的视频类，直接申请终端对应的视频缓存区，存储终端发送的视频数据。如果不满足预设视频类型时，则多媒体设备丢弃终端发送的音频数据，继续检测下一个连接到多媒体设备的终端的视频类型是否满足预设的视频类型，直到找到后，申请音频缓存区，将目标终端发送的音频数据存储在音频缓存区内，同时对之后连接到多媒体设备的视频不进行判断是否满足预设的视频类，直接申请终端对应的视频缓存区，存储终端发送的视频数据。
方式3：从多个终端中选择第N个连接的终端作为目标终端，其中N为正整数。
由于连接到多媒体设备的终端为一个一个的进行连接的，本申请提出了上述方式。其中，第N个连接的终端可以为第一个，或者最后一个。即从多个终端中选择第一个连接的终端作为目标终端，或者从多个终端中选择最后一个连接的终端作为目标终端。
具体来说，多媒体设备自动选择第一个连接到多媒体设备的终端作为目标终端的投屏播放方法，结合图14所示，包括：
S1400：在第一个终端连接多媒体设备后，向多媒体设备的内存申请两个内存空间分别作为音频缓存区以及视频缓存区；
S1410：将第一个终端发送的音频数据存放在音频缓存区中，将第一个终端发送视频数据存放在对应的视频缓存区中；
S1420：针对其他连接到多媒体设备的终端，向多媒体设备的内存申请内存空间作为对应的视频缓存区；
S1430：将其他终端发送视频数据存放在对应的视频缓存区中；
S1440：丢弃其他终端发送的音频数据。
由于本申请中将第一个连接到多媒体设备的终端作为目标终端，所以，将第一个终端发送的音频数据存放在音频缓存区中，将第一个终端发送视频数据存放在对应的视频缓存区中，将除去第一个终端之外的其他连接到多媒体设备的终端来说，只存储视频数据，将发送的音频数据丢弃。
多媒体设备自动选择最后一个连接到多媒体设备的终端作为目标终端的投屏播放方法，以连接三个终端到多媒体设备为例，结合图15所示，包括：
S1500：在第一个终端连接多媒体设备后，向多媒体设备的内存申请两个内存空间分别作为音频缓存区以及视频缓存区；
S1510：将第一个终端发送的音频数据存放在音频缓存区中，将第一个终端发送视频数据存放在对应的视频缓存区中；
S1520：在第二个终端连接多媒体设备后，将音频缓存区内的第一终端发送的音频数据删除，将第二个终端发送的音频数据存储在音频缓存区内；
S1530：向多媒体设备的内存申请内存空间作为视频缓存区；
S1540：将第二个终端发送视频数据存放在对应的视频缓存区中；
S1550：在第三个终端连接多媒体设备后，将音频缓存区内的第二终端发送的音频数据删除，将第三个终端发送的音频数据存储在音频缓存区内；
S1560：向多媒体设备的内存申请内存空间作为视频缓存区；
S1570：将第三个终端发送视频数据存放在对应的视频缓存区中。
对于上述方法中，由于本申请规定最后一个连接到多媒体设备的终端为目标终端，所以在第一个终端连接到多媒体设备后，由于多媒体设备中只有一个终端所以，该终端即是第一连接的终端也是最后一个连接的终端，所以，本申请将申请的音频缓存区内存储第一个终端的音频数据，当第二个终端连接到多媒体设备时，多媒体设备中包括两个终端，第二个终端为最后一个连接到多媒体设备的终端，所以需要在音频缓存区存储第二个终端发送的音频数据，同时删除第一个终端存储在音频缓存区内的音频数据，在将第二个终端作为目标终端时，第一个终端发送的音频数据丢弃，第一个终端发送的视频数据可以继续存储对应的视频缓存区。同样的，当第三个终端连接到多媒体设备时，多媒体设备中包括三个终端，第三个终端为最后一个连接到多媒体设备的终端，所以需要在音频缓存区存储第三个终端发送的音频数据，同时删除第二个终端存储在音频缓存区内的音频数据，在将第三个终端作为目标终端时，第一个终端和第二个终端发送的音频数据丢弃，第一个终端发送的视频数据可以继续存储对应的视频缓存区，第二个终端发送的视频数据可以继续存储对应的视频缓存区。
针对上述提供的方式来说，可以组合使用，例如，在用户想要查看到所有连接到多媒体设备的终端的视频内容后，再进行选择的情况下，可以采用多媒体设备首先自动选择一个终端作为目标终端，再将目标终端切换至用户选择的终端，具体过程，结合图16所示：
S1600：在与多个终端同时进行投屏过程中，从多个终端中选择第N个连接的终端作为目标终端；
S1610：将目标终端发送的音频数据存放在音频缓存区中，以及丢弃其他终端发送的音频数据；并将多个终端发送视频数据存放在各终端对应的视频缓存区中；
S1620：播放音频缓存区内的音频数据，以及将至少一个视频缓存区内的视频数据进行显示；
S1630：若检测到用户在多媒体设备上操作的选择视频指令，则将目标终端切换为选择视频指令所选择的视频对应的终端；
S1640：删除音频缓存区的音频数据以及将选择视频指令所选择的视频对应的终端发送的音频数据存储在音频缓存区内；
S1650：播放选择视频指令所选择的视频对应的终端发送的音频数据。
若终端与多媒体设备断开连接时，多媒体设备针对终端存储过的数据以及分配给终端存储数据的内存空间，如果不释放，该内存空间依然不属于可用的内存空间，出于合理利用多媒体设备的内存空间的考虑，本申请实施例还提供了一种在终端与多媒体设备断开连接后的处理方法，包括：
在与多个终端同时进行投屏过程中，检测多个终端的连接状态；
若检测到目标终端断开连接，则删除音频缓存区的音频数据以及释放目标终端对应的视频缓存区的内存空间，并从剩余的终端中确定新的目标终端；
将新的目标终端发送的音频数据存放在音频缓存区中。
其中，释放内存空间完成后，该释放的内存空间为可用空间。本申请由于目标终端断开连接时，释放对应的视频缓存区的内存空间，可以使得除去目标终端发送的视频数据之外的其他数据使用该视频缓存区的内存空间，从而提高了内存的利用率。
同时，如果音频缓存区内的目标终端不删除，虽然目标终端没有连接多媒体设备进行投屏，但多媒体设备会一直播放存储在音频缓存区内的音频，会造成投屏的视频与播放的音频不符合的情况，所以，本申请需要从剩余的终端中重新确定新的目标终端，将该新的目标终端发送的音频数据由丢弃的操作，变成存储到音频缓存区的操作。
其中，当检测到非目标终端断开连接时，则释放非目标终端对应的视频缓存区的内存空间。这样使得除去非目标终端发送的视频数据之外的其他数据使用该视频缓存区的内存空间，从而提高了内存的利用率。
由于非目标终端断开连接，音频缓存区内依然存储目标终端的音频数据，所以，对音频缓存区无影响，仅需要释放非目标终端对应的视频缓存区的内存空间即可。
同时，当检测到所有终端均断开连接，则释放音频缓存区的内存空间。
由于断开所有终端时，音频缓存区没有用途，所以，出于提高内存的利用率考虑，将释放音频缓存区的内存空间，使得其他数据可以使用该音频缓存区的内存空间。
本申请实施例提供了一种显示设备以及与显示设备建立连接的控制方法，例如可以适用于智能终端(例如手机、电脑、平板电脑等终端设备)与电视设备(例如智能电视机)之间多屏互动技术领域。
本申请实施例提供的技术方案，在电视上保存与其建立连接的智能终端的白名单以及黑名单，在白名单中的设备可以直接允许其与电视端进行连接并响应其操控或者直接显示投屏内容；在黑名单中的设备则直接屏蔽在连接阶段，用户将无感知。对于不在白名单和 黑名单中的设备，即新请求与电视端建立连接的设备，连接时会弹出对话框让用户选择是允许、拒绝还是永远拒绝。另外，用户也可以手动将白名单和黑名单中的设备移除或进行其他方面的编辑等。
通过本申请实施例提供的技术方案，在电视上将通过所有连接方式的终端设备的标识分类保存，分为白名单，黑名单，并可以进行管理，只有在白名单中的手机设备允许其进行连接并响应其操控指令，黑名单中的设备则直接屏蔽在连接阶段，用户将无感知。如果不在白名单和黑名单中的设备，每次连接时都会弹出对话框让用户选择是屏蔽、不信任、信任并连接，从而，可以大大提高多屏互动功能的安全性，避免非法终端与智能电视建立连接，从而避免电视受到威胁，避免用户在不知情的情况下，家中的电视被其它人进行了非法连接操控，或者将非法内容投到电视上，影响电视的正常使用。
其中，本申请实施例所述的方法和设备是基于同一申请构思的，由于方法和设备解决问题的原理相似，因此方法和设备的实施可以相互参见，重复之处不再赘述。
下面结合说明书附图对本申请各个实施例进行详细描述。需要说明的是，本申请实施例的展示顺序仅代表实施例的先后顺序，并不代表实施例所提供的技术方案的优劣。
图17中示例性示出了根据本申请实施例中显示设备与控制装置之间操作场景的示意图。如图17所示，用户可通过控制装置100来操作显示设备200。
其中，控制装置100可以是遥控器100A，包括红外协议通信或蓝牙协议通信，及其他短距离通信方式等，通过无线或其他有线方式来控制显示设备200。用户可以通过遥控器上按键、语音输入、控制面板输入等输入用户指令，来控制显示设备200。如：用户可以通过遥控器上音量加减键、频道控制键、上/下/左/右的移动按键、语音输入按键、菜单键、开关机按键等输入相应控制指令，来实现控制显示设备200的功能。
控制装置100也可以是智能设备，如移动终端100B、平板电脑、计算机、笔记本电脑等。例如，使用在智能设备上运行的应用程序控制显示设备200。该应用程序可以在与智能设备关联的屏幕上通过直观的用户界面(UI)为用户提供各种控制。
示例的，移动终端100B可与显示设备200安装软件应用，通过网络通信协议实现连接通信，实现一对一控制操作的和数据通信的目的。如：可以使移动终端100B与显示设备200建立控制指令协议，将遥控控制键盘同步到移动终端100B上，通过控制移动终端100B上用户界面，实现控制显示设备200的功能。也可以将移动终端100B上显示的音视频内容传输到显示设备200上，实现同步显示功能。
如图17所示，显示设备200还与服务器200A通过多种通信方式进行数据通信。可允许显示设备200通过局域网(LAN)、无线局域网(WLAN)和其他网络进行通信连接。服务器200A可以向显示设备200提供各种内容和互动。示例的，显示设备200通过发送和接收信息，以及电子节目指南(EPG)互动，接收软件程序更新，或访问远程储存的数字媒体库。服务器200A可以是一组，也可以是多组，可以是一类或多类服务器。通过服务器200A提供视频点播和广告服务等其他网络服务内容。
显示设备200，可以是液晶显示器、OLED显示器、投影显示设备。具体显示设备类型，尺寸大小和分辨率等不作限定，本领技术人员可以理解的是，显示设备200可以根据需要做性能和配置上的一些改变。
显示设备200除了提供广播接收电视功能之外，还可以附加提供计算机支持功能的智能网络电视功能。示例的包括，网络电视、智能电视、互联网协议电视(IPTV)等。
参见图18，本申请实施例提供的技术方案的具体实现过程，例如包括：
步骤S201a、电视端接收外部设备发送的连接请求，所述连接请求中携带有该外部设备的标识，所述电视端检测此标识是否在该电视端预先设置的黑名单或白名单中，如果都不在，则执行步骤S202；
否则，如果该外部设备的标识在该电视端预先设置的白名单中，则执行步骤S201b，即判断此外部设备是否是通过电视端协议请求连接的，如果不是，则执行步骤S202；如果是，则执行步骤S203。如果该外部设备的标识在该电视端预先设置的黑名单中，则执行步骤S205。
其中，所述电视端协议的协议类型包括：
投屏协议：例如DLNA、Miracast、Airplay等；
连接协议：私有协议。
步骤S202、电视端弹出对话框，向用户提供不信任、信任并连接、屏蔽等选择按钮，例如图19所示的用户界面。
如果用户选择信任并连接按钮，则执行步骤S204；
如果用户选择不信任按钮，则执行步骤S205；
如果用户选择屏蔽按钮，则执行步骤S206。
步骤S203、如果所述电视端协议是连接协议，则直接建立该外部设备与电视端的连接；如果所述电视端协议是投屏协议，则在电视端显示该外部设备的投屏内容。
步骤S204、若所述白名单中没有所述外部设备的标识，则将所述外部设备的标识添加到所述电视端预先设置的白名单中，并且进一步将所述外部设备与所述电视端建立连接所采用的电视端协议的协议类型(投屏协议或私有协议)也添加到所述电视端预先设置的白名单中。
若所述白名单中有所述外部设备的标识，则进一步将所述外部设备与所述电视端建立连接所采用的电视端协议的协议类型(投屏协议或私有协议)也添加到所述电视端预先设置的白名单中；
如果所述外部设备与所述电视端建立连接所采用的电视端协议是连接协议，则直接建立所述外部设备与所述电视端之间的连接，如果所述外部设备与所述电视端建立连接所采用的电视端协议是投屏协议，则在电视端显示所述外部设备的投屏内容。
步骤S205、电视端不做任何处理，或者拒绝该外部设备的连接请求，进一步，还可以发出告警信息。
步骤S206、将所述外部设备与所述电视端建立连接所采用的电视端协议的协议类型，和所述外部设备的标识添加到黑名单中。
另外，用户还可以在电视设置中，打开黑、白名单的管理界面，用户可以调出黑、白名单进行维护管理，例如将之前加入黑、白名单的设备标识、电视端协议删除，删除后若下次该设备再此请求连接电视端，则重复执行上述步骤一。具体地，例如：
若电视端的黑、白名单为空，则呈现给用户的黑、白名单的管理界面例如图20所示，其中常用设备，即白名单，屏蔽设备，即黑名单。
若用户需要添加常用设备，则呈现给用户的黑、白名单的管理界面例如图21所示，在常用设备中可以添加新的设备的标识，例如设备一，当然还可以添加其他相关内容，例如电视端协议等。
若用户需要移除常用设备，则呈现给用户的黑、白名单的管理界面例如图22所示，在常用设备中可以移除设备的标识，例如要移除设备一，则进一步呈现给用户的提示界面如图23所示，请用户选择确认是否移除设备一，该界面上可以有三个选择按钮：取消、移除、屏蔽。若用户选择取消，则取消此次移除设备一的操作，若用户选择移除，则确认在常用设备列表中移除设备一，若用户选择屏蔽，则将设备一的标识添加到屏蔽设备列表中，即从白名单中转移到黑名单中。当然，用户还可以移除其他相关内容，例如电视端协议等。
同理，若用户需要移除屏蔽设备，则呈现给用户的黑、白名单的管理界面例如图24所示，在屏蔽设备中可以移除设备的标识，例如要移除设备五，则进一步呈现给用户的提示界面如图25所示，请用户选择确认是否取消对设备五的屏蔽，该界面上可以有两个选择按钮：取消屏蔽、继续屏蔽。若用户选择取消屏蔽，则取消此次移除设备五的操作，若用户选择继续屏蔽，则确认在屏蔽设备列表中移除设备五。当然，用户还可以移除其他相关内容，例如电视端协议等。
同理，用户也可以添加屏蔽设备，则呈现给用户的黑、白名单的管理界面例如图26所示，用户可以在屏蔽设备列表中添加新的设备的标识，例如当用户选择了图19中所示的屏蔽按钮，以及用户选择了图23中所示的屏蔽按钮时，都会呈现图26所示界面，将用户确认要添加的屏蔽设备的标识添加到屏蔽设备列表中。当然，在屏蔽设备列表中还可以添加其他相关内容，例如电视端协议等。
综上所述，参见图27，本申请实施例提供的一种显示设备，包括：
显示器11，该显示器被配置为显示用户界面，以及与所述显示设备建立连接的外部设备的投屏内容；
与所述显示器通信的控制器12，所述控制器被配置为执行如下流程以实现所述显示器11显示用户界面、与所述显示设备建立连接的外部设备的投屏内容等：
当接收到外部设备的连接请求时，从所述连接请求中获取所述外部设备的标识；例如，接收到的外部设备的连接请求中携带的该外部设备的标识是“设备一”；
判断所述连接请求中携带的所述外部设备的标识是否包含在所述黑名单或所述白名单中，根据判断结果确定是否建立所述显示设备与所述外部设备的连接，其中所述白名单中包括可以与所述显示设备建立连接的外部设备的标识，所述黑名单中包括不可以与所述显示设备建立连接的外部设备的标识。
该显示设备，当接收到外部设备的连接请求时，通过判断所述连接请求中携带的所述外部设备的标识是否包含在所述黑名单或所述白名单中，并根据判断结果确定是否建立所述显示设备与所述外部设备的连接，从而提高了多屏互动功能的安全性，避免了非法终端与智能电视建立连接。
在本申请一些实施例中，若所述外部设备的标识不包含在所述黑名单中也不包含在所述白名单中，则所述控制器12还用于：
控制所述显示器呈现包含不信任、信任并连接、屏蔽选择按钮的用户界面(例如图19所示的界面，其中“XX手机”为请求建立连接的外部设备的标识)；其中，
如果用户选择信任并连接按钮，则将所述外部设备的标识添加到所述白名单中，并且，如果所述外部设备与所述显示设备建立连接所采用的协议是连接协议，则直接建立所述外部设备与所述显示设备之间的连接，如果所述外部设备与所述显示设备建立连接所采用的 协议是投屏协议，则控制显示器11显示所述外部设备的投屏内容；
如果用户选择不信任按钮，则控制所述显示设备不与所述外部设备建立连接；例如执行上述步骤S205的流程；
如果用户选择屏蔽按钮，则将所述外部设备的标识添加到所述黑名单中。
在本申请一些实施例中，将所述外部设备的标识添加到所述白名单中的同时，所述控制器12还将所述外部设备与所述显示设备建立连接所采用的协议添加到所述白名单中。
在本申请一些实施例中，将所述外部设备的标识添加到所述黑名单中的同时，所述控制器12还将所述外部设备与所述显示设备建立连接所采用的协议添加到所述黑名单中。
在本申请一些实施例中，如果所述外部设备的标识在所述白名单中，并且所述外部设备是通过所述显示设备的协议请求建立连接的，则：
如果所述外部设备与所述显示设备建立连接所采用的协议是连接协议，则所述控制器控制直接建立所述外部设备与所述显示设备之间的连接，如果所述外部设备与所述显示设备建立连接所采用的协议是投屏协议，则所述控制器控制所述显示器显示所述外部设备的投屏内容。
在本申请一些实施例中，如果所述外部设备的标识在所述白名单中，并且所述外部设备不是通过所述显示设备的协议请求建立连接的，则所述控制器12还用于：
控制所述显示器呈现包含不信任、信任并连接、屏蔽选择按钮的用户界面(例如图19所示的界面，其中“XX手机”为请求建立连接的外部设备的标识)；其中，
如果用户选择信任并连接按钮，则将所述外部设备的标识添加到所述白名单中，并且，如果所述外部设备与所述显示设备建立连接所采用的协议是连接协议，则直接建立所述外部设备与所述显示设备之间的连接，如果所述外部设备与所述显示设备建立连接所采用的协议是投屏协议，则控制所述显示器显示所述外部设备的投屏内容；
如果用户选择不信任按钮，则控制所述显示设备不与所述外部设备建立连接；
如果用户选择屏蔽按钮，则将所述外部设备的标识添加到所述黑名单中。
在本申请一些实施例中，如果所述外部设备的标识在所述黑名单中，则所述控制器12控制所述显示设备不与所述外部设备建立连接。
在本申请一些实施例中，所述控制器12还用于：
控制所述显示器呈现所述显示设备的黑、白名单的管理界面(例如图20～图26所示的界面)，实现用户通过该管理界面对所述黑名单和/或白名单进行编辑管理，例如在所述黑名单和/或白名单中删除、添加外部设备标识等操作，具体参见上述内容，在此不再赘述。
参见图28，相应地，本申请实施例提供的一种与显示设备建立连接的控制方法，包括：
S101、当接收到外部设备的连接请求时，从所述连接请求中获取所述外部设备的标识；
S102、判断所述连接请求中携带的所述外部设备的标识是否包含在预先设置的黑名单或白名单中，根据判断结果确定是否建立所述外部设备与所述显示设备的连接，其中所述白名单中包括可以与所述显示设备建立连接的外部设备的标识，所述黑名单中包括不可以与所述显示设备建立连接的外部设备的标识。
其中，根据判断结果确定是否建立所述外部设备与所述显示设备的连接，具体可以参见上述图2所示流程，在此不再赘述。
综上所述，本申请实施例提供的技术方案可以在电视上将通过所有连接方式的终端设备的标识分类保存，分为白名单，黑名单，并可以进行管理，只有在白名单中的手机设备 等终端允许其与电视进行连接并响应其操控指令，黑名单中的设备则直接屏蔽在连接阶段，用户将无感知。如果不在白名单和黑名单中的设备，每次连接时都会弹出对话框让用户选择屏蔽、不信任、或信任并连接。从而，可以大大提高多屏互动功能的安全性。
本申请实施例提供的投屏方法，可以适用于图29所示的投屏系统。如图29所示，该投屏系统包括：第一终端设备291、接入网设备292和第二终端设备293。其中，第一终端设备291和第二终端设备293可以是手机、pad、电脑、电视等设备，接入网设备292可以是路由器。
本申请一些实施例中，第一终端设备291可以是手机，接入网设备292可以是路由器，第二终端设备293可以是电视。本实施例中，路由器可以作为访问接入点(Access Point，AP)建立局域网，手机和电视均位于该局域网内。在一些场景中，由于手机屏幕小，观看视频等会存在不方便，因此，有些用户会想要将手机上的一些图片、视频等多媒体数据投屏到电视上，目前的投屏主要是基于DLNA协议。例如，用户打开爱奇艺、优酷、腾讯视频等视频软件上的某一个视频资源，想要将该视频资源投屏到电视上进行观看，可以通过点击视频软件上的投屏按键发起投屏请求，此时，用户点击投屏按键相当于是发起了设备发现请求，该设备发现请求会首先到达路由器，由路由器搜索位于上述局域网范围内的其他设备，并将搜索结果反馈给用户。由于上述过程需要路由器的参与，因此，如果路由器发生丢包就会导致设备发现率低，以及路由器如果设置了AP隔离也会造成无法发现设备，进而导致搜索不到电视，也就是发现不了电视，从而无法实现DLNA投屏，影响用户体验。
本申请实施例提供的投屏方法，旨在解决现有技术的如上技术问题。
下面以具体地实施例对本申请的技术方案以及本申请的技术方案如何解决上述技术问题进行详细说明。下面这几个具体的实施例可以相互结合，对于相同或相似的概念或过程可能在某些实施例中不再赘述。下面将结合附图，对本申请的实施例进行描述。
图30为本申请实施例提供的投屏方法流程图。本申请实施例针对现有技术的如上技术问题，提供了投屏方法，该方法具体步骤如下：
步骤901、发送第一发现请求。
其中，第一发现请求采用DLNA通信协议。本实施例的执行主体可以是终端设备，例如手机。在本实施例的DLNA投屏过程中，终端设备作为控制点，终端设备可以通过发送组播搜索消息的形式发送第一发现请求。
在手机和电视的示例中，可以是根据手机接收到的用户在手机上的操作信息，发送第一发现请求，例如，用户打开视频软件，并点击视频软件上的投屏按键，从而发起第一发现请求，之后，第一发现请求被发送至手机所在的局域网内的路由器。
本实施例中，并不限于以上述投屏按键的方式发起第一发现请求，还可以是通过其他预定的手势操作或者对终端设备本身进行的操作，例如对终端设备进行施力，例如旋转等操作使终端设备的运动状态发生变化，从而发起第一发现请求。
步骤902、在根据第一发现请求的响应结果确定未发现目标设备的情况下，发送第二发现请求。
其中，第二发现请求采用端到端的通信协议，例如P2P(Peer to Peer)协议。第二发现请求包括WIFI直连请求。
本实施例中，上述第一发现请求在发送至路由器后，由路由器会将该第一发现请求发送至上述局域网内的其他设备，其他设备接收到第一发现请求后，会回复一个确认消息给 终端设备，并将数字媒体喧染器(Digital Media Renderer，DMR)的描述文件等信息回复给终端设备。如果终端设备没有接收到任何确认消息，或者终端设备根据接收到的确认消息和DMR描述文件的回复消息确定没有发现目标设备的情况下，则自动切换至P2P协议，也就是启动P2P协议发起第二发现请求。
值得注意的是，本实施例中的终端设备和目标设备须连接同一WIFI网络。
在本申请一些实施例中，在根据第一发现请求的响应结果确定未发现目标设备的情况下，发送第二发现请求，包括：在预设时间段内未接收到组播搜索消息的确认消息，发送第二发现请求。例如，用户在手机上通过点击投屏按键发送组播搜索消息后，路由器就会将该组播搜索消息发送至局域网内的其他设备，其他设备接收到该组播搜索消息后，会回复一个确认收到的消息(确认消息)给手机，如果手机等待T1时间之后，没有接收到任何确认消息，则认为发现失败。
在本申请一些实施例中，在根据第一发现请求的响应结果确定未发现目标设备的情况下，发送第二发现请求，包括：在未接收到组播搜索消息的确认消息的情况下，再次发送第一发现请求；在未接收到组播搜索消息的确认消息的次数超过预设次数的情况下，发送第二发现请求。在上述举例的基础上例如，用户在手机上通过点击投屏按键发送组播搜索消息后，路由器就会将该组播搜索消息发送至局域网内的其他设备，其他设备接收到该组播搜索消息后，会回复一个确认收到的消息(确认消息)给手机，这个过程通常耗时为T2，如果手机等待T2时间之后，没有接收到任何确认消息，则再次发送组播搜索消息，并记录发送组播搜索消息的次数，如果连续发送3次组播搜索消息且均未接收到确认消息，则认为发现失败。
步骤903、在通过第二发现请求发现目标设备的情况下，获取目标设备的IP地址。
在本申请一些实施例中，在通过第二发现请求发现目标设备的情况下，获取目标设备的IP地址，包括：
步骤a1、接收第二发现请求的回复消息，回复消息包括发现的设备的mac地址和/或设备名称。
步骤a2、基于回复消息确定目标设备的mac地址和/或设备名称。
步骤a3、根据目标设备的mac地址和/或设备名称，向目标设备发送P2P连接。
步骤a4、在确定终端设备是服务端，且目标设备是客户端的情况下，接收目标设备发送的TCP连接请求，TCP连接请求携带有目标设备的IP地址。
步骤a5、对TCP连接请求进行解析，得到目标设备的IP地址。
示例性地，以第二发现请求是WIFI直连请求为例，如图31所示，则在通过第二发现请求发现目标设备的情况下，获取目标设备的IP地址，包括：终端设备发起WIFI直连请求，与终端设备连接至同一WIFI网络，且与终端设备之间的距离小于预设距离的终端设备(以下简称其他终端设备)在接收到WIFI直连请求后，回复自身的设备名称和MAC信息给终端设备；终端设备根据设备名称和MAC信息向与设备名称和MAC信息对应的其他终端设备发起建立连接请求，其他终端设备接收到建立连接请求后，与发起建立连接请求的终端设备进行协商，协商内容包括确定客户端(group client)和服务端(group owner)，之后，由服务端创建TCPserver，并由客户端向服务端发送TCP连接请求，服务端接收到TCP连接请求后，解析出客户端的IP地址，再通过TCP通信方式将客户端的IP地址和服务端的IP地址均返回给客户端。
例如，电视和手机都连接至同一WIFI网络的情况下，手机发起WIFI直连请求，电视接收到WIFI直连请求的情况下，将电视的设备名称和MAC信息回复给手机，手机根据电视的设备名称和MAC信息向电视发起建立连接请求，电视接收到该建立连接请求后，与手机进行协商，在协商结果是电视是客户端，手机是服务端，则由手机创建TCPserver，以及接收电视向手机发送的TCP连接请求，手机接收到TCP连接请求后，从TCP连接请求中解析出来电视的IP地址，然后通过TCP方式将电视的IP地址和手机的IP地址发送给电视。
在本申请一些实施例中，在上述步骤a3之后，本实施例的方法还包括：
步骤a6、在确定终端设备是客户端，且目标设备是服务端的情况下，则获取目标设备的IP地址。
示例性地，如果协商结果是电视是服务端，手机是客户端，则电视和手机都知道电视的IP地址，手机和电视均可以直接获取电视的IP地址。
在本申请一些实施例中，得到目标设备的IP地址之后，本实施例的方法还包括：发送目标设备的IP地址至目标设备，以使目标设备根据目标设备的IP地址将DLNA通信协议进行初始化。例如，在手机是服务端，电视是客户端的情况下，手机将电视的IP地址发送给电视，电视用电视自身的IP地址来初始化DLNA协议，以便后续通过单播的形式与手机建立数据传输通道。再例如，在手机是客户端，电视是服务端的情况下，电视可以直接获取电视自身的IP地址，进而自身的IP地址来初始化DLNA协议，以便后续通过单播的形式与手机建立数据传输通道。
步骤904、通过DLNA通信协议与IP地址对应的目标设备建立数据传输通道。
在本申请一些实施例中，通过DLNA通信协议与IP地址对应的目标设备建立数据传输通道，包括：向IP地址对应的目标设备发送单播消息；接收目标设备返回的描述文件的地址信息；根据描述文件的地址信息，从与描述文件的地址信息对应的地址获取描述文件，描述文件包括目标设备的描述信息；根据描述信息，将目标设备保存至设备列表中。
例如，手机在获取到电视的IP地址之后，手机就可以通过单播的方式向电视发起DLNA连接，以建立数据传输通道。具体的，是手机向IP地址对应的电视发送单播消息，电视接收到该单播消息后，会返回DMR描述文件地址信息，然后手机根据DMR描述文件地址信息向电视请求获取DMR描述文件，电视将DMR描述文件回复给手机，手机就可以获取到更加详细的设备描述信息，例如电视的唯一识别码(Universally Unique Identifier，UUID)和DLNA设备类型信息，之后手机将获取到的电视的所有信息都保存至设备列表中。至此，数据传输通道建立完毕。
步骤905、通过数据传输通道向目标设备传输多媒体数据，以通过目标设备将多媒体数据进行显示和/或播放。
在本申请一些实施例中，通过数据传输通道向目标设备传输多媒体数据，以通过目标设备将所述多媒体数据进行显示和/或播放，包括：根据接收到的对多媒体数据的选择操作信息，确定待推送的多媒体数据；基于待推送的多媒体数据形成推送链接；发送推送链接至目标设备，以使目标设备根据推送链接解析得到文件地址，以及根据文件地址获取相应的文件，并将获取到的文件发送至播放器进行播放。
例如，数据传输通道建立好之后，用户可以在手机上选择图片、视频等多媒体数据，并点击推送到电视，电视接收到推送内容后，解析出文件地址，然后根据文件地址获取相 应的文件，并将获取到的文件发送至播放器进行播放。
本申请实施例通过发送第一发现请求，第一发现请求采用DLNA通信协议；在根据第一发现请求的响应结果确定未发现目标设备的情况下，发送第二发现请求，第二发现请求采用端到端的通信协议；在通过第二发现请求发现目标设备的情况下，获取目标设备的IP地址；通过DLNA通信协议与IP地址对应的目标设备建立数据传输通道；通过数据传输通道向目标设备传输多媒体数据，以通过目标设备将多媒体数据进行显示和/或播放。本实施例在设备发现过程中，如果未发现目标设备，则切换至端到端的通信协议来发现目标设备，由于端到端的通信协议是去中心化的直连通信方式，不需要中间中转者，因此，对于路由器丢包导致设备发现率低，或者是路由器设置了AP隔离等情况导致无法设备的情形而言，也能够绕过路由器来发现目标设备。进一步的，在通过端到端的通信方式获取到目标设备的IP地址之后，通过DLNA协议与目标设备建立数据传输通道，然后将要投屏的多媒体数据通过该数据传输通道传输至目标设备，再通过目标设备的播放器和/或显示进行显示和/或播放，因此，能够实现DLNA投屏。
下面以手机和电视为例，通过一个完整的示例对本申请实施例的投屏方法进行详细说明：
图32为本申请实施例的一示例提供的投屏方法流程图。如图312所示，本实施例的示例提供的投屏方法具体包括如下步骤：
步骤4001、电视连接WIFI后，初始化DLNA协议，并向组播地址，例如239.255.255.250地址发送组播加入请求，以加入组播。
步骤4002、手机连接WIFI后，向组播地址发送组播搜索消息。
例如，手机向239.255.255.250地址发送组播搜索消息。
步骤4003、手机判断是否接收到组播搜索消息的确认消息。
如果手机接收到组播搜索消息的确认消息，则代表电视接收到了组播搜索消息，并向手机返回了确认消息，同时，电视也会向手机返回DMR描述文件地址。
步骤4004、手机在接收到组播搜索消息的确认消息的情况下，根据DMR描述文件地址向电视发送DMR描述文件的获取请求。
步骤4005、电视接收到DMR描述文件的获取请求后，向手机返回DMR描述文件。
其中，DMR描述文件内容至少包括设备名称、IP地址、MAC信息、UUID、DLNA设备类型。
步骤4006、手机接收到DMR描述文件后，解析DMR描述文件，并将解析得到的DMR描述文件内容保存至设备列表。
步骤4007、手机根据用户在手机上选择的推送内容，向电视发送推送链接。
步骤4008、电视接收到推送链接后，解析该推送链接，并调用播放器播放推送链接中的推送内容。
步骤4009、在手机没有接收到组播搜索消息的确认消息的情况下，则发起P2P发现请求。
本步骤的实施过程可以参见上述步骤202的介绍，此处不再赘述。
步骤4010、电视接收到P2P发现请求后，向手机返回电视的MAC信息和/或设备名称。
步骤4011、手机将接收到的电视的MAC信息和/或设备名称保存。
步骤4012、手机根据电视的MAC信息和/或设备名称，向电视发起P2P连接请求。
步骤4013、电视接收到P2P连接请求后，与手机协商确定客户端和服务端。
步骤4014、在电视是客户端，手机是服务端的情况下，电视向手机发送TCP连接请求。
具体的，在电视是客户端，手机是服务端的情况下，由手机创建一个服务器线程tcpserver，电视主动连接该tcpserver，然后手机接收来自电视的TCP连接请求。
步骤4015、手机接收到该TCP连接请求后，解析出电视的IP地址，并通过TCP方式将电视的IP地址和手机的IP地址返回给电视。
步骤4016、电视通过电视自身的IP地址初始化DLNA协议。
具体的，电视通过电视自身的IP地址初始化DLNA协议，是对后续手机和电视通过单播方式建立连接做准备。
步骤4017、手机向电视IP地址发送单播消息。
步骤4018、电视接收到单播消息后，返回描述文件地址信息给手机。
步骤4019、手机根据描述文件地址信息，向电视的描述文件地址发送描述文件获取请求。
步骤4020、电视接收到描述文件获取请求后，返回描述文件给手机。
步骤4021、手机接收到描述文件后，解析描述文件，得到电视的描述文件信息，并将电视的描述文件信息保存至设备列表。
其中，电视的描述文件信息至少包括UUID和DLNA设备类型。
步骤4022、手机根据接收到的推送内容的选择信息，选定对应的推送内容，并向电视发送DLNA推送链接。
步骤4023、电视接收到DLNA推送链接后，解析DLNA推送链接，并调用播放器播放推送内容。
在上述实施例的基础上，在本申请一些实施例中，手机还可以是客户端，以及电视是服务端，则在手机是客户端，电视是服务端的情况下，电视和手机都知道电视的IP地址。电视获取电视自身的IP地址，并直接使用电视IP初始化DLNA协议，之后由手机直接向电视的IP地址发送单播消息，并执行上述步骤4018-4023。
图33为本申请实施例提供的投屏装置的结构示意图。该投屏装置具体可以是上述实施例中的终端设备。本申请实施例提供的投屏装置可以执行投屏方法实施例提供的处理流程，如图33所示，投屏装置50包括：第一发送模块51、第二发送模块52、获取模块53、连接模块54和推送模块55；其中，第一发送模块51，用于发送第一发现请求，所述第一发现请求采用DLNA通信协议；第二发送模块52，用于在根据所述第一发现请求的响应结果确定未发现目标设备的情况下，发送第二发现请求，所述第二发现请求采用端到端的通信协议；获取模块53，用于在通过所述第二发现请求发现所述目标设备的情况下，获取所述目标设备的IP地址；连接模块54，用于通过所述DLNA通信协议与所述IP地址对应的目标设备建立数据传输通道；推送模块55，用于通过所述数据传输通道向所述目标设备传输多媒体数据，以通过所述目标设备将所述多媒体数据进行显示和/或播放。
在本申请一些实施例中，获取模块53在通过所述第二发现请求发现所述目标设备的情况下，获取所述目标设备的IP地址时，具体包括：接收所述第二发现请求的回复消息， 所述回复消息包括发现的设备mac地址和/或设备名称；基于所述回复消息确定所述目标设备的mac地址和/或设备名称；根据所述目标设备的mac地址和/或设备名称，向所述目标设备发送P2P连接；在确定所述终端设备是服务端，且所述目标设备是客户端的情况下，接收所述目标设备发送的TCP连接请求，所述TCP连接请求携带有所述目标设备的IP地址；对所述TCP连接请求进行解析，得到所述目标设备的IP地址。
在本申请一些实施例中，所述获取模块53还用于在确定所述终端设备是客户端，且所述目标设备是服务端的情况下，所述目标设备获取自身的IP地址。
在本申请一些实施例中，连接模块54在通过DLNA通信协议与所述IP地址对应的目标设备建立数据传输通道时，具体包括：向所述IP地址对应的目标设备发送单播消息；接收所述目标设备返回的描述文件的地址信息；根据所述描述文件的地址信息，从与所述描述文件的地址信息对应的地址获取所述描述文件，所述描述文件包括所述目标设备的描述信息；根据所述描述信息，将所述目标设备保存至设备列表中。
在本申请一些实施例中，所述第一发送模块51在发送第一发现请求时，具体包括：发送组播搜索消息；所述第二发送模块52在根据所述第一发现请求的响应结果确定未发现目标设备的情况下，发送第二发现请求时，具体包括：在预设时间段内未接收到所述组播搜索消息的确认消息，发送第二发现请求。
在本申请一些实施例中，所述第一发送模块51在发送第一发现请求时，具体包括：发送组播搜索消息；所述第二发送模块52在根据所述第一发现请求的响应结果确定未发现目标设备的情况下，发送第二发现请求时，具体包括：在未接收到所述组播搜索消息的确认消息的情况下，再次发送所述第一发现请求；在未接收到所述组播搜索消息的确认消息的次数超过预设次数的情况下，发送第二发现请求。
在本申请一些实施例中，第二发送模块52，还用于发送所述目标设备的IP地址至所述目标设备，以使所述目标设备根据所述目标设备的IP地址将所述DLNA通信协议进行初始化。
图33所示实施例的投屏装置可用于执行上述方法实施例的技术方案，其实现原理和技术效果类似，此处不再赘述。
图34为本申请实施例提供的投屏设备的结构示意图。该投屏设备具体可以是上述实施例中的终端设备。本申请实施例提供的投屏设备可以执行投屏方法实施例提供的处理流程，如图314所示，投屏设备60包括：存储器61、处理器62、计算机程序和通讯接口63；其中，计算机程序存储在存储器61中，并被配置为由处理器62执行以下操作：发送第一发现请求，所述第一发现请求采用DLNA通信协议；在根据所述第一发现请求的响应结果确定未发现目标设备的情况下，发送第二发现请求，所述第二发现请求采用端到端的通信协议；在通过所述第二发现请求发现所述目标设备的情况下，获取所述目标设备的IP地址；通过所述DLNA通信协议与所述IP地址对应的目标设备建立数据传输通道；通过所述数据传输通道向所述目标设备传输多媒体数据，以通过所述目标设备将所述多媒体数据进行显示和/或播放。
在本申请一些实施例中，所述处理器62在通过所述第二发现请求发现所述目标设备的情况下，获取所述目标设备的IP地址时，具体包括：接收所述第二发现请求的回复消息，所述回复消息包括发现的设备mac地址和/或设备名称；基于所述回复消息确定所述目标设 备的mac地址和/或设备名称；根据所述目标设备的mac地址和/或设备名称，向所述目标设备发送P2P连接；在确定所述终端设备是服务端，且所述目标设备是客户端的情况下，接收所述目标设备发送的TCP连接请求，所述TCP连接请求携带有所述目标设备的IP地址；对所述TCP连接请求进行解析，得到所述目标设备的IP地址。
在本申请一些实施例中，所述处理器62还用于：在确定所述终端设备是客户端，且所述目标设备是服务端的情况下，所述目标设备获取自身的IP地址。
在本申请一些实施例中，所述处理器62通过所述DLNA通信协议与所述IP地址对应的目标设备建立数据传输通道，包括：向所述IP地址对应的目标设备发送单播消息；接收所述目标设备返回的描述文件的地址信息；根据所述描述文件的地址信息，从与所述描述文件的地址信息对应的地址获取所述描述文件，所述描述文件包括所述目标设备的描述信息；根据所述描述信息，将所述目标设备保存至设备列表中。
在本申请一些实施例中，所述处理器62发送第一发现请求时，具体包括：发送组播搜索消息；在本申请一些实施例中，所述处理器62在根据所述第一发现请求的响应结果确定未发现目标设备的情况下，发送第二发现请求时，具体包括：在预设时间段内未接收到所述组播搜索消息的确认消息，发送第二发现请求。
在本申请一些实施例中，所述处理器62在发送第一发现请求时，具体包括：发送组播搜索消息；所述处理器62在根据所述第一发现请求的响应结果确定未发现目标设备的情况下，发送第二发现请求时，具体包括：在未接收到所述组播搜索消息的确认消息的情况下，再次发送所述第一发现请求；在未接收到所述组播搜索消息的确认消息的次数超过预设次数的情况下，发送第二发现请求。
在本申请一些实施例中，所述处理器62还用于：发送所述目标设备的IP地址至所述目标设备，以使所述目标设备根据所述目标设备的IP地址将所述DLNA通信协议进行初始化。
图34所示实施例的投屏设备可用于执行上述方法实施例的技术方案，其实现原理和技术效果类似，此处不再赘述。
另外，本申请实施例还提供一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行以实现上述实施例所述的投屏方法。
在本申请一些实施例中，还提供了一种包括指令的存储介质，例如包括指令的存储器，上述指令可由多媒体设备的处理器执行以完成上述投屏播放方法。可选地，存储介质可以是非临时性计算机可读存储介质，例如，所述非临时性计算机可读存储介质可以是ROM、随机存取存储器(RAM)、CD-ROM、磁带、软盘和光数据存储设备等。
本申请实施例还提供一种计算机程序产品，当所述计算机程序产品在电子设备上运行时，使得所述电子设备执行实现本申请实施例上述任意一项所述的投屏播放方法。
本领域技术人员在考虑说明书及实践这里发明的发明后，将容易想到本申请的其它实施方案。本申请旨在涵盖本申请的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本申请的一般性原理并包括本申请未发明的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本申请的真正范围和精神由下面的权利要求指出。
应当理解的是，本申请并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本申请的范围仅由所附的权利要求来限制。

Claims (10)

  1. 一种多媒体设备，其特征在于，包括：处理器、显示单元以及语音单元；
    所述显示单元，用于显示至少一个视频缓存区内的视频数据；
    所述语音单元，用于播放所述音频缓存区内的音频数据；
    所述处理器，用于在与多个终端同时进行投屏过程中，从所述多个终端中确定目标终端；
    将所述目标终端发送的音频数据存放在音频缓存区中，以及丢弃其他终端发送的音频数据；并将多个终端发送视频数据存放在各终端对应的视频缓存区中。
  2. 根据权利要求1所述的多媒体设备，其特征在于，所述处理器，具体用于：
    将用户从所述多个终端中选择的终端作为目标终端；或
    从所述多个终端中选择一个终端作为目标终端；或
    从所述多个终端中选择第N个连接的终端作为目标终端，其中N为正整数。
  3. 根据权利要求1或2所述的多媒体设备，其特征在于，所述处理器，还用于：
    在与多个终端同时进行投屏过程中，检测所述多个终端的连接状态；
    若检测到所述目标终端断开连接，则删除所述音频缓存区的音频数据以及释放所述目标终端对应的视频缓存区的内存空间，并从剩余的终端中确定新的目标终端；
    将新的目标终端发送的音频数据存放在所述音频缓存区中。
  4. 根据权利要求1所述的多媒体设备，其特征在于，所述处理器，还用于：
    若检测到所有终端均断开连接，则释放所述音频缓存区的内存空间。
  5. 根据权利要求2所述的多媒体设备，其特征在于，所述处理器，具体用于：
    响应用户在所述多媒体设备上操作的选择视频指令，将所述选择视频指令所选择的视频对应的终端作为目标终端。
  6. 一种投屏播放方法，其特征在于，应用于多媒体设备，所述方法包括：
    在与多个终端同时进行投屏过程中，从所述多个终端中确定目标终端；
    将所述目标终端发送的音频数据存放在音频缓存区中，以及丢弃其他终端发送的音频数据；并将多个终端发送视频数据存放在各终端对应的视频缓存区中；
    播放所述音频缓存区内的音频数据，以及将至少一个视频缓存区内的视频数据进行显示。
  7. 根据权利要求6所述的投屏播放方法，其特征在于，从所述多个终端中确定目标终端，包括：
    将用户从所述多个终端中选择的终端作为目标终端；或
    从所述多个终端中选择一个终端作为目标终端；或
    从所述多个终端中选择第N个连接的终端作为目标终端，其中N为正整数。
  8. 根据权利要求6或7所述的投屏播放方法，其特征在于，所述方法还包括：
    在与多个终端同时进行投屏过程中，检测所述多个终端的连接状态；
    若检测到所述目标终端断开连接，则删除所述音频缓存区的音频数据以及释放所述目标终端对应的视频缓存区的内存空间，并从剩余的终端中确定新的目标终端；
    将新的目标终端发送的音频数据存放在所述音频缓存区中。
  9. 根据权利要求8所述的投屏播放方法，其特征在于，在与多个终端同时进行投屏过程中，检测所述多个终端的连接状态之后，所述方法还包括：
    若检测到所有终端均断开连接，则释放所述音频缓存区的内存空间。
  10. 根据权利要求7所述的投屏播放方法，其特征在于，将用户从所述多个终端中选择的终端作为目标终端，包括：
    响应用户在所述多媒体设备上操作的选择视频指令，将所述选择视频指令所选择的视频对应的终端作为目标终端。
PCT/CN2021/081561 2020-03-20 2021-03-18 多媒体设备及投屏播放方法 Ceased WO2021185318A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/804,257 US12273586B2 (en) 2020-03-20 2022-05-26 Multimedia apparatus and cast method

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
CN202010199814.8 2020-03-20
CN202010199814.8A CN111427527B (zh) 2020-03-20 2020-03-20 投屏方法、装置、设备及计算机可读存储介质
CN202010519510.5 2020-06-09
CN202010519510.5A CN111711843A (zh) 2020-06-09 2020-06-09 多媒体设备及投屏播放方法
CN202010522106.3 2020-06-10
CN202010522106.3A CN111726663A (zh) 2020-06-10 2020-06-10 显示设备以及与显示设备建立连接的控制方法

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/804,257 Continuation US12273586B2 (en) 2020-03-20 2022-05-26 Multimedia apparatus and cast method

Publications (1)

Publication Number Publication Date
WO2021185318A1 true WO2021185318A1 (zh) 2021-09-23

Family

ID=77770235

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/081561 Ceased WO2021185318A1 (zh) 2020-03-20 2021-03-18 多媒体设备及投屏播放方法

Country Status (2)

Country Link
US (1) US12273586B2 (zh)
WO (1) WO2021185318A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113852715A (zh) * 2021-10-13 2021-12-28 四川启睿克科技有限公司 一种快捷镜像投屏系统及方法
CN115022694A (zh) * 2022-06-27 2022-09-06 北京奇艺世纪科技有限公司 一种投屏时间统计方法、装置、系统、后台服务器及介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101489091A (zh) * 2009-01-23 2009-07-22 深圳华为通信技术有限公司 一种语音信号传输处理方法及装置
CN101510990A (zh) * 2009-02-27 2009-08-19 深圳华为通信技术有限公司 一种处理远程呈现会议用户信号的方法及系统
US20120039219A1 (en) * 2010-08-10 2012-02-16 Blabbelon, Inc. Highly scalable voice conferencing service
CN107276777A (zh) * 2017-07-27 2017-10-20 苏州科达科技股份有限公司 会议系统的音频处理方法及装置
CN111049848A (zh) * 2019-12-23 2020-04-21 腾讯科技（深圳）有限公司 通话方法、装置、系统、服务器及存储介质
CN111711843A (zh) * 2020-06-09 2020-09-25 海信视像科技股份有限公司 多媒体设备及投屏播放方法

Family Cites Families (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH05113864A (ja) 1991-10-22 1993-05-07 Canon Inc マルチウインドウ動画表示方法及び装置
CN100581226C (zh) 2005-08-05 2010-01-13 三星电子株式会社 用于提供多屏幕的设备以及动态配置多屏幕的方法
CN101646221A (zh) 2008-08-08 2010-02-10 比亚迪股份有限公司 一种蓝牙连接请求的处理方法、系统及移动终端
CN101494777A (zh) 2009-02-25 2009-07-29 中兴通讯股份有限公司 一种解码控制器、多媒体播放终端及其媒体播放方法
CN101588488B (zh) 2009-06-18 2011-12-28 北京浪弯融科科技有限责任公司 多机位流媒体播放方法及系统
US20110116540A1 (en) * 2009-11-18 2011-05-19 General Instrument Corporation Multimedia Content Handling in a Home-Network System
WO2012001905A1 (ja) 2010-06-30 2012-01-05 パナソニック株式会社 再生装置、音声選択方法および音声選択プログラム
FR2966997A1 (fr) * 2010-10-27 2012-05-04 France Telecom Execution deportee d'une application logicielle au sein d'un reseau
US20130298171A1 (en) * 2010-11-18 2013-11-07 Sharp Kabushiki Kaisha Device for generating content data, method for generating content data, and recording medium
CN201887900U (zh) 2010-12-07 2011-06-29 叶惠仙 可支持多路音/视频同时播放的装置
KR101867334B1 (ko) * 2011-08-25 2018-06-18 삼성전자주식회사 홈 네트워크 환경에서 컨텐츠를 공유하는 방법 및 장치
KR101889378B1 (ko) * 2011-10-14 2018-08-21 삼성전자주식회사 사용자 단말 장치 및 그 컨텐츠 공유 방법
FR2982110B3 (fr) 2012-01-10 2014-03-14 Samsung Electronics Co Ltd Dispositif de lunettes pour visionner une image d'affichage
AU2013276983B2 (en) * 2012-12-26 2017-05-18 Samsung Electronics Co., Ltd. Terminal device and method for controlling thereof
KR20140099359A (ko) * 2013-02-01 2014-08-12 (주)휴맥스 콘텐츠 재생 방법 및 장치
US9043822B1 (en) * 2013-03-14 2015-05-26 Cox Communications, Inc. Remote time shift buffering with shadow tuning
US10701450B2 (en) * 2013-05-29 2020-06-30 Interdigital Ce Patent Holdings, Sas Apparatus and method for navigating through media content
CN104754426B (zh) * 2013-12-25 2018-11-16 华为终端有限公司 一种媒体处理方法、设备及系统
KR102202576B1 (ko) 2014-12-12 2021-01-13 삼성전자주식회사 음향 출력을 제어하는 디바이스 및 그 방법
CN104717522A (zh) 2015-03-20 2015-06-17 深圳市环球数码创意科技有限公司 一种多视频设备同步一个音频设备进行播放的系统及方法
CN105005722A (zh) 2015-06-26 2015-10-28 北京北信源软件股份有限公司 移动存储设备的接入控制方法及装置
US10673907B2 (en) * 2015-07-16 2020-06-02 Arris Enterprises Llc Systems and methods for providing DLNA streaming to client devices
US10009654B2 (en) * 2015-12-15 2018-06-26 At&T Intellectual Property I, L.P. Media interface device
CN105916015A (zh) 2015-12-15 2016-08-31 乐视致新电子科技（天津）有限公司 一种车载多设备共享音视频数据的方法、装置及系统
CN106488294A (zh) 2016-09-28 2017-03-08 乐视控股（北京）有限公司 弹幕信息传输、显示方法及装置
CN109842959A (zh) 2017-09-08 2019-06-04 苏州必捷网络有限公司 一种多个iOS终端连接到一个Linux终端的方法
CN108256312B (zh) 2018-01-19 2020-11-27 Oppo广东移动通信有限公司 网页打开本地应用程序的方法、装置及移动终端
CN108737690A (zh) 2018-05-30 2018-11-02 湖北安心智能科技有限公司 一种分布式协同交互系统和方法
CN109963178B (zh) 2019-03-15 2022-04-26 北京领度信息技术有限公司 使用安全投屏器进行投屏的方法和安全投屏器
CN110069229B (zh) 2019-04-22 2023-12-08 努比亚技术有限公司 屏幕共享方法、移动终端及计算机可读存储介质
US10965984B1 (en) * 2019-05-28 2021-03-30 Amazon Technologies, Inc. Minimization of video re-buffering using local animation
CN110381197B (zh) 2019-06-27 2021-06-15 华为技术有限公司 多对一投屏中音频数据的处理方法、装置及系统

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101489091A (zh) * 2009-01-23 2009-07-22 深圳华为通信技术有限公司 一种语音信号传输处理方法及装置
CN101510990A (zh) * 2009-02-27 2009-08-19 深圳华为通信技术有限公司 一种处理远程呈现会议用户信号的方法及系统
US20120039219A1 (en) * 2010-08-10 2012-02-16 Blabbelon, Inc. Highly scalable voice conferencing service
CN107276777A (zh) * 2017-07-27 2017-10-20 苏州科达科技股份有限公司 会议系统的音频处理方法及装置
CN111049848A (zh) * 2019-12-23 2020-04-21 腾讯科技（深圳）有限公司 通话方法、装置、系统、服务器及存储介质
CN111711843A (zh) * 2020-06-09 2020-09-25 海信视像科技股份有限公司 多媒体设备及投屏播放方法

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113852715A (zh) * 2021-10-13 2021-12-28 四川启睿克科技有限公司 一种快捷镜像投屏系统及方法
CN115022694A (zh) * 2022-06-27 2022-09-06 北京奇艺世纪科技有限公司 一种投屏时间统计方法、装置、系统、后台服务器及介质
CN115022694B (zh) * 2022-06-27 2023-10-27 北京奇艺世纪科技有限公司 一种投屏时间统计方法、装置、系统、后台服务器及介质

Also Published As

Publication number Publication date
US20220286731A1 (en) 2022-09-08
US12273586B2 (en) 2025-04-08

Similar Documents

Publication Publication Date Title
US11962840B2 (en) Services over wireless communication with high flexibility and efficiency
US8880098B2 (en) Communication server and session control method
US12598657B2 (en) Projection connection control method and electronic device
US9559869B2 (en) Video call handling
US8300079B2 (en) Apparatus and method for transferring video
CN102812693B (zh) 用户器件的系统
EP3043599B1 (en) Media data transmission method and device
WO2022089088A1 (zh) 显示设备、移动终端、投屏数据传输方法及传输系统
US9826021B2 (en) Communication apparatus and method for controlling the same
WO2017107470A1 (zh) 建立无线连接的方法及装置
WO2018184266A1 (zh) 直播频道中的通信方法、装置及系统
US20130219423A1 (en) Algorithmic Media Stream Selection
WO2024198710A1 (zh) 显示设备及音频控制方法
CN119653161A (zh) 显示设备及分屏播放方法
CN113691790A (zh) 多媒体数据处理方法及装置、电子设备及存储介质
US12273586B2 (en) Multimedia apparatus and cast method
CN111711843A (zh) 多媒体设备及投屏播放方法
WO2026067027A1 (zh) 显示设备和投屏方法
WO2013167069A2 (zh) 一种实现业务切换的方法、系统及中心设备
WO2023035630A1 (zh) 一种音频控制方法及显示设备
WO2019085925A1 (zh) 一种媒体内容同步方法、装置及存储介质
WO2025102986A1 (zh) 一种显示设备及多路投屏控制方法
WO2024260182A1 (zh) 一种显示设备及投屏方法
CN115705221A (zh) 显示设备和内容接力方法
CN113784187B (zh) 一种显示设备及共享资源的控制方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21771533

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21771533

Country of ref document: EP

Kind code of ref document: A1