WO2017097088A1 - Procédé et système de commande de projection à écrans-multiples, dispositif terminal, et support de stockage informatique - Google Patents

Procédé et système de commande de projection à écrans-multiples, dispositif terminal, et support de stockage informatique Download PDF

Info

Publication number
WO2017097088A1
WO2017097088A1 PCT/CN2016/105522 CN2016105522W WO2017097088A1 WO 2017097088 A1 WO2017097088 A1 WO 2017097088A1 CN 2016105522 W CN2016105522 W CN 2016105522W WO 2017097088 A1 WO2017097088 A1 WO 2017097088A1
Authority
WO
WIPO (PCT)
Prior art keywords
devices
screen
screen data
unit
projection control
Prior art date
Application number
PCT/CN2016/105522
Other languages
English (en)
Chinese (zh)
Inventor
邓敏
王柏祥
Original Assignee
努比亚技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 努比亚技术有限公司 filed Critical 努比亚技术有限公司
Publication of WO2017097088A1 publication Critical patent/WO2017097088A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/40—Network security protocols
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/643—Communication protocols
    • H04N21/6437—Real-time Transport Protocol [RTP]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/4104—Peripherals receiving signals from specially adapted client devices
    • H04N21/4122—Peripherals receiving signals from specially adapted client devices additional display device, e.g. video projector
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/4104—Peripherals receiving signals from specially adapted client devices
    • H04N21/4126—The peripheral being portable, e.g. PDAs or mobile phones
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H04N21/4363—Adapting the video stream to a specific local network, e.g. a Bluetooth® network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H04N21/4363—Adapting the video stream to a specific local network, e.g. a Bluetooth® network
    • H04N21/43637—Adapting the video stream to a specific local network, e.g. a Bluetooth® network involving a wireless protocol, e.g. Bluetooth, RF or wireless LAN [IEEE 802.11]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/44227—Monitoring of local network, e.g. connection or bandwidth variations; Detecting new devices in the local network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/12—Picture reproducers
    • H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H04N9/3141—Constructional details thereof
    • H04N9/3147—Multi-projection systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/12—Picture reproducers
    • H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H04N9/3141—Constructional details thereof
    • H04N9/3173—Constructional details thereof wherein the projection device is specially adapted for enhanced portability

Definitions

  • the present invention relates to the field of communications technologies, and more particularly to a multi-screen projection control method, system, terminal device, and computer storage medium.
  • Wi-Fi Display (WFD) network is a Wi-Fi (Wireless-Fidelity) alliance recommended to enable Wi-Fi devices to connect to each other in a peer-to-peer manner without participating in home networks or office networks. Or a network system of hotspot networks.
  • Wi-Fi Wireless-Fidelity
  • the existing peer-to-peer (P2P, Peer to Peer) Wi-Fi Display (WFD) technology enables one-to-one screen sharing between devices.
  • the screen sharing in the prior art is limited to "one-on-one" sharing, which is a one-way sharing and passive communication.
  • Embodiments of the present invention are directed to providing a multi-screen projection control method, system, terminal device, and computer storage medium.
  • an embodiment of the present invention provides a multi-screen projection control method, including:
  • the first device sends a discovery request
  • the first device accepts one or more discovery responses, the discovery response being sent from one or more second devices in response to the received discovery request;
  • the first device performs screen data transmission based on the WFD connection and the one or more second devices.
  • the first device initiates one or more process services to process a screen data transfer process with the one or more second devices, respectively.
  • the first device allocates an Internet Protocol (IP) address to each of the one or more second devices, and assigns the assigned IP address to the second device.
  • IP Internet Protocol
  • the correspondence of the devices is stored in an IP list.
  • the method further includes:
  • the first device displays the received screen data.
  • the method further includes:
  • the first device sends the received screen data to other second devices by inter-process communication according to the IP list.
  • the method further includes:
  • the first device performs a corresponding operation according to the input signal.
  • an embodiment of the present invention further provides a multi-screen projection control method, where the method includes:
  • the second device establishes a WFD connection with the first device based on the discovery response
  • the second device receives screen data sent by the first device based on the WFD connection.
  • the method further includes:
  • the second device detects an input signal, and sends the input signal to the first device, so that the first device performs a corresponding operation according to the input signal;
  • the input signal includes a coordinate value of the converted input point and an input duration
  • the conversion includes:
  • the second device includes an input unit
  • the first preset value is a width of the input unit; and the second preset value is a length of the input unit.
  • the embodiment of the present invention further provides a terminal device, where the terminal device is a first device, and includes:
  • Detecting a connection unit configured to send a discovery request, and configured to receive a discovery response to establish a WFD connection with one or more second devices;
  • control unit configured to control, by the screen connection of the display unit, the WFD connection established by the probe connection unit to the one or more second devices.
  • the probe connection unit is further configured to receive screen data
  • the control unit is further configured to control the display unit to display the received screen data.
  • the probe connection unit is further configured to receive an input signal sent by any one of the one or more second devices;
  • the control unit is further configured to perform a corresponding operation in accordance with the input signal.
  • the embodiment of the present invention further provides a terminal device, where the terminal device is Two devices; including:
  • a second probe connection unit configured to receive a discovery request, and configured to send a discovery response to establish a WFD connection with the first device; and configured to receive screen data sent by the first device;
  • a second control unit configured to control the display unit to display screen data received by the second probe connection unit.
  • the terminal device further includes an input unit configured to detect an input signal
  • the second detecting connection unit is configured to send the input signal to the first device, so that the first device performs a corresponding operation according to the input signal;
  • the input signal includes a coordinate value of the converted input point and an input duration
  • the conversion includes:
  • an embodiment of the present invention further provides a multi-screen projection control system, including:
  • a first device configured to send a discovery request
  • One or more second devices configured to send a discovery response in response to the received discovery request
  • the first device is further configured to establish a WFD connection with the one or more second devices according to the discovery response, and perform screen data transmission between the WFD connection and the one or more second devices. .
  • the first device is further configured to initiate one or more process services to process a screen data transfer process with the one or more second devices, respectively.
  • the first device is further configured to be the one or more second devices
  • Each of the IP addresses is assigned an IP address, and the corresponding relationship between the assigned IP address and the second device is stored in an IP list.
  • any one of the one or more second devices transmits its own screen data to the first device
  • the first device is further configured to receive and display the received screen data, and configured to transmit the received screen data to the other second device by inter-process communication according to the IP list.
  • the embodiment of the present invention further provides a computer storage medium, where the computer storage medium stores computer executable instructions, where the computer executable instructions are configured to be applied to the first device in the embodiment of the present invention.
  • Multi-screen projection control method Multi-screen projection control method
  • the computer storage medium stores computer executable instructions configured to perform a multi-screen projection control method applied to the second electronic device in the embodiment of the present invention.
  • a multi-screen projection control method, system, terminal device and computer storage medium embodying embodiments of the present invention have the following beneficial effects: one-to-many screen projection and sharing can be realized, and the dependence on the location can be reduced through the WFD network It can be applied to scenarios such as multimedia conference rooms or audio-visual function classrooms to realize rapid establishment of WFD network and resource sharing; and multi-screen interaction can be realized, and any device can share through role switching; Through the split-screen display, it is convenient to manage the local interface and improve the user experience; and multi-screen interaction can be realized by using a very common terminal device, which greatly reduces the cost.
  • FIG. 1 is a schematic structural diagram of hardware of a terminal device according to an embodiment of the present invention.
  • FIG. 2 is a search flow chart of a projection side device according to an embodiment of the present invention.
  • FIG. 3 is a schematic diagram of a network structure capable of implementing multi-screen projection control according to an embodiment of the present invention
  • FIG. 4 is a projection side device for opening a plurality of processes for screen data transmission according to an embodiment of the present invention. Schematic diagram of control;
  • FIG. 5 is a schematic diagram of a receiving side device receiving screen data and displaying it according to an embodiment of the present invention
  • FIG. 6 is a schematic diagram of a receiving side device controlling a projection side device according to an embodiment of the present invention.
  • Figure 7 is a schematic diagram of the control effect shown in Figure 6;
  • FIG. 8 is a schematic diagram showing a display effect after a split screen is adopted on a receiving side according to an embodiment of the present invention.
  • FIG. 9 is a schematic diagram of a screen data transmission effect after the device B on the receiving side is switched to the projection side device according to an embodiment of the present invention.
  • FIG. 10 is a schematic diagram of an interaction interface of a terminal device according to an embodiment of the present invention.
  • FIG. 11 is a flowchart of a multi-screen projection control method according to an embodiment of the present invention.
  • FIG. 12 is a schematic structural diagram of a multi-screen projection control system according to an embodiment of the present invention.
  • FIG. 1 is a schematic structural diagram of hardware of a terminal device according to an embodiment of the present invention.
  • the terminal device 100 of the embodiment of the present invention includes: a probe connection unit 101, a control unit 102, and a display unit 103.
  • the terminal device 100 may be a mobile terminal with a display screen, such as a mobile phone, a pad, a notebook, or the like.
  • the probe connection unit 101 is configured to perform device discovery to search for neighboring other devices available to the Wi-Fi display service. When other devices are found, the probe connection unit 101 interacts with other devices through the RTSP protocol and negotiates capabilities to connect the device 100 with other devices.
  • the control unit 102 controls basic operations within the terminal device 100. Specifically, the control unit 102 performs control such that the content (screen data) displayed by the display unit 103 is based on a real-time transmission protocol. (RTP, Real-time Transport Protocol) is transmitted to other devices through the probe connection unit 101.
  • the control unit 102 determines the input signal form from the user through the display unit 103 to control the terminal device 100 in accordance with the input signal. For example, a device search or the like is initiated based on an input signal.
  • the probe connection unit 101 is configured to receive a discovery request and configured to transmit a discovery response to establish a WFD connection with the projection side device; and is further configured to establish a connection with the device on the projection side to receive the screen data.
  • the control unit 102 is also configured to perform control such that the display unit 103 displays the received screen data.
  • the display unit, the detection connection unit, and the control unit may be referred to as a second display unit, a second detection connection unit, and a second control unit, as being different from the projection side terminal device.
  • the search process of the embodiment of the present invention includes:
  • the projection side device performs device discovery for finding neighboring external devices that support Wi-Fi Display.
  • the projection side device sets a connection with the device, performs switching capability, and performs negotiation.
  • Playback step The device uses the Real Time Streaming Protocol (RTSP) protocol to negotiate the capability and match the relevant parameters.
  • RTSP Real Time Streaming Protocol
  • the current screen data of the projection side device is first packaged into the video and audio stream, and then the RTP is passed.
  • the protocol directly transmits the audio stream packet to the receiving side device, and the screen data on the projection side is transmitted to the receiving side by a stream type (audio/video (AV) stream).
  • RTSP belongs to an application layer protocol in the TCP/IP protocol system and is mainly responsible for device capability negotiation.
  • FIG. 3 is a schematic structural diagram of a network capable of implementing multi-screen projection control according to an embodiment of the present invention.
  • the network 1000 includes: one projection side device 200 and at least one reception side device 300.
  • the hardware configuration of the projection side device 200 and the reception side device 300 can be referred to the above FIG.
  • Network 1000 has the same meaning as a Wi-Fi peer-to-peer (P2P) network.
  • the projection side device and the reception side device in the network 1000 are directly connected to each other through Wi-Fi Direct without using a wireless local area network (WLAN) access point (AP) or the like.
  • WLAN wireless local area network
  • the projection side device 200 performs device discovery in the network 1000 environment to discover the receiving side device 300.
  • the projection side device 200 transmits a probe request frame on a predefined channel.
  • the probe request frame includes information such as a peer-to-peer (P2P) information element (IE), a P2P wildcard SSID, a wildcard SSID, a destination address, a device type, and a device ID.
  • P2P peer-to-peer
  • IE peer-to-peer
  • P2P wildcard SSID a P2P wildcard SSID
  • a wildcard SSID a destination address
  • a device type a device ID
  • the receiving side device 300 in the network 1000 environment listens to the probe request frame on a predefined channel. Each device determines that the information in the probe request frame matches the search criteria. If a match is found, the receiving side device 300 transmits a probe request frame having the device description to the projection side device 200.
  • the projection side device 200 After the device discovery is completed, the projection side device 200 performs service discovery to obtain a service supported by the reception side device 300. During the service discovery, the projection side device 200 transmits a GAS initial request to the receiving side device 300. In response, the receiving side device 300 transmits the GAS initial response with the service information to the projection side device 200. Thereafter, the projection side device 200 establishes a WFD connection with the reception side device 300 using the device and service description obtained from the reception side device 300, that is, the device connection establishment in FIG.
  • each receiving side device 300 in the embodiment of the present invention establishes a connection with the projection side device in the above manner.
  • the projection side device 200 and the plurality of receiving side devices 300 can be referred to as one group.
  • the projection side device 300 is designated as a group owner (GO).
  • the receiving side device 300 functions as a group client.
  • device A is a projection side device, and device B, device C, and device D are both receiving. Side equipment. Device B, device C, and device D are connected to device A through Wireless.
  • Device A starts multiple process services, and each process manages communication with one receiving device separately.
  • the sub-process 1 corresponds to the communication connection data of the processing device B
  • the sub-process 2 corresponds to the communication connection data of the processing device C
  • the sub-process 3 corresponds to the communication connection data of the processing device D.
  • device A also starts a parent process for listening connections and so on.
  • each process is transmitted in accordance with the WFD protocol.
  • device A assigns an IP address to each device, and stores the correspondence between each receiving device and its IP address in an IP list.
  • any device connected to the device A can access other devices through the IP list, establish a TCP/IP connection, and implement two-way communication, so that any device can display the screen.
  • Data is passed to other devices over a TCP/IP link.
  • the screen data transmission process between the entire group includes:
  • the projection side device listens for a connection request from another device according to the discovery process of the receiving side device that performs the above discovery. And according to the discovered device, select the specified device to establish a connection.
  • the projection side device creates a plurality of sub-processes to process the bidirectional data transmission with each device separately.
  • the projection side device uses the underlying Wireless link to package the current screen video and audio data according to the process service created when the connection is established, and transmits it to all receiving side devices in combination with the RTP protocol. Specifically, the projection side device captures the current screen data every preset time (for example, 30 ms), packages the video into the video and audio stream, and sends the screen data to the receiving device according to the RTP protocol.
  • preset time for example, 30 ms
  • the receiving side device is mainly responsible for displaying the screen data transmitted by the projection side device.
  • Receiving side device The received media data stream containing the screen data is cached and displayed. Referring to FIG. 5, after the screen data starts to be transmitted, the devices B, C, and D receive the screen data of the device A and display it.
  • the projection side device manages the list of devices that establish the connection to ensure information security and prevent irrelevant devices from joining, and also manages the joining of new devices in the middle or the exit of devices in the link.
  • the multi-screen projection control method of the embodiment of the invention can realize one-to-many screen projection and sharing, and can reduce the dependence on the location through the WFD network; and can be applied to a specific demand scene such as a multimedia conference room or an audio-visual function classroom. Quickly set up a WFD network to achieve resource sharing.
  • the receiving side device can control the projection side device in addition to displaying the received screen data.
  • the receiving device when the terminal device is the receiving device, the receiving device further includes: an input unit 104 configured to generate an input signal according to an operation of the user, and transmit the input signal to the control unit.
  • the input signal includes information such as the coordinates of the input point, duration, and the like.
  • the control unit detects an input signal generated by the input unit.
  • the control unit converts the coordinates of the input points contained in the input signal according to the physical parameters of the input unit. For example, if the input unit is a touch screen with a width of W0 and a height of H0, and the coordinates of the input point are (X0, Y0), the coordinates of the input point are converted to (X0/W0, Y0/H0), that is, it is converted into a The ratio of the input unit in the width direction and the length direction. In addition, conversion can also be performed in other ways.
  • the control unit controls the input signal such as the converted coordinates and the duration to be transmitted to the projection side device through the probe connection unit.
  • the coordinates of the input point are converted into coordinates adapted to the input unit of the projection side device. For example, if the projection side device is of width W1, the touch screen with height H1, the received coordinates are (X0/W0, Y0/H0), and the coordinates of the input point of the receiving device correspond to the coordinates of the projection side device ((X0/W0) ⁇ W1, ( Y0/H0) ⁇ H1).
  • the projection side device can recognize the input point when the input point of the receiving side device corresponds to the projection side device according to the coordinate, and perform a corresponding operation according to the input point.
  • the input point is 600.
  • the corresponding input point is 601. If the command corresponding to the input point 601 is: "Content 1" is turned on, when the user operates at the device B, "Content 1" in the device A is turned on, see the effect diagram shown in FIG.
  • Manner 1 The projection side device responds to the input signal of the receiving side device that is received first according to the timing.
  • Manner 2 setting the priority, if receiving multiple input signals at the same time, responding to the input signal of the receiving device with the highest priority or higher.
  • Method 3 Set a fixed projection side device to respond only to its input signal.
  • the projection side device by transmitting the input signal received by the receiving side device to the projection side device, the projection side device performs a matching response, and performs a corresponding operation, so that the receiving side device can control the projection side device, thereby improving Interactive control efficiency and enhance user experience.
  • Embodiment 1 or 2 On the basis of the above-mentioned Embodiment 1 or 2, in order to enable the receiving side device to receive and display the screen data of the projection side device, the free control of the machine can be realized, and the screen data and the local interface are separately displayed by using the split screen. .
  • the split screen is to divide the display screen of the receiving side device into the first display area and the second display area, and correspondingly divide the input unit into an input area corresponding to the first display area and an input corresponding to the second display area. Zone, this is prior art and will not be described here. In this embodiment of the invention, any display area in the split screen is used to display the received screen data on the basis of the split screen.
  • device B, device C, and device D both use any display area for displaying the screen data of the received device A, and another display area is used to display the interface of the device. Thereby, the most native free control can be realized through the local interface without interrupting the screen projection in order to achieve control of the local machine.
  • the receiving side device adopts the split screen mode
  • the control of the projection side device by the receiving side device in Embodiment 2 it is first determined whether the input point is located in the display area where the local interface is located, or is the screen data display. The display area where it is located. If the display data is in the display area where the screen data is located, the coordinates of the input point are converted and sent to the projection side device.
  • the conversion can include:
  • the coordinates of the input points are expanded according to the split screen ratio. For example, if the display screen of the receiving device is split up and down, and the first display area and the second display area are equal in size, if the coordinates of the touch point are (xc, yc), the coordinates are expanded to (2) ⁇ xc, 2 ⁇ yc).
  • the enlarged coordinates are converted into the ratio of the input unit in the width direction and the length direction, and then sent to the projection side device.
  • the projection side device can correctly recognize the input operation and respond.
  • the split screen display facilitates management of the local interface and improves the user experience.
  • the receiving side device can be converted into a projection side device by switching to share the screen data to other members.
  • the screen data is first suspended, and then the screen data is sent to the projection side device by using the underlying communication link.
  • the projection side device transmits the screen data to other receiving side devices through inter-process communication. Thereby, sharing the screen data of any receiving side device to other members in the group is achieved.
  • FIG. 9 an embodiment is shown. After the device B is switched to the projection side device, the screen data is sent to the device A, and the device A displays the screen data, and sends the screen data of the device B to the device C through the communication of the process. Device D is displayed. Thus, device B implements projection of its own screen data.
  • a switch button (which may be a virtual button or a physical button) may be provided, and the role can be converted by operating the button. Specifically, when the button is operated to perform role conversion, the receiving or sending of the screen data is suspended, and after the role is successfully converted, the screen data is continuously received or transmitted.
  • the embodiment of the present invention performs role switching by keeping the links in communication, which not only realizes switching of screen data sharing, but also shortens the switching time in the screen mutual investment process, and effectively improves the switching time. Efficiency and participation.
  • the embodiment of the invention further provides a terminal device having a hardware structure as shown in FIG. 1 .
  • the control device is configured to:
  • the screen data of the display unit is transmitted to the one or more second devices by the WFD connection established by the probe connection unit.
  • the probe connection unit is further configured to receive screen data; the control unit is further configured to control the display unit to display the received screen data.
  • the display unit includes a first display area and a second display area; the received screen data is displayed by the first display area or the second display area.
  • the probe connection unit is further configured to receive an input signal transmitted by any one of the one or more second devices; the control unit is further configured to perform a corresponding operation according to the input signal Wherein each of the one or more second devices includes an input unit configured to generate an input signal in accordance with a user operation.
  • the terminal device can function as a projection side device or as a receiving side device.
  • the screen projection function When the screen projection function is turned on, the user can see the interactive interface shown on the left side of FIG. 10 through the display unit.
  • the interactive interface jumps to the interface on the upper right side, and the terminal device acts as a projection side device, and performs a device discovery step to find other external devices supporting Wi-Fi display.
  • the device list is displayed, and a connection is established with the selected user device according to the user's selection, and the exchange capability is performed using RTSP and negotiation is performed.
  • the terminal device After the connection is established, the terminal device directly transmits the screen data of the projection side device to the receiving side device after the RTSP play command; when receiving the response from the receiving side, starts the audio/video (AV) stream for mirroring, and the projection is performed.
  • the screen data of the side is transmitted to the receiving side device by the stream type.
  • the terminal device acts as a receiving side device, waits for the projection party to initiate an invitation, and when the invitation is received, that is, after establishing a connection with the projection side, the screen data sent by the projection side can be received.
  • the terminal device of the embodiment of the invention can realize one-to-many screen projection and sharing, and can reduce the dependence on the location through the WFD network; and can be applied to a specific demand scene such as a multimedia conference room or an audio-visual function classroom, and realizes fast
  • the WFD network is set up to realize resource sharing; and multi-screen interaction can be realized. Any device can share through role switching; on the other hand, it is convenient to manage the local interface and improve user experience through split screen display; The terminal device can realize multi-screen interaction, which greatly reduces the cost.
  • the control unit in the terminal device may be used by a central processing unit (CPU, Central Processing) in the terminal device in an actual application. Unit), a digital signal processor (DSP), a Microcontroller Unit (MCU), or a Field-Programmable Gate Array (FPGA); the display unit in the terminal device
  • the actual application may be implemented by a display or a display screen in the terminal device
  • the detection connection unit in the terminal device may be a communication module in the terminal device in actual application (including: a basic communication suite, an operating system, Communication module, standardized interface and protocol, etc.) and transceiver antenna implementation.
  • the multi-screen projection control method of the embodiment of the invention comprises the following steps:
  • the first device sends a discovery request.
  • the discovery request may be a WFD discovery request.
  • the first device is subjected to one or more discovery responses.
  • the discovery response is sent from one or more second devices in response to the received discovery request.
  • the discovery response can be a WFD discovery response.
  • the first device initiates one or more process services to process a screen data transmission process with the one or more second devices, respectively.
  • the first device allocates an Internet Protocol (IP) address to each of the one or more second devices, and assigns the assigned IP address to the second device.
  • IP Internet Protocol
  • S103 Perform transmission of screen data between the first device and the one or more second devices.
  • the first device is equivalent to a projection side device
  • the second device is equivalent to a receiving side device.
  • the first device can be converted into a second device, that is, converted into a receiving side device
  • the second device can be converted into a first device, that is, converted into a projection side device, which will be described later.
  • the multi-screen projection control method further includes: the first device receiving screen data sent by any one of the one or more second devices; the first device receiving and displaying Received screen data.
  • the first device further sends the received screen data to the other second device by inter-process communication according to the IP list.
  • the receiving side device can be converted into a projection side device, and the flexible switching of the role can be realized.
  • a switch button (which may be a virtual button or a physical button) may be provided, and the role can be converted by operating the button. Specifically, when the button is operated to perform role conversion, the receiving or sending of the screen data is suspended, and after the role is successfully converted, the screen data is continuously received or transmitted.
  • the multi-screen projection control method further includes: the first device receiving an input signal detected by any one of the one or more second devices; the first device Perform the corresponding operation according to the input signal.
  • the input signal includes coordinate values of the converted input point and an input duration.
  • the conversion includes: converting the abscissa of the input point to a ratio of the abscissa to the first preset value; converting the ordinate of the input point to a ratio of the ordinate to the second preset value.
  • the first preset value is the width of the input unit; the second preset value is the length of the input unit.
  • the input unit can be a touch screen that includes a touch panel and a touch controller. Further, the input unit may be a non-touch input unit (for example, an infrared input unit or the like), a key input unit, or the like.
  • each of the one or more second devices includes a display screen.
  • the display screen includes a first display area and a second display area, whereby the received screen data can be displayed through the first display area or the second display area, and the other display area is used to display its own interface. Thereby, it is possible to control the own device while receiving the screen data.
  • the multi-screen projection control method of the embodiment of the invention can realize one-to-many screen projection and division Enjoy, through the WFD network, reduce the dependence on the location; can be applied to the specific needs of the multimedia conference room or audio-visual function classroom, to achieve rapid establishment of the WFD network, to achieve resource sharing; and multi-screen interaction, through role switching Any device can share; on the other hand, through the split screen display, it is convenient to manage the local interface and improve the user experience; and the multi-screen interaction can be realized by using a very common terminal device, which greatly reduces the cost.
  • an embodiment of the present invention further provides a multi-screen projection control system.
  • the system includes:
  • the first device 700 is configured to send a discovery request.
  • the one or more second devices 701 are configured to send a discovery response responsive to the received discovery request.
  • the first device 700 is further configured to establish a WFD connection with the one or more second devices 701 according to the discovery response to perform transmission of screen data with the one or more second devices 701.
  • the first device 700 is further configured to initiate one or more process services to process a screen data transfer process with the one or more second devices 701, respectively.
  • the first device 700 is further configured to allocate an IP address to each of the one or more second devices 701, and store the corresponding relationship between the assigned IP address and the second device in an IP list.
  • Any one of the one or more second devices 701 transmits its own screen data to the first device.
  • the first device 700 is further configured to receive and display the received screen data, and is configured to transmit the received screen data to the other second device 701 by inter-process communication according to the IP list.
  • the second device of the one or more second devices 701 transmits the detected input signal to the first device 700; the first device 700 performs a corresponding operation according to the input signal.
  • Each of the one or more second devices 701 includes a display screen; the display screen includes a first display area and a second display area; one or more second devices The received screen data is displayed through the first display area or the second display area.
  • the multi-screen projection control system of the embodiment of the invention can realize one-to-many screen projection and sharing, and can reduce the dependence on the location through the WFD network; and can be applied to a specific demand scene such as a multimedia conference room or an audio-visual function classroom.
  • a specific demand scene such as a multimedia conference room or an audio-visual function classroom.
  • any device can share through role switching; on the other hand, through split screen display, it is convenient to manage the local interface and improve user experience; Multi-screen interaction can be achieved with very popular terminal equipment, which greatly reduces the cost.
  • the terminal device, the first device, and the second device of the embodiments of the present invention may be implemented in various forms.
  • the device described in the present invention may include, for example, a mobile phone, a mobile phone, a smart phone, a notebook computer, a digital broadcast receiver, a PDA (Personal Digital Assistant), a PAD (Tablet), a PMP (Portable Multimedia Player), a navigation device Mobile devices such as the like and fixed devices such as digital TVs, desktop computers, and the like.
  • the disclosed apparatus and method may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the unit is only a logical function division.
  • there may be another division manner such as: multiple units or components may be combined, or Can be integrated into another system, or some features can be ignored or not executed.
  • the coupling, or direct coupling, or communication connection of the components shown or discussed may be indirect coupling or communication connection through some interfaces, devices or units, and may be electrical, mechanical or other forms. of.
  • the units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place.
  • the party may also be distributed to multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
  • each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may be separately used as one unit, or two or more units may be integrated into one unit;
  • the unit can be implemented in the form of hardware or in the form of hardware plus software functional units.
  • the foregoing program may be stored in a computer readable storage medium, and the program is executed when executed.
  • the foregoing storage device includes the following steps: the foregoing storage medium includes: a mobile storage device, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.
  • ROM read-only memory
  • RAM random access memory
  • magnetic disk or an optical disk.
  • optical disk A medium that can store program code.
  • the above-described integrated unit of the present invention may be stored in a computer readable storage medium if it is implemented in the form of a software function module and sold or used as a standalone product.
  • the technical solution of the embodiments of the present invention may be embodied in the form of a software product in essence or in the form of a software product stored in a storage medium, including a plurality of instructions.
  • a computer device (which may be a personal computer, server, or network device, etc.) is caused to perform all or part of the methods described in various embodiments of the present invention.
  • the foregoing storage medium includes various media that can store program codes, such as a mobile storage device, a ROM, a RAM, a magnetic disk, or an optical disk.
  • the embodiment of the invention can realize one-to-many screen projection and sharing, and can reduce the dependence on the location through the WFD network; and can be applied to a specific demand scene such as a multimedia conference room or an audio-visual function classroom to realize rapid establishment of a WFD network. Realize resource sharing; and realize multi-screen interaction. Any role can be shared through role switching. On the other hand, through split-screen display, it is convenient to manage the local interface and improve user experience; and the use of very popular terminal devices Multi-screen interaction can be realized, which greatly reduces the cost.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Computer Security & Cryptography (AREA)
  • Human Computer Interaction (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Information Transfer Between Computers (AREA)
  • Controls And Circuits For Display Device (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

L'invention concerne un procédé et un système de commande de projection à écrans multiples, un dispositif terminal, et un support de stockage informatique. Le procédé comprend : un premier dispositif qui envoie une requête de découverte; le premier dispositif reçoit une ou plusieurs réponse(s) de découverte, les réponses de découverte étant envoyées à partir d'un ou plusieurs second(s) dispositif(s) en réponse à la demande de découverte reçue; le premier dispositif établit une connexion d'affichage Wi-Fi (WFD) avec ledit ou lesdits second(s) dispositif(s); et le premier dispositif transmet des données d'écran audit ou auxdits second(s) dispositif(s) sur la base de la connexion WFD.
PCT/CN2016/105522 2015-12-07 2016-11-11 Procédé et système de commande de projection à écrans-multiples, dispositif terminal, et support de stockage informatique WO2017097088A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510896486.6A CN105578227B (zh) 2015-12-07 2015-12-07 多屏投影控制方法、系统及终端设备
CN201510896486.6 2015-12-07

Publications (1)

Publication Number Publication Date
WO2017097088A1 true WO2017097088A1 (fr) 2017-06-15

Family

ID=55887821

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/105522 WO2017097088A1 (fr) 2015-12-07 2016-11-11 Procédé et système de commande de projection à écrans-multiples, dispositif terminal, et support de stockage informatique

Country Status (2)

Country Link
CN (1) CN105578227B (fr)
WO (1) WO2017097088A1 (fr)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108965995A (zh) * 2018-07-17 2018-12-07 北京奇艺世纪科技有限公司 一种投屏方法、装置、投屏端设备及被投屏端设备
CN110244927A (zh) * 2019-07-03 2019-09-17 明基智能科技（上海）有限公司 一种智能显示系统及装置
CN114691067A (zh) * 2020-12-31 2022-07-01 华为技术有限公司 投屏方法及电子设备
US12028300B2 (en) 2020-05-29 2024-07-02 Huawei Technologies Co., Ltd. Method, apparatus, and system for sending pictures after thumbnail selections

Families Citing this family (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105578227B (zh) * 2015-12-07 2019-08-16 努比亚技术有限公司 多屏投影控制方法、系统及终端设备
CN105892977A (zh) * 2016-05-30 2016-08-24 乐视控股（北京）有限公司 一种多屏系统交互方法、装置及智能电视
CN106371784A (zh) * 2016-08-29 2017-02-01 武汉启目科技有限公司 一种将智能终端得到的屏幕分享给其他终端的方法
CN106354461A (zh) * 2016-08-29 2017-01-25 武汉启目科技有限公司 一种android终端屏幕可相互无线分享的方法
CN107801088A (zh) * 2016-08-31 2018-03-13 南京极域信息科技有限公司 一种投屏接收分享系统及实现方法
CN107885558A (zh) * 2016-09-29 2018-04-06 努比亚技术有限公司 一种投影数据处理装置、方法及投影数据分享设备
CN108289340A (zh) * 2016-09-29 2018-07-17 努比亚技术有限公司 一种无线保真投影的设备、系统及方法
CN108289241B (zh) * 2016-09-29 2021-05-21 努比亚技术有限公司 一种无线投影装置、系统及方法
CN108289289A (zh) * 2016-09-29 2018-07-17 努比亚技术有限公司 一种投影数据的传输设备、系统及方法
CN108289341A (zh) * 2016-09-29 2018-07-17 努比亚技术有限公司 一种音频处理装置以及方法
CN107886001A (zh) * 2016-09-29 2018-04-06 努比亚技术有限公司 一种投影数据处理装置、方法及投影数据分享设备
CN106375843A (zh) * 2016-09-29 2017-02-01 努比亚技术有限公司 一种无线投影装置、系统及方法
CN106507072A (zh) * 2016-10-11 2017-03-15 努比亚技术有限公司 一种无线投影装置、系统及方法
EP3488317B1 (fr) * 2016-10-19 2022-07-06 Hewlett-Packard Development Company, L.P. Partage d'écran
CN106572518A (zh) * 2016-10-31 2017-04-19 努比亚技术有限公司 一种信息处理方法及移动终端
CN106528019A (zh) * 2016-10-31 2017-03-22 努比亚技术有限公司 一种信息处理方法及终端
CN106528016A (zh) * 2016-10-31 2017-03-22 努比亚技术有限公司 一种信息处理方法及终端
CN106358319A (zh) * 2016-11-03 2017-01-25 努比亚技术有限公司 一种无线投影装置、系统及方法
CN108289116A (zh) * 2017-09-13 2018-07-17 深圳银澎云计算有限公司 一种设备间的控制管理方法与装置
JP6528820B2 (ja) * 2017-09-19 2019-06-12 カシオ計算機株式会社 コンピュータ、情報機器、これらを動作させるプログラム、情報処理システム、及び情報処理システムの制御方法
CN108683731B (zh) * 2018-05-14 2021-09-21 浙江口碑网络技术有限公司 一种管理多屏互动的方法、装置
CN109246845B (zh) * 2018-10-25 2020-11-03 Oppo广东移动通信有限公司 一种连接方法、终端及计算机可读存储介质
CN111221483B (zh) * 2018-11-26 2023-07-04 中强光电股份有限公司 投影机系统及投影机设定方法
CN110267084B (zh) * 2019-07-25 2021-11-19 北京奇艺世纪科技有限公司 一种同步信号处理方法以及相关设备
CN113709555B (zh) * 2021-08-27 2023-03-21 联想(北京)有限公司 一种无线视频连接方法、装置以及系统

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103150132A (zh) * 2013-02-21 2013-06-12 海信集团有限公司 一种传屏方法及电子设备
CN103298068A (zh) * 2012-02-24 2013-09-11 三星电子株式会社 用于发现在无线通信网络中的装置的方法和设备
US20130346562A1 (en) * 2012-06-20 2013-12-26 Samsung Electronics Co., Ltd. Data transmission method and apparatus for mobile terminal
US20140215359A1 (en) * 2013-01-25 2014-07-31 Hon Hai Precision Industry Co., Ltd. Desktop sharing method and system
CN104219551A (zh) * 2014-09-29 2014-12-17 武汉诚迈科技有限公司 一种智能设备屏幕共享与远程控制的方法
US20140372620A1 (en) * 2013-06-18 2014-12-18 Samsung Electronics Co., Ltd. Method and apparatus for controlling content shared between devices in wireless communication system
CN105578227A (zh) * 2015-12-07 2016-05-11 努比亚技术有限公司 多屏投影控制方法、系统及终端设备
CN105739937A (zh) * 2016-01-27 2016-07-06 努比亚技术有限公司 一种移动终端投影方法、终端设备及系统

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130027289A1 (en) * 2011-07-27 2013-01-31 Lg Electronics Inc. Electronic device
KR101286358B1 (ko) * 2011-08-11 2013-07-15 엘지전자 주식회사 영상표시방법 및 장치
US9294883B2 (en) * 2012-03-01 2016-03-22 Nokia Technologies Oy Method, apparatus, and computer program product for probe request and response exchange
KR102001215B1 (ko) * 2012-07-20 2019-07-17 삼성전자주식회사 컨텐트 공유 방법 및 시스템, 그 기기 및 기록 매체
CN103582165B (zh) * 2012-08-07 2017-05-31 中国移动通信集团公司 连接建立方法、设备及系统、数据传输方法、设备及系统
CN104618793B (zh) * 2015-02-28 2018-11-09 联想(北京)有限公司 一种信息处理方法及电子设备

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103298068A (zh) * 2012-02-24 2013-09-11 三星电子株式会社 用于发现在无线通信网络中的装置的方法和设备
US20130346562A1 (en) * 2012-06-20 2013-12-26 Samsung Electronics Co., Ltd. Data transmission method and apparatus for mobile terminal
US20140215359A1 (en) * 2013-01-25 2014-07-31 Hon Hai Precision Industry Co., Ltd. Desktop sharing method and system
CN103150132A (zh) * 2013-02-21 2013-06-12 海信集团有限公司 一种传屏方法及电子设备
US20140372620A1 (en) * 2013-06-18 2014-12-18 Samsung Electronics Co., Ltd. Method and apparatus for controlling content shared between devices in wireless communication system
CN104219551A (zh) * 2014-09-29 2014-12-17 武汉诚迈科技有限公司 一种智能设备屏幕共享与远程控制的方法
CN105578227A (zh) * 2015-12-07 2016-05-11 努比亚技术有限公司 多屏投影控制方法、系统及终端设备
CN105739937A (zh) * 2016-01-27 2016-07-06 努比亚技术有限公司 一种移动终端投影方法、终端设备及系统

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108965995A (zh) * 2018-07-17 2018-12-07 北京奇艺世纪科技有限公司 一种投屏方法、装置、投屏端设备及被投屏端设备
CN110244927A (zh) * 2019-07-03 2019-09-17 明基智能科技（上海）有限公司 一种智能显示系统及装置
US12028300B2 (en) 2020-05-29 2024-07-02 Huawei Technologies Co., Ltd. Method, apparatus, and system for sending pictures after thumbnail selections
CN114691067A (zh) * 2020-12-31 2022-07-01 华为技术有限公司 投屏方法及电子设备
WO2022143087A1 (fr) * 2020-12-31 2022-07-07 华为技术有限公司 Procédé de projection sur écran et dispositif électronique

Also Published As

Publication number Publication date
CN105578227A (zh) 2016-05-11
CN105578227B (zh) 2019-08-16

Similar Documents

Publication Publication Date Title
WO2017097088A1 (fr) Procédé et système de commande de projection à écrans-multiples, dispositif terminal, et support de stockage informatique
CN105739937B (zh) 一种移动终端投影方法、终端设备及系统
EP3430824B1 (fr) Procédés et systèmes de gestion d'une connectivité entre dispositifs
TWI536855B (zh) 內容分享方法與裝置
WO2018045561A1 (fr) Procédé de transmission de fichier, procédé de réception de fichier et terminal
US8880098B2 (en) Communication server and session control method
EP2858322B1 (fr) Procédé et terminal pour partager un flux de média
US10264038B2 (en) Discovery and management of synchronous audio or video streaming service to multiple sinks in wireless display system
WO2020134720A1 (fr) Procédé de commande de transmission de données et produit associé
US20100330910A1 (en) Method for transmitting and receiving data using e-field in portable terminal and apparatus thereof
CN104219551A (zh) 一种智能设备屏幕共享与远程控制的方法
KR20110048457A (ko) Ａｖ 시스템에서 사용자 입력 백 채널을 제공하는 방법 및 기기
WO2019015427A1 (fr) Procédé et dispositif de transmission de données, terminal et support de stockage lisible par ordinateur
WO2016110169A1 (fr) Procédé et dispositif de traitement d'affichage
CN103856374A (zh) 一种数据传输方法、终端及家庭互联系统
EP2936735A1 (fr) Accueil sans fil de multiples utilisateurs
CN103858406A (zh) 媒体数据传输方法及设备
TWI540903B (zh) 視訊通話轉換方法、行動終端及智慧電視
US20220286731A1 (en) Multimedia apparatus and cast method
WO2013166762A1 (fr) Procédé et système de réseautage pour des dispositifs de réseau personnel
WO2015062500A1 (fr) Télécommande, et télévision intelligente
WO2017092323A1 (fr) Dispositif de commande principal, dispositif de lecture et procédé de transmission de données associé
CN112770078B (zh) 一种视频会议终端融合方法、装置及可读存储介质
US20240089526A1 (en) Method for playback in split screen and display device
CN105791965B (zh) 一种屏幕共享中设备连接处理方法、系统和相关设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16872280

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16872280

Country of ref document: EP

Kind code of ref document: A1