WO2022028189A1 - Screen projection method and apparatus, electronic device, and computer readable storage medium - Google Patents

Screen projection method and apparatus, electronic device, and computer readable storage medium Download PDF

Info

Publication number
WO2022028189A1
WO2022028189A1 PCT/CN2021/104752 CN2021104752W WO2022028189A1 WO 2022028189 A1 WO2022028189 A1 WO 2022028189A1 CN 2021104752 W CN2021104752 W CN 2021104752W WO 2022028189 A1 WO2022028189 A1 WO 2022028189A1
Authority
WO
WIPO (PCT)
Prior art keywords
client
server
search
response message
file
Prior art date
Application number
PCT/CN2021/104752
Other languages
French (fr)
Chinese (zh)
Inventor
朱云龙
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Priority to JP2022572724A priority Critical patent/JP2023527393A/en
Priority to US18/019,853 priority patent/US20230333800A1/en
Priority to EP21853925.2A priority patent/EP4180942A1/en
Priority to KR1020227038309A priority patent/KR20220162775A/en
Publication of WO2022028189A1 publication Critical patent/WO2022028189A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/06Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/2866Architectures; Arrangements
    • H04L67/30Profiles

Definitions

  • the present disclosure relates to the technical fields of the Internet and smart terminals, and in particular, to a screen projection method and apparatus, an electronic device, and a computer-readable storage medium.
  • Screencasting refers to casting media files from one device to other devices for playback/display.
  • the video or picture on the current mobile phone is placed on another mobile phone, computer or smart TV for playback/display, and the user can watch the video or picture on the current mobile phone on the other mobile phone, computer or smart TV.
  • the current screencasting relies on the establishment of a connection between the server and the client to transmit video or pictures.
  • the transmission parameters are parsed, and there may be some screencasting display problems; in addition, it is also possible that the client is not necessarily
  • the problem with the screen cast object that is, the device that does not need to accept the screen cast also receives the screen cast file, which will lead to a waste of network resources.
  • an embodiment of the present disclosure provides a screen projection method, which includes: sending a search file to search for a client; the search file includes a randomly generated device identifier of a server and configuration parameters of the server, the configuration parameters of the server Including parameters for selecting a target client; receiving a search response message returned by the client; the search response message including the client's device identifier and device parameters; determining the target client that meets the screen projection conditions according to the search response message and sending the screen projection file to the target client.
  • an embodiment of the present disclosure provides a screen projection method, including: monitoring a search file sent by a server; the search file includes a randomly generated device identifier of the server and configuration parameters of the server, and the configuration parameters of the server include parameters for selecting a target client; when a search file is monitored, a search response message is generated based on the search file and returned to the server, the search response message including the client's device identification and device parameters; and receiving The screen projection file sent by the server.
  • an embodiment of the present disclosure provides a screen projection device, including: a server sending module configured to send the search file to the outside to search for a client; the search file includes a randomly generated device identifier and service of the server
  • the configuration parameters of the server end, the configuration parameters of the server end include parameters used to select the target client; the server receiving module is configured to receive the search response message returned by the client; the search response message includes the device identifier of the client and device parameters; and a server determination module configured to determine a target client that satisfies the screen projection condition according to the search response message; the server sending module is further configured to send a screen projection file to the target client.
  • an embodiment of the present disclosure provides a screen projection device, including: a client receiving module configured to receive a search file sent by a server; the search file includes a randomly generated device identifier of the server and configuration parameters of the server, The configuration parameters of the server include parameters for selecting a target client; and a client sending module is configured to return a search response message to the server; the search response message includes the client's device identifier and device parameters; The client receiving module is further configured to receive the screen projection file sent by the server.
  • an electronic device including:
  • processors one or more processors
  • a memory on which one or more computer programs are stored, when the one or more computer programs are executed by the one or more processors, the one or more processors implement the aforementioned screen projection method; as well as
  • One or more I/O interfaces connected between the processor and the memory, are configured to implement information interaction between the processor and the memory.
  • an embodiment of the present disclosure provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the foregoing screen projection method.
  • FIG. 1 is a flowchart of a screen projection method according to an embodiment of the present disclosure
  • FIG. 2 is a flowchart of a screen projection method provided by an embodiment of the present disclosure
  • FIG. 3 is a schematic block diagram of a screen projection device provided by an embodiment of the present disclosure.
  • FIG. 4 is a schematic block diagram of a screen projection device provided by an embodiment of the present disclosure.
  • FIG. 5 is a flowchart of a screen projection method according to an embodiment of the present disclosure.
  • FIG. 6 is a schematic block diagram of an electronic device according to an embodiment of the present disclosure.
  • the server and client Before screencasting between devices, the server and client need to establish a connection, and then transfer the screencasting file to the client for the client to perform screencasting. Since the configuration file is sent to the client device together with the screencasting file, the server cannot control the client, resulting in some unnecessary clients receiving the screencasting file, wasting network resources.
  • the server may also be referred to as a sender, and the client may also be referred to as a receiver.
  • the server sends the screencast file, and the client receives the screencast file and displays it.
  • an embodiment of the present disclosure provides a screen projection method, and the method is applied to a server, that is, a side that provides a screen projection file.
  • FIG. 1 is a flowchart of a screen projection method according to an embodiment of the present disclosure. As shown in FIG. 1 , the screen projection method may include the following steps 101 to 104 .
  • Step 101 sending a search file to search for a client.
  • the search file includes the device ID of the server and the configuration parameters of the server.
  • the device ID of the server can be randomly generated, and the configuration parameters of the server include parameters for selecting the target client, that is, the server can select the target client according to the configuration parameters.
  • the device identifier of the server can be generated according to a preset rule.
  • the server may randomly generate an ip, and the generation method of the ip may adopt an existing manner, which is not limited in the present disclosure.
  • the server After the server generates an IP, it generates a search file based on the IP and configuration parameters. Configuration parameters can be entered manually or automatically generated in the background.
  • the configuration parameters may include regional characteristics by which clients can be filtered, clients in a specific region can be excluded, or only clients in a specific region can be selected. For example, if the region feature is set as the region A connection feature, then after the server sends the search file, only the client located in region A can receive the search file and establish a connection with it.
  • the configuration parameter is equivalent to a screening mechanism, which blocks some clients in advance during the connection establishment phase, so that the network resource occupation can be reduced in the subsequent screen-casting file transfer process.
  • a client list can be generated through the configuration parameters of the client, that is, a client list is generated according to the device parameters and device identifiers of the client.
  • the client list can be a list arranged in a preset priority order.
  • the list includes at least one client, and the priority can be set by region. For example, the priority of area A is high, the priority of area B is medium, and the priority of area C is low.
  • the target client is selected from the client list according to the configuration parameters of the server, and the client with lower priority can be shielded to achieve a differentiated configuration effect and reduce subsequent network resource occupation.
  • the server may sort the priorities of the clients according to the device parameters of the clients. For example, set clients that are not suitable for display to low priority and clients that are suitable for display to high priority. Generate a list of clients according to their priority. The server can screen clients during the connection establishment phase, and block clients with lower priority.
  • the configuration parameters of the server may include the device type, that is, the device type of the server.
  • the configuration parameters are adjusted according to the device type, and the fixed parameters in the subsequent data interaction process are dynamically added or deleted, and the server actively shields Inappropriate clients, or lower the priority of inappropriate clients, so as to achieve a differentiated configuration effect and reduce subsequent network resource occupation.
  • the configuration parameters of the server may include one or more parameters of system version, screen size or resolution.
  • the server can send its own system version, screen size or projection resolution to the client in the search ip stage.
  • the client can determine whether to support the server's projection request according to the system version, and the client can be adjusted according to the screen size and resolution. Displays the screen size and resolution of the device.
  • the system version refers to the version of the operating system, for example, in the case of a mobile phone, the system version may be various versions of the Android system.
  • the search file may include a device identifier and configuration parameters
  • the device identifier is the unique identifier of the device corresponding to the server
  • the client can determine the server according to the device identifier of the server.
  • the configuration parameters include at least one of screen size, resolution, system version, and regional characteristics.
  • Step 102 Receive a search response message returned by the client.
  • the search response message may include the client's device identification and device parameters.
  • the device identifier is the unique identifier of the device used by the client, and the device parameter may include at least one of screen resolution, screen size, and system version.
  • the client after receiving the search file sent by the server, the client identifies the device identifier and configuration parameters of the server, determines whether its own device supports the display of the server, and returns a search response message.
  • the client after the client receives the search file sent by the server, it determines whether its own device supports the display of the screen-casting file, and when it supports the display, it returns to the server a message indicating that the ip match is successful; or, when it does not support the display , and return a message that the ip match failed to the server.
  • Step 103 according to the search response message, determine a target client that satisfies the screen projection condition.
  • the search response message returned by the client includes a successful matching message or a matching unsuccessful message.
  • the search response message is a successful matching message, it is determined that the client corresponding to the search response message meets the screen casting conditions. target client.
  • the search response message is a message that fails to match, it is determined that the client corresponding to the search response message is a client that does not meet the screen casting conditions.
  • the server and the client can also implement pre-decoding in the search ip stage according to the mutual system version and device parameters, that is, to agree on a communication method, so as to avoid parsing the configuration parameters during the transmission of the screencast file, and improve the investment. screen efficiency and effectiveness.
  • the server and client After the client returns a successful IP matching message to the server, the server and client complete the preliminary work of screen projection.
  • the server can send the screencast file to the client, and the client receives the screencast file and displays it.
  • Step 104 sending the screen projection file to the target client.
  • the screen projection file includes any suitable file such as video or picture, and when the server needs to project a picture on the client side, it sends a picture stream to the client. When the server needs to project video on the client, it sends a video stream to the client.
  • the screen projection method may further include: saving the device identification of the client, so that the server can easily obtain the device identification of the client when needed.
  • the screen projection method provided by the embodiment of the present disclosure includes: sending a search file to search for a client; the search file includes a device identifier of the server and configuration parameters of the server, and the configuration parameters of the server include parameters for selecting a target client; Search response message; the search response message includes the client's device identification and device parameters; the target client that meets the screen projection conditions is determined according to the search response message; the screen projection file is sent to the target client, and the server sends the device identification and configuration parameters during the search phase. It is transmitted to the client, and the target client to be projected can be selected through the configuration parameters, that is, only the selected client can accept the projection, which realizes the differentiated configuration parameters. Moreover, the transmission parameters are pre-decoded in the search stage, and no need The transmission parameters are parsed during the video file transmission process, which improves the screen projection stability and user experience.
  • an embodiment of the present disclosure provides a screen projection method, which is applied to a client, that is, an end that accepts projection.
  • FIG. 2 is a flowchart of a screen projection method provided by an embodiment of the present disclosure. As shown in FIG. 2 , the screen projection method includes the following steps 201 to 204 .
  • Step 201 Receive the search file sent by the server.
  • the search file may include the device ID of the server and the configuration parameters of the server.
  • the device ID of the server may be randomly generated, and the configuration parameters of the server may include parameters for selecting the target client, that is, the server may select the target client according to the configuration parameters.
  • each in order to realize the connection between the server and the client, each can generate a device identifier ip, and transmit the device identifier to the other party in the search phase.
  • the server may randomly generate an ip, and the generation method of the ip may adopt an existing manner, which is not limited in the present disclosure.
  • the server After the server generates an IP, it generates a search file based on the IP and configuration parameters. Configuration parameters can be entered manually or automatically generated in the background.
  • the configuration parameters may include regional characteristics by which clients can be filtered, clients in a specific region can be excluded, or only clients in a specific region can be selected. For example, if the region feature is set as the region A connection feature, then after the server sends the search file, only the client located in region A can receive the search file and establish a connection with it.
  • the configuration parameter is equivalent to a screening mechanism, which blocks some clients in advance during the connection establishment phase, so that the network resource occupation can be reduced in the subsequent screen-casting file transfer process.
  • a client list may be generated through configuration parameters of the client, the client list may be a list arranged in a preset priority order, and the client list includes at least one client, such as the priority of area A is high, the priority of area B is medium, and the priority of area C is low.
  • the target client is selected from the client list according to the configuration parameters of the server, and the client with lower priority is shielded to achieve a differentiated configuration effect and reduce subsequent network resource occupation.
  • the configuration parameters of the server may include the device type, that is, the device type of the server.
  • the configuration parameters are adjusted according to the device type, and the fixed parameters in the subsequent data interaction process are dynamically added or deleted, and the server actively shields Inappropriate clients, or lower the priority of inappropriate clients, so as to achieve a differentiated configuration effect and reduce subsequent network resource occupation.
  • the configuration parameters of the server may include one or more parameters of system version, screen size and resolution, and the server sends its own system version, screen size or projection resolution to the ip search stage.
  • Client the client can determine whether to support the screen projection request of the server according to the system version, and can adjust the screen size and resolution of the client display device according to the screen size and resolution.
  • the system version refers to the version of the operating system, for example, in the case of a mobile phone, the system version may be various versions of the Android system.
  • the search file may include the device identification and configuration parameters of the server.
  • the device identification is the unique identification of the device corresponding to the server, and the client can determine the server according to the device identification of the server.
  • the configuration parameters include at least one of screen size, resolution, system version, and regional characteristics.
  • the server sends the search file to the client in the search ip stage. Since the server sets configuration parameters, only the client designated by the server can receive the search file. When the client receives the search file in the ip search stage, it establishes a connection with the server, and at the same time, it can agree on the communication method, that is, to realize the mutual transfer of terminal parameters in the ip search stage, which effectively solves and avoids the interaction between the differentiated terminal screens. The display problem of the vote.
  • Step 202 generating a search response message based on the search file.
  • the search response message includes the client's device identification and the client's device parameters.
  • the device ID of the client is a unique ID used to represent the client.
  • the client after listening to the search file sent by the server, the client identifies the device identifier and configuration parameters of the server, determines whether its own device supports display, and returns a search response message.
  • the client after the client receives the search file sent by the server, it determines whether its own device supports the display of the screen-casting file.
  • the server returns a message that IP matching failed.
  • the client can determine whether its own device supports the display of the screencast file according to the configuration parameters of the server.
  • Step 203 returning a search response message to the server.
  • the search response message returned to the server includes a successful matching message or a matching unsuccessful message.
  • the client corresponding to the search response message is eligible for screen projection. target client.
  • the search response message is a message that fails to match, the client corresponding to the search response message is a client that does not meet the screen casting conditions.
  • the server and the client can also implement pre-decoding in the search ip stage according to the mutual system version and device parameters, that is, to agree on a communication method, so as to avoid parsing the configuration parameters during the transmission of the screencast file, and improve the investment. screen efficiency and effectiveness.
  • the server and the client After returning a message indicating that the ip match is successful to the server, the server and the client complete the preliminary work of screen projection.
  • the server can send the screencast file to the client, and the client receives the screencast file and displays it.
  • Step 204 Receive the screen projection file sent by the server.
  • the screen projection file includes a video or a picture
  • the server when the server needs to project a picture on the client side, it sends a picture stream to the client.
  • the server needs to project video files on the client, it sends a video stream to the client.
  • the client after receiving the screen projection file, uses a display device to display the screen projection file.
  • FIG. 3 is a schematic block diagram of a screen projection device provided by an embodiment of the present disclosure.
  • the screen projection device includes a server sending module 301 , a server receiving module 302 and a server determining module 303 .
  • the server-side sending module 301 is configured to send a search file to search clients.
  • the search file is a file generated by a server-side generating module.
  • the search file includes the randomly generated device identification and configuration parameters of the server, and the configuration parameters are used to select target clients that meet the conditions.
  • the device identifier of the server can be generated according to a preset rule.
  • the server may randomly generate an ip, and the generation method of the ip may adopt an existing manner, which is not limited in the present disclosure.
  • the server After the server generates an IP, it generates a search file based on the IP and configuration parameters. Configuration parameters can be entered manually or automatically generated in the background.
  • the configuration parameters of the server may include one or more parameters of screen size, resolution, system version, regional characteristics and device type.
  • the functions of the screen size, resolution, system version, regional characteristics and device type reference may be made to the description of step 101, which will not be repeated here.
  • the server receiving module 302 is configured to receive the search response message returned by the client.
  • the search response message may include the client's device identification and device parameters.
  • the device identifier is the unique identifier of the device used by the client, and the device parameter includes at least one of screen resolution, screen size, and system version.
  • the client after receiving the search file sent by the server, the client identifies the device identifier and configuration parameters of the server, determines whether its own device supports display, and then returns a search response message.
  • the client after the client receives the search file sent by the server, it determines whether its own device supports the display of the screencast file. Returns an IP matching failure message to the server. In some embodiments, the client can determine whether its own device supports the display of the screencast file according to the configuration parameters of the server.
  • the server determination module 303 is configured to determine the client that meets the screen projection condition according to the search response message.
  • the search response message returned by the client includes a successful matching message and a matching unsuccessful message.
  • the search response message is a successful matching message, it is determined that the client corresponding to the search response message meets the screen casting conditions. target client.
  • the search response message is a message that fails to match, it is determined that the client corresponding to the search response message is a client that does not meet the screen casting conditions.
  • the server-side sending module 301 is further configured to send a screen-casting file to a target client that satisfies the screen-casting condition.
  • the screen projection file may include a video or a picture.
  • the server sending module 301 sends a video stream or a picture stream to the client.
  • the screen projection device further includes: a list generation module configured to generate a client list according to configuration parameters of the client; the client list is a list arranged according to the priority of the clients.
  • the screen projection apparatus further includes: a storage module configured to store the device identifier of the client.
  • the screen projection device further includes a communication determination module configured to determine the communication mode according to the configuration parameters and the device parameters returned by the client.
  • the server sending module sends a search file to search for the client;
  • the search file includes the device identifier of the server and the configuration parameters of the server, and the configuration parameters of the server may include parameters for selecting the target client;
  • the server receiving module receives the search response message returned by the client;
  • the search response message includes the device identifier and device parameters of the client;
  • the server determination module determines the target client that meets the screen projection conditions according to the search response message; the server sends the module to the target client.
  • the client decides to send the screencast file, and the server transmits the device identification and configuration parameters to the client during the search phase.
  • the target client to be casted can be selected, that is, only the selected client can accept the screencast, which realizes Differentiated configuration parameters, and the transmission parameters are pre-decoded in the search stage, and there is no need to parse the transmission parameters during the video file transmission process, which improves the screen projection stability and user experience.
  • FIG. 4 is a schematic block diagram of a screen projection device provided by an embodiment of the present disclosure. As shown in FIG. 4 , the screen projection device includes a client receiving module 401 , a search response generating module 402 and a client sending module 403 .
  • the client receiving module 401 is configured to monitor the search file sent by the server.
  • the search file may include a randomly generated device identifier of the server and configuration parameters of the server, and the configuration parameters of the server are used to select a client that meets the conditions.
  • each in order to realize the connection between the server and the client, each needs to generate a device identifier ip, and transmit the device identifier to each other in the search phase.
  • the search response generating module 402 is configured to generate a search response message based on the search file when the search file is monitored.
  • the search response message includes the client's device identification and the client's device parameters.
  • the client after receiving the search file sent by the server, the client identifies the device identifier and configuration parameters of the server, determines whether its own device supports display, and returns a search response message.
  • the client after the client receives the search file sent by the server, it determines whether its own device supports the display of the screencast file, and if it supports the display, it returns a message indicating that the ip match is successful to the server; if it does not support the display, then Returns a message that IP matching failed to the server.
  • the client sending module 403 is configured to return a search response message to the server.
  • the client receiving module 401 is further configured to receive the screen projection file sent by the server.
  • the screen projection file includes a video or a picture
  • the server when the server needs to project a picture on the client side, it sends a picture stream to the client.
  • the server needs to project video on the client, it sends a video stream to the client.
  • the screen projection device may further include: a display module configured to display the screen projection file on the client.
  • the client receiving module is configured to receive a search file sent by the server; the search file includes a randomly generated device identifier and configuration parameters of the server, and the configuration parameters may include parameters for selecting a target client ;
  • the client sending module is configured to return a search response message to the server; the search response message includes the client's device identification and device parameters, and the server transmits the device identification and configuration parameters to the client during the search phase.
  • the target client of the screen projection that is, only the selected client can accept the screen projection, which realizes the differentiated configuration parameters.
  • the transmission parameters are pre-decoded in the search stage, and there is no need to parse the transmission parameters during the video file transmission process. The stability and user experience of the projection screen are improved.
  • the screen projection method and the screen projection device are further described below.
  • the server and the client perform the following steps 501 to 504 .
  • Step 501 the server sends the search file to the client.
  • the server sends a search file to the client to search for clients that accept screencasting.
  • the search file includes the randomly generated device identification and configuration parameters of the server, and the configuration parameters may include parameters for selecting a target client.
  • Step 502 the client returns a search response to the server.
  • the client returns a search response message to the server; the search response message includes the client's device identification and device parameters.
  • Step 503 the server saves the device identifier of the client.
  • Step 504 the server sends the screen projection file to the client.
  • the server sends a screen projection file to the client that meets the requirements, and the screen projection file includes video or pictures.
  • the server needs to project a picture on the client, it sends a picture stream to the client.
  • the server needs to project video on the client, it sends a video stream to the client.
  • the server and the client can exchange data and perform screen projection. If the screen size of the server is not sent to the client during the search ip phase, then during data interaction, the client can send a request to the server to obtain the screen size, and the server sends the actual screen size to the client based on the request. After that, transfer the screen projection data.
  • the screencast file can be an image or a video.
  • the server can use the Android Debug Bridge (ADB) command to capture the image or obtain the image through the ImageReader of the Android system, and then transmit the image through the network.
  • the video stream can be obtained by obtaining the screen stream through the operating system, and then transmitted after H264 encoding, which can make the screen display smoother.
  • the system agent has a MediaCodec class and a VirtualDisplay class that can read the screen stream and convert it into an H264 encoded video stream.
  • the server receives the image data information sent by the client through a socket (Socket), decompresses it and displays it on the user interface (UI), and when the user clicks on the UI, the coordinates of the clicked position can be obtained by scaling The conversion is converted into actual coordinates, and then the corresponding command is executed through ADB, and then the change of the image is transmitted to the client through Socket in real time.
  • Socket Socket
  • Socket user interface
  • the web application first obtains the H264 encoded video stream of the Android agent through socket on node.js, then transmits the byte array to WebRtc in real time through WebSocket, displays it through the video tag of WebRtc, and then obtains the coordinate event of the mouse click through js.
  • the subsequent operations are very similar to those of the PC. They all get the command and execute it, and then the AndroidClient casts the screen image, and so on.
  • modules involved in the embodiments of the present disclosure are all logical modules, and in practical applications, a logical module/unit may pass through a physical unit, a part of a physical unit, or a combination of multiple physical units. Combination implementation.
  • the embodiments of the present disclosure do not introduce units that are not very closely related to solving the technical problems raised by the present disclosure, but this does not mean that there are no other units in the embodiments of the present disclosure. .
  • an electronic device which includes:
  • processors 601 one or more processors 601;
  • One or more I/O interfaces 603 are connected between the processor 601 and the memory 602 , and are configured to implement information interaction between the processor 601 and the memory 602 .
  • the processor 601 is a device with data processing capability, which includes but is not limited to a central processing unit (CPU), etc.;
  • the memory 602 is a device with data storage capability, which includes but is not limited to random access memory (RAM, more specifically SDRAM) , DDR, etc.), read-only memory (ROM), electrified erasable programmable read-only memory (EEPROM), flash memory (FLASH);
  • I/O interface (read-write interface) 603 is connected between the processor 601 and the memory 602, and can The information interaction between the processor 601 and the memory 602 is realized, which includes but is not limited to a data bus (Bus) and the like.
  • Buss data bus
  • the processor 601, memory 602, and I/O interface 603 are interconnected by a bus and, in turn, connected to other components of the computing device.
  • an embodiment of the present disclosure provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements any one of the above screen projection methods.
  • Such software may be distributed on computer-readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media).
  • computer storage media includes both volatile and nonvolatile implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data flexible, removable and non-removable media.
  • Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cartridges, magnetic tape, magnetic disk storage or other magnetic storage, or available with Any other medium that stores the desired information and can be accessed by a computer.
  • communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and can include any information delivery media, as is well known to those of ordinary skill in the art .
  • Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should only be construed in a general descriptive sense and not for purposes of limitation. In some instances, it will be apparent to those skilled in the art that features, characteristics and/or elements described in connection with a particular embodiment may be used alone or in combination with features, characteristics described in connection with other embodiments unless expressly stated otherwise and/or components used in combination. Accordingly, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the scope of the present disclosure as set forth in the appended claims.

Abstract

A screen projection method, a screen projection apparatus, an electronic device, and a computer readable storage medium. The screen projection method comprises: sending a search file to search a client, the search file comprising a device identifier of a server and configuration parameters of the server, and the configuration parameters of the server comprising parameters for selecting a target client; receiving a search response message returned by the client, the search response message comprising a device identifier and device parameters of the client; determining, according to the search response message, a target client meeting a screen projection condition; and sending a screen projection file to the target client.

Description

投屏方法及装置、电子设备和计算机可读存储介质Screen projection method and apparatus, electronic device and computer-readable storage medium
相关申请的交叉引用CROSS-REFERENCE TO RELATED APPLICATIONS
本申请要求于2020年8月5日提交的中国专利申请NO.202010777220.0的优先权,该中国专利申请的内容通过引用的方式整体合并于此。This application claims the priority of Chinese patent application No. 202010777220.0 filed on August 5, 2020, the contents of which are incorporated herein by reference in their entirety.
技术领域technical field
本公开涉及互联网、智能终端技术领域,尤其涉及投屏方法及装置、电子设备和计算机可读存储介质。The present disclosure relates to the technical fields of the Internet and smart terminals, and in particular, to a screen projection method and apparatus, an electronic device, and a computer-readable storage medium.
背景技术Background technique
投屏是指将一个设备上的媒体文件投放到其他设备上进行播放/显示。例如,将当前手机上的视频或图片投放到其它手机、电脑或智能电视上进行播放/显示,用户可以在其它手机、电脑或智能电视上观看当前手机上的视频或图片。Screencasting refers to casting media files from one device to other devices for playback/display. For example, the video or picture on the current mobile phone is placed on another mobile phone, computer or smart TV for playback/display, and the user can watch the video or picture on the current mobile phone on the other mobile phone, computer or smart TV.
目前的投屏依赖于服务端和客户端之间建立连接后传输视频或图片,在视频流的传输过程中解析传输参数,可能存在一些投屏显示问题;另外,也有可能出现客户端不一定是投屏对象的问题,即不需要接受投屏的设备也收到投屏文件,这将导致网络资源的浪费。The current screencasting relies on the establishment of a connection between the server and the client to transmit video or pictures. During the transmission of the video stream, the transmission parameters are parsed, and there may be some screencasting display problems; in addition, it is also possible that the client is not necessarily The problem with the screen cast object, that is, the device that does not need to accept the screen cast also receives the screen cast file, which will lead to a waste of network resources.
发明内容SUMMARY OF THE INVENTION
第一方面,本公开实施例提供一种投屏方法,其包括:发送搜索文件以搜索客户端;所述搜索文件包括随机产生的服务端的设备标识和服务端的配置参数,所述服务端的配置参数包括用于选择目标客户端的参数;接收所述客户端返回的搜索响应消息;所述搜索响应消息包括所述客户端的设备标识和设备参数;依据所述搜索响应消息确定满足投屏条件的目标客户端;以及向所述目标客户端发送投屏文件。In a first aspect, an embodiment of the present disclosure provides a screen projection method, which includes: sending a search file to search for a client; the search file includes a randomly generated device identifier of a server and configuration parameters of the server, the configuration parameters of the server Including parameters for selecting a target client; receiving a search response message returned by the client; the search response message including the client's device identifier and device parameters; determining the target client that meets the screen projection conditions according to the search response message and sending the screen projection file to the target client.
第二方面,本公开实施例提供一种投屏方法,包括:监听服务端发出的搜索文件;所述搜索文件包括随机产生的服务端的设备标识和服务端的配置参数,所述服务端的配置参数包括用于选择目标客户端的参数;当监听到搜索文件时,基于所述搜索文件生成并向所述服务端返回搜索响应消息,所述搜索响应消息包括所述客户端的设备标识和设备参数;以及接收所述服务端发送的投屏文件。In a second aspect, an embodiment of the present disclosure provides a screen projection method, including: monitoring a search file sent by a server; the search file includes a randomly generated device identifier of the server and configuration parameters of the server, and the configuration parameters of the server include parameters for selecting a target client; when a search file is monitored, a search response message is generated based on the search file and returned to the server, the search response message including the client's device identification and device parameters; and receiving The screen projection file sent by the server.
第三方面,本公开实施例提供一种投屏装置,包括:服务端发送模块,配置为向外发送所述搜索文件以搜索客户端;所述搜索文件包括随机产生的服务端的设备标识和服务端的配置参数,所述服务端的配置参数包括用于选择目标客户端的参数;服务端接收模块,配置为接收所述客户端返回的搜索响应消息;所述搜索响应消息包括所述客户端的设备标识和设备参数;以及服务端确定模块,配置为依据所述搜索响应消息确定满足投屏条件的目标客户端;服务端发送模块还配置为向所述目标客户端发送投屏文件。In a third aspect, an embodiment of the present disclosure provides a screen projection device, including: a server sending module configured to send the search file to the outside to search for a client; the search file includes a randomly generated device identifier and service of the server The configuration parameters of the server end, the configuration parameters of the server end include parameters used to select the target client; the server receiving module is configured to receive the search response message returned by the client; the search response message includes the device identifier of the client and device parameters; and a server determination module configured to determine a target client that satisfies the screen projection condition according to the search response message; the server sending module is further configured to send a screen projection file to the target client.
第四方面,本公开实施例提供一种投屏装置,包括:客户端接收模块,配置为接收服务端发出的搜索文件;所述搜索文件包括随机产生的服务端的设备标识和服务端的配置参数,所述服务端的配置参数包括用于选择目标客户端的参数;以及客户端发送模块,配置为向所述服务端返回搜索响应消息;所述搜索响应消息包括所述客户端的设备标识和设备参数;所述客户端接收模块还配置为接收所述服务端发送的投屏文件。In a fourth aspect, an embodiment of the present disclosure provides a screen projection device, including: a client receiving module configured to receive a search file sent by a server; the search file includes a randomly generated device identifier of the server and configuration parameters of the server, The configuration parameters of the server include parameters for selecting a target client; and a client sending module is configured to return a search response message to the server; the search response message includes the client's device identifier and device parameters; The client receiving module is further configured to receive the screen projection file sent by the server.
第五方面,本公开实施例提供了一种电子设备,包括:In a fifth aspect, an embodiment of the present disclosure provides an electronic device, including:
一个或多个处理器;one or more processors;
存储器,其上存储有一个或多个计算机程序,当所述一个或多个计算机程序被所述一个或多个处理器执行时,使得所述一个或多个处理器实现前述的投屏方法;以及a memory, on which one or more computer programs are stored, when the one or more computer programs are executed by the one or more processors, the one or more processors implement the aforementioned screen projection method; as well as
一个或多个I/O接口,连接在所述处理器与所述存储器之间,配置为实现所述处理器与所述存储器的信息交互。One or more I/O interfaces, connected between the processor and the memory, are configured to implement information interaction between the processor and the memory.
第六方面,本公开实施例提供了一种计算机可读存储介质,其上存储有计算机程序,所述计算机程序被处理器执行时实现前述的投 屏方法。In a sixth aspect, an embodiment of the present disclosure provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the foregoing screen projection method.
附图说明Description of drawings
图1为本公开实施例提供的一种投屏方法的流程图;FIG. 1 is a flowchart of a screen projection method according to an embodiment of the present disclosure;
图2为本公开实施例提供的一种投屏方法的流程图;FIG. 2 is a flowchart of a screen projection method provided by an embodiment of the present disclosure;
图3是本公开实施例提供的一种投屏装置的原理框图;3 is a schematic block diagram of a screen projection device provided by an embodiment of the present disclosure;
图4是本公开实施例提供的一种投屏装置的原理框图;4 is a schematic block diagram of a screen projection device provided by an embodiment of the present disclosure;
图5为本公开实施例提供的一种投屏方法的流程图;以及FIG. 5 is a flowchart of a screen projection method according to an embodiment of the present disclosure; and
图6为本公开实施例提供的一种电子设备的原理框图。FIG. 6 is a schematic block diagram of an electronic device according to an embodiment of the present disclosure.
具体实施方式detailed description
为使本领域的技术人员更好地理解本公开的技术方案,下面结合附图对本公开提供的投屏方法及装置、电子设备和计算机可读存储介质进行详细描述。In order to make those skilled in the art better understand the technical solutions of the present disclosure, the screen projection method and device, electronic device, and computer-readable storage medium provided by the present disclosure are described in detail below with reference to the accompanying drawings.
在下文中将参考附图更充分地描述示例实施例,但是所述示例实施例可以以不同形式来体现且不应当被解释为限于本文阐述的实施例。提供这些实施例的目的在于使本公开更加透彻和完整,并使本领域技术人员充分理解本公开的范围。Example embodiments are described more fully hereinafter with reference to the accompanying drawings, but which may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
如本文所使用的,术语“和/或”包括一个或多个相关列举条目的任何和所有组合。As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
本文所使用的术语仅用于描述特定实施例,且不意欲限制本公开。如本文所使用的,单数形式“一个”和“该”也意欲包括复数形式,除非上下文另外清楚指出。还将理解的是,当本说明书中使用术语“包括”和/或“由……制成”时,指定存在特定特征、整体、步骤、操作、元件和/或组件,但不排除存在或可添加一个或多个其它特征、整体、步骤、操作、元件、组件和/或其群组。The terminology used herein is used to describe particular embodiments only and is not intended to limit the present disclosure. As used herein, the singular forms "a" and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will also be understood that when the terms "comprising" and/or "made of" are used in this specification, it is specified that particular features, integers, steps, operations, elements and/or components are present, but not excluding the presence or possibility of Add one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
除非另外限定,否则本文所用的所有术语(包括技术术语和科学术语)的含义与本领域普通技术人员通常理解的含义相同。还将理解,诸如在常用字典中限定的那些术语应当被解释为具有与其在相关技术以及本公开的背景下的含义一致的含义,且将不解释为具有理想化 或过度形式上的含义,除非本文明确如此限定。Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be construed as having meanings consistent with their meanings in the context of the related art and this disclosure, and will not be construed as having idealized or over-formal meanings unless This document expressly so limited.
在设备之间进行投屏之前,服务端和客户端需要建立连接,然后将投屏文件传输至客户端,以供客户端进行投屏。由于配置文件随投屏文件一起发送至客户端设备,服务端无法对客户端进行控制,导致一些不必要的客户端接收投屏文件,浪费了网络资源。Before screencasting between devices, the server and client need to establish a connection, and then transfer the screencasting file to the client for the client to perform screencasting. Since the configuration file is sent to the client device together with the screencasting file, the server cannot control the client, resulting in some unnecessary clients receiving the screencasting file, wasting network resources.
在下述各实施例中,服务端也可称之为发送端,客户端也可称之为接收端。服务端发送投屏文件,客户端接收投屏文件并进行显示。In the following embodiments, the server may also be referred to as a sender, and the client may also be referred to as a receiver. The server sends the screencast file, and the client receives the screencast file and displays it.
第一方面,本公开实施例提供一种投屏方法,该方法应用于服务端,即提供投屏文件的一端。图1为本公开实施例提供的一种投屏方法的流程图。如图1所示,投屏方法可以包括以下步骤101至104。In a first aspect, an embodiment of the present disclosure provides a screen projection method, and the method is applied to a server, that is, a side that provides a screen projection file. FIG. 1 is a flowchart of a screen projection method according to an embodiment of the present disclosure. As shown in FIG. 1 , the screen projection method may include the following steps 101 to 104 .
步骤101,发送搜索文件以搜索客户端。 Step 101, sending a search file to search for a client.
搜索文件包括服务端的设备标识和服务端的配置参数,服务端的设备标识可以是随机产生的,服务端的配置参数包括用于选择目标客户端的参数,即服务端可以依据配置参数选择目标客户端。当然,在其他实施方式中,服务端的设备标识可以按照预设规则产生。The search file includes the device ID of the server and the configuration parameters of the server. The device ID of the server can be randomly generated, and the configuration parameters of the server include parameters for selecting the target client, that is, the server can select the target client according to the configuration parameters. Of course, in other implementations, the device identifier of the server can be generated according to a preset rule.
在一些实施方式中,服务端和客户端为了实现彼此连接,需要各自生成一个设备标识ip,并在搜索阶段将设备标识传输给对方。In some implementation manners, in order to realize the connection between the server and the client, each needs to generate a device identification ip, and transmit the device identification to each other in the search phase.
在一些实施方式中,服务端可以随机生成ip,ip的生成方式可以采用已有的方式,本公开对此不作限定。服务端生成ip后,依据ip和配置参数生成搜索文件。配置参数可以手动输入,也可以通过后台自动生成。In some implementation manners, the server may randomly generate an ip, and the generation method of the ip may adopt an existing manner, which is not limited in the present disclosure. After the server generates an IP, it generates a search file based on the IP and configuration parameters. Configuration parameters can be entered manually or automatically generated in the background.
在一些实施方式中,配置参数可以包括地域特性,通过地域特性可以筛选客户端,将特定地域的客户端排除,或者只选择特定地域的客户端。例如,设定地域特性为地域A连接特性,那么,当服务端发出搜索文件后,只有位于地域A的客户端能够接收到搜索文件,并与其建立连接。换言之,配置参数相当于一个筛选机制,在建立连接阶段事先屏蔽一些客户端,从而在后续的投屏文件传输过程中可以减少网络资源占用。In some implementations, the configuration parameters may include regional characteristics by which clients can be filtered, clients in a specific region can be excluded, or only clients in a specific region can be selected. For example, if the region feature is set as the region A connection feature, then after the server sends the search file, only the client located in region A can receive the search file and establish a connection with it. In other words, the configuration parameter is equivalent to a screening mechanism, which blocks some clients in advance during the connection establishment phase, so that the network resource occupation can be reduced in the subsequent screen-casting file transfer process.
在一些实施方式中,通过客户端的配置参数可以生成客户端列表,即依据客户端的设备参数和设备标识生成客户端列表,客户端列 表可以是按照预设的优先级顺序排列的列表,在客户端列表中包括至少一个客户端,优先级可以按照地域来设定,如A区域的优先级为高,B区域的优先级为中,C区域的优先级为低。在一些实施方式中,依据服务端的配置参数从客户端列表中选择目标客户端,可以将优先级较低的客户端屏蔽,达到差异化配置效果,减少后续网络资源占用。In some embodiments, a client list can be generated through the configuration parameters of the client, that is, a client list is generated according to the device parameters and device identifiers of the client. The client list can be a list arranged in a preset priority order. The list includes at least one client, and the priority can be set by region. For example, the priority of area A is high, the priority of area B is medium, and the priority of area C is low. In some embodiments, the target client is selected from the client list according to the configuration parameters of the server, and the client with lower priority can be shielded to achieve a differentiated configuration effect and reduce subsequent network resource occupation.
在一些实施方式中,服务端可以根据客户端的设备参数对客户端的优先级进行排序。例如,将不适合显示的客户端设置为低优先级,将适合显示的客户端设置为高优先级。按照客户端的优先级生成客户端列表。服务端可在建立连接阶段对客户端进行筛选,将优先级较低的客户端屏蔽掉。In some embodiments, the server may sort the priorities of the clients according to the device parameters of the clients. For example, set clients that are not suitable for display to low priority and clients that are suitable for display to high priority. Generate a list of clients according to their priority. The server can screen clients during the connection establishment phase, and block clients with lower priority.
在一些实施方式中,服务端的配置参数可以包括设备类型,即服务端的设备类型,针对设备类型调整配置参数,对后续的数据交互过程中的固定参数做到动态增加或删减,服务端主动屏蔽不合适的客户端,或者,降低不合适的客户端的优先级,从而达到差异化配置效果,减少后续网络资源占用。In some embodiments, the configuration parameters of the server may include the device type, that is, the device type of the server. The configuration parameters are adjusted according to the device type, and the fixed parameters in the subsequent data interaction process are dynamically added or deleted, and the server actively shields Inappropriate clients, or lower the priority of inappropriate clients, so as to achieve a differentiated configuration effect and reduce subsequent network resource occupation.
在一些实施方式中,服务端的配置参数可以包括系统版本、屏幕大小或分辨率中的一种或多种参数。服务端可以将自己的系统版本、屏幕大小或投屏分辨率在搜索ip阶段发送给客户端,客户端依据系统版本可以确定是否支持服务端的投屏请求,依据屏幕大小和分辨率可以调整客户端显示设备的屏幕大小和分辨率。系统版本是指操作系统的版本,例如,如果是手机,系统版本可以是安卓系统的各种版本。In some embodiments, the configuration parameters of the server may include one or more parameters of system version, screen size or resolution. The server can send its own system version, screen size or projection resolution to the client in the search ip stage. The client can determine whether to support the server's projection request according to the system version, and the client can be adjusted according to the screen size and resolution. Displays the screen size and resolution of the device. The system version refers to the version of the operating system, for example, in the case of a mobile phone, the system version may be various versions of the Android system.
需要说明的是,搜索文件可以包括设备标识和配置参数,设备标识是服务端对应设备的唯一标识,客户端可以依据服务端的设备标识确定服务端。配置参数至少包括屏幕大小、分辨率、系统版本和地域特性之一。It should be noted that the search file may include a device identifier and configuration parameters, the device identifier is the unique identifier of the device corresponding to the server, and the client can determine the server according to the device identifier of the server. The configuration parameters include at least one of screen size, resolution, system version, and regional characteristics.
步骤102,接收客户端返回的搜索响应消息。Step 102: Receive a search response message returned by the client.
搜索响应消息可以包括客户端的设备标识和设备参数。设备标识是客户端使用的设备的唯一标识,设备参数可以包括屏幕分辨率、屏幕大小和系统版本中的至少之一。The search response message may include the client's device identification and device parameters. The device identifier is the unique identifier of the device used by the client, and the device parameter may include at least one of screen resolution, screen size, and system version.
在一些实施方式中,客户端接收服务端发出的搜索文件后,识 别服务端的设备标识和配置参数,判断自身设备是否支持服务端的显示,并返回搜索响应消息。In some embodiments, after receiving the search file sent by the server, the client identifies the device identifier and configuration parameters of the server, determines whether its own device supports the display of the server, and returns a search response message.
在一些实施方式中,客户端接收服务端发出的搜索文件后,判断自身设备是否支持投屏文件的显示,当支持显示时,向服务端返回ip匹配成功的消息;或者,当不支持显示时,向服务端返回ip匹配失败的消息。In some embodiments, after the client receives the search file sent by the server, it determines whether its own device supports the display of the screen-casting file, and when it supports the display, it returns to the server a message indicating that the ip match is successful; or, when it does not support the display , and return a message that the ip match failed to the server.
步骤103,依据搜索响应消息确定满足投屏条件的目标客户端。 Step 103 , according to the search response message, determine a target client that satisfies the screen projection condition.
在一些实施方式中,客户端返回的搜索响应消息包括匹配成功的消息或匹配失败的消息,当搜索响应消息为匹配成功的消息时,确定该搜索响应消息对应的客户端是符合投屏条件的目标客户端。当搜索响应消息为匹配失败的消息时,确定该搜索响应消息对应的客户端是不符合投屏条件的客户端。In some embodiments, the search response message returned by the client includes a successful matching message or a matching unsuccessful message. When the search response message is a successful matching message, it is determined that the client corresponding to the search response message meets the screen casting conditions. target client. When the search response message is a message that fails to match, it is determined that the client corresponding to the search response message is a client that does not meet the screen casting conditions.
在一些实施方式中,服务端和客户端还可以依据相互的系统版本和设备参数在搜索ip阶段实现预解码,即约定通讯方式,从而避免在投屏文件的传输过程中解析配置参数,提高投屏效率和效果。In some embodiments, the server and the client can also implement pre-decoding in the search ip stage according to the mutual system version and device parameters, that is, to agree on a communication method, so as to avoid parsing the configuration parameters during the transmission of the screencast file, and improve the investment. screen efficiency and effectiveness.
当客户端向服务端返回ip匹配成功的消息后,服务端和客户端完成投屏的前期工作。服务端可以向客户端发送投屏文件,客户端接收投屏文件并进行显示。After the client returns a successful IP matching message to the server, the server and client complete the preliminary work of screen projection. The server can send the screencast file to the client, and the client receives the screencast file and displays it.
步骤104,向目标客户端发送投屏文件。 Step 104, sending the screen projection file to the target client.
在一些实施方式中,投屏文件包括视频或图片等任意适用的文件,当服务端需要在客户端投屏图片时,向客户端发送图片流。当服务端需要在客户端投屏视频时,向客户端发送视频流。In some embodiments, the screen projection file includes any suitable file such as video or picture, and when the server needs to project a picture on the client side, it sends a picture stream to the client. When the server needs to project video on the client, it sends a video stream to the client.
在一些实施方式中,在步骤102后,所述投屏方法还可以包括:保存客户端的设备标识,以供服务端在需要时,方便获得客户端的设备标识。In some implementations, after step 102, the screen projection method may further include: saving the device identification of the client, so that the server can easily obtain the device identification of the client when needed.
本公开实施例提供的投屏方法包括:发送搜索文件以搜索客户端;搜索文件包括服务端的设备标识和服务端的配置参数,服务端的配置参数包括用于选择目标客户端的参数;接收客户端返回的搜索响应消息;搜索响应消息包括客户端的设备标识和设备参数;依据搜索响应消息确定满足投屏条件的目标客户端;向目标客户端发送投屏文 件,服务端在搜索阶段将设备标识和配置参数传输给客户端,通过配置参数可以选择要投屏的目标客户端,即只有选定的客户端能够接受投屏,实现了差异化配置参数,而且,传输参数在搜索阶段实现预解码,不需要在视频文件传输过程中解析传输参数,提升了投屏的稳定性和用户体验。The screen projection method provided by the embodiment of the present disclosure includes: sending a search file to search for a client; the search file includes a device identifier of the server and configuration parameters of the server, and the configuration parameters of the server include parameters for selecting a target client; Search response message; the search response message includes the client's device identification and device parameters; the target client that meets the screen projection conditions is determined according to the search response message; the screen projection file is sent to the target client, and the server sends the device identification and configuration parameters during the search phase. It is transmitted to the client, and the target client to be projected can be selected through the configuration parameters, that is, only the selected client can accept the projection, which realizes the differentiated configuration parameters. Moreover, the transmission parameters are pre-decoded in the search stage, and no need The transmission parameters are parsed during the video file transmission process, which improves the screen projection stability and user experience.
第二方面,本公开实施例提供的一种投屏方法,该方法应用于客户端,即接受投屏的一端。图2为本公开实施例提供的一种投屏方法的流程图。如图2所示,投屏方法包括以下步骤201至204。In a second aspect, an embodiment of the present disclosure provides a screen projection method, which is applied to a client, that is, an end that accepts projection. FIG. 2 is a flowchart of a screen projection method provided by an embodiment of the present disclosure. As shown in FIG. 2 , the screen projection method includes the following steps 201 to 204 .
步骤201,接收服务端发出的搜索文件。Step 201: Receive the search file sent by the server.
搜索文件可以包括服务端的设备标识和服务端的配置参数,服务端的设备标识可以是随机产生的,服务端的配置参数可以包括用于选择目标客户端的参数,即服务端可以依据配置参数选择目标客户端。The search file may include the device ID of the server and the configuration parameters of the server. The device ID of the server may be randomly generated, and the configuration parameters of the server may include parameters for selecting the target client, that is, the server may select the target client according to the configuration parameters.
在一些实施方式中,服务端和客户端为了实现彼此连接,可以各自生成一个设备标识ip,并在搜索阶段将设备标识传输给对方。In some embodiments, in order to realize the connection between the server and the client, each can generate a device identifier ip, and transmit the device identifier to the other party in the search phase.
在一些实施方式中,服务端可以随机生成ip,ip的生成方式可以采用已有的方式,本公开对此不作限定。服务端生成ip后,依据ip和配置参数生成搜索文件。配置参数可以手动输入,也可以通过后台自动生成。In some implementation manners, the server may randomly generate an ip, and the generation method of the ip may adopt an existing manner, which is not limited in the present disclosure. After the server generates an IP, it generates a search file based on the IP and configuration parameters. Configuration parameters can be entered manually or automatically generated in the background.
在一些实施方式中,配置参数可以包括地域特性,通过地域特性可以筛选客户端,将特定地域的客户端排除,或者只选择特定地域的客户端。例如,设定地域特性为地域A连接特性,那么,当服务端发出搜索文件后,只有位于地域A的客户端能够接收到搜索文件,并与其建立连接。换言之,配置参数相当于一个筛选机制,在建立连接阶段事先屏蔽一些客户端,从而在后续的投屏文件传输过程中可以减少网络资源占用。In some implementations, the configuration parameters may include regional characteristics by which clients can be filtered, clients in a specific region can be excluded, or only clients in a specific region can be selected. For example, if the region feature is set as the region A connection feature, then after the server sends the search file, only the client located in region A can receive the search file and establish a connection with it. In other words, the configuration parameter is equivalent to a screening mechanism, which blocks some clients in advance during the connection establishment phase, so that the network resource occupation can be reduced in the subsequent screen-casting file transfer process.
在一些实施方式中,通过客户端的配置参数可以生成客户端列表,客户端列表可以是按照预设的优先级顺序排列的列表,在客户端列表中包括至少一个客户端,如A区域的优先级为高,B区域的优先级为中,C区域的优先级为低。在一些实施方式中,依据服务端的配 置参数从客户端列表中选择目标客户端,将优先级较低的客户端屏蔽,达到差异化配置效果,减少后续网络资源占用。In some embodiments, a client list may be generated through configuration parameters of the client, the client list may be a list arranged in a preset priority order, and the client list includes at least one client, such as the priority of area A is high, the priority of area B is medium, and the priority of area C is low. In some embodiments, the target client is selected from the client list according to the configuration parameters of the server, and the client with lower priority is shielded to achieve a differentiated configuration effect and reduce subsequent network resource occupation.
在一些实施方式中,服务端的配置参数可以包括设备类型,即服务端的设备类型,针对设备类型调整配置参数,对后续的数据交互过程中的固定参数做到动态增加或删减,服务端主动屏蔽不合适的客户端,或者,降低不合适的客户端的优先级,从而达到差异化配置效果,减少后续网络资源占用。In some embodiments, the configuration parameters of the server may include the device type, that is, the device type of the server. The configuration parameters are adjusted according to the device type, and the fixed parameters in the subsequent data interaction process are dynamically added or deleted, and the server actively shields Inappropriate clients, or lower the priority of inappropriate clients, so as to achieve a differentiated configuration effect and reduce subsequent network resource occupation.
在一些实施方式中,服务端的配置参数可以包括系统版本、屏幕大小和分辨率中的一种或多种参数,服务端将自己的系统版本、屏幕大小或投屏分辨率在搜索ip阶段发送给客户端,客户端依据系统版本可以确定是否支持服务端的投屏请求,依据屏幕大小和分辨率可以调整客户端显示设备的屏幕大小和分辨率。系统版本是指操作系统的版本,例如,如果是手机,系统版本可以是安卓系统的各种版本。In some embodiments, the configuration parameters of the server may include one or more parameters of system version, screen size and resolution, and the server sends its own system version, screen size or projection resolution to the ip search stage. Client, the client can determine whether to support the screen projection request of the server according to the system version, and can adjust the screen size and resolution of the client display device according to the screen size and resolution. The system version refers to the version of the operating system, for example, in the case of a mobile phone, the system version may be various versions of the Android system.
需要说明的是,搜索文件可以包括服务端的设备标识和配置参数,设备标识是服务端对应设备的唯一标识,客户端可以依据服务端的设备标识确定服务端。配置参数至少包括屏幕大小、分辨率、系统版本和地域特性之一。It should be noted that the search file may include the device identification and configuration parameters of the server. The device identification is the unique identification of the device corresponding to the server, and the client can determine the server according to the device identification of the server. The configuration parameters include at least one of screen size, resolution, system version, and regional characteristics.
在一些实施方式中,服务端是在搜索ip阶段向客户端发送搜索文件,由于服务端设置了配置参数,只有服务端指定的客户端能够接收搜索文件。当客户端在搜索ip阶段接收到搜索文件后,与服务端建立连接,同时可以约定通讯方式,即在ip搜索阶段实现终端参数互传,有效地解决和规避了差异化的终端屏幕之间互投的显示问题。In some implementations, the server sends the search file to the client in the search ip stage. Since the server sets configuration parameters, only the client designated by the server can receive the search file. When the client receives the search file in the ip search stage, it establishes a connection with the server, and at the same time, it can agree on the communication method, that is, to realize the mutual transfer of terminal parameters in the ip search stage, which effectively solves and avoids the interaction between the differentiated terminal screens. The display problem of the vote.
步骤202,基于搜索文件生成搜索响应消息。 Step 202, generating a search response message based on the search file.
搜索响应消息包括客户端的设备标识和客户端的设备参数。客户端的设备标识是用于表示客户端的唯一标识。The search response message includes the client's device identification and the client's device parameters. The device ID of the client is a unique ID used to represent the client.
在一些实施方式中,当客户端监听到服务端发出的搜索文件后,识别服务端的设备标识和配置参数,判断自身设备是否支持显示,并返回搜索响应消息。In some embodiments, after listening to the search file sent by the server, the client identifies the device identifier and configuration parameters of the server, determines whether its own device supports display, and returns a search response message.
在一些实施方式中,客户端接收服务端发出的搜索文件后,判断自身设备是否支持投屏文件的显示,当支持显示时,向服务端返回 ip匹配成功的消息;当不支持显示时,向服务端返回ip匹配失败的消息。在一些实施方式中,客户端可以依据服务端的配置参数判断自身设备是否支持投屏文件的显示。In some implementations, after the client receives the search file sent by the server, it determines whether its own device supports the display of the screen-casting file. The server returns a message that IP matching failed. In some embodiments, the client can determine whether its own device supports the display of the screencast file according to the configuration parameters of the server.
步骤203,向服务端返回搜索响应消息。 Step 203, returning a search response message to the server.
在一些实施方式中,向服务端返回的搜索响应消息包括匹配成功的消息或匹配失败的消息,当搜索响应消息为匹配成功的消息时,该搜索响应消息对应的客户端是符合投屏条件的目标客户端。当搜索响应消息为匹配失败的消息时,该搜索响应消息对应的客户端是不符合投屏条件的客户端。In some embodiments, the search response message returned to the server includes a successful matching message or a matching unsuccessful message. When the search response message is a successful matching message, the client corresponding to the search response message is eligible for screen projection. target client. When the search response message is a message that fails to match, the client corresponding to the search response message is a client that does not meet the screen casting conditions.
在一些实施方式中,服务端和客户端还可以依据相互的系统版本和设备参数在搜索ip阶段实现预解码,即约定通讯方式,从而避免在投屏文件的传输过程中解析配置参数,提高投屏效率和效果。In some embodiments, the server and the client can also implement pre-decoding in the search ip stage according to the mutual system version and device parameters, that is, to agree on a communication method, so as to avoid parsing the configuration parameters during the transmission of the screencast file, and improve the investment. screen efficiency and effectiveness.
当向服务端返回ip匹配成功的消息后,服务端和客户端完成投屏的前期工作。服务端可以向客户端发送投屏文件,客户端接收投屏文件并进行显示。After returning a message indicating that the ip match is successful to the server, the server and the client complete the preliminary work of screen projection. The server can send the screencast file to the client, and the client receives the screencast file and displays it.
步骤204,接收服务端发送的投屏文件。Step 204: Receive the screen projection file sent by the server.
在一些实施方式中,投屏文件包括视频或图片,当服务端需要在客户端投屏图片时,向客户端发送图片流。当服务端需要在客户端投屏视频文件时,向客户端发送视频流。In some implementations, the screen projection file includes a video or a picture, and when the server needs to project a picture on the client side, it sends a picture stream to the client. When the server needs to project video files on the client, it sends a video stream to the client.
在一些实施方式中,客户端接收到投屏文件后,利用显示装置显示投屏文件。In some embodiments, after receiving the screen projection file, the client uses a display device to display the screen projection file.
第三方面,提供的一种投屏装置,应用于服务端。图3是本公开实施例提供的一种投屏装置的原理框图。如图3所示,所述投屏装置包括服务端发送模块301、服务端接收模块302和服务端确定模块303。In a third aspect, a screen projection device is provided, which is applied to a server. FIG. 3 is a schematic block diagram of a screen projection device provided by an embodiment of the present disclosure. As shown in FIG. 3 , the screen projection device includes a server sending module 301 , a server receiving module 302 and a server determining module 303 .
服务端发送模块301配置为向外发送搜索文件以搜索客户端。The server-side sending module 301 is configured to send a search file to search clients.
在一些实施方式中,搜索文件是由服务端生成模块生成的文件。搜索文件包括随机产生的服务端的设备标识和配置参数,配置参数是用于选择满足条件的目标客户端。当然,在其他实施方式中,服务端 的设备标识可以按照预设规则产生。In some embodiments, the search file is a file generated by a server-side generating module. The search file includes the randomly generated device identification and configuration parameters of the server, and the configuration parameters are used to select target clients that meet the conditions. Of course, in other implementations, the device identifier of the server can be generated according to a preset rule.
在一些实施方式中,服务端和客户端为了实现彼此连接,需要各自生成一个设备标识ip,并在搜索阶段将设备标识传输给对方。In some implementation manners, in order to realize the connection between the server and the client, each needs to generate a device identification ip, and transmit the device identification to each other in the search phase.
在一些实施方式中,服务端可以随机生成ip,ip的生成方式可以采用已有的方式,本公开对此不作限定。服务端生成ip后,依据ip和配置参数生成搜索文件。配置参数可以手动输入,也可以通过后台自动生成。In some implementation manners, the server may randomly generate an ip, and the generation method of the ip may adopt an existing manner, which is not limited in the present disclosure. After the server generates an IP, it generates a search file based on the IP and configuration parameters. Configuration parameters can be entered manually or automatically generated in the background.
服务端的配置参数可以包括屏幕大小、分辨率、系统版本、地域特性和设备类型中的一种或多种参数。屏幕大小、分辨率、系统版本、地域特性和设备类型的作用可参见步骤101的描述,在此不再赘述。The configuration parameters of the server may include one or more parameters of screen size, resolution, system version, regional characteristics and device type. For the functions of the screen size, resolution, system version, regional characteristics and device type, reference may be made to the description of step 101, which will not be repeated here.
服务端接收模块302配置为接收客户端返回的搜索响应消息。The server receiving module 302 is configured to receive the search response message returned by the client.
搜索响应消息可以包括客户端的设备标识和设备参数。设备标识是客户端使用的设备的唯一标识,设备参数包括屏幕分辨率、屏幕大小和系统版本中的至少之一。The search response message may include the client's device identification and device parameters. The device identifier is the unique identifier of the device used by the client, and the device parameter includes at least one of screen resolution, screen size, and system version.
在一些实施方式中,客户端接收到服务端发出的搜索文件后,识别服务端的设备标识和配置参数,判断自身设备是否支持显示,然后返回搜索响应消息。In some embodiments, after receiving the search file sent by the server, the client identifies the device identifier and configuration parameters of the server, determines whether its own device supports display, and then returns a search response message.
在一些实施方式中,客户端接收到服务端发出的搜索文件后,判断自身设备是否支持投屏文件的显示,当支持显示时,向服务端返回ip匹配成功的消息;当不支持显示时,向服务端返回ip匹配失败的消息。在一些实施方式中,客户端可以依据服务端的配置参数判断自身设备是否支持投屏文件的显示。In some embodiments, after the client receives the search file sent by the server, it determines whether its own device supports the display of the screencast file. Returns an IP matching failure message to the server. In some embodiments, the client can determine whether its own device supports the display of the screencast file according to the configuration parameters of the server.
服务端确定模块303配置为依据搜索响应消息确定满足投屏条件的客户端。The server determination module 303 is configured to determine the client that meets the screen projection condition according to the search response message.
在一些实施方式中,客户端返回的搜索响应消息包括匹配成功的消息和匹配失败的消息,当搜索响应消息为匹配成功的消息时,确定该搜索响应消息对应的客户端是符合投屏条件的目标客户端。当搜索响应消息为匹配失败的消息时,确定该搜索响应消息对应的客户端是不符合投屏条件的客户端。In some embodiments, the search response message returned by the client includes a successful matching message and a matching unsuccessful message. When the search response message is a successful matching message, it is determined that the client corresponding to the search response message meets the screen casting conditions. target client. When the search response message is a message that fails to match, it is determined that the client corresponding to the search response message is a client that does not meet the screen casting conditions.
在一些实施方式中,服务端发送模块301还配置为向满足投屏条件的目标客户端发送投屏文件。In some embodiments, the server-side sending module 301 is further configured to send a screen-casting file to a target client that satisfies the screen-casting condition.
投屏文件可包括视频或图片,对应地,服务端发送模块301向客户端发送视频流或图片流。The screen projection file may include a video or a picture. Correspondingly, the server sending module 301 sends a video stream or a picture stream to the client.
在一些实施方式中,所述投屏装置还包括:列表生成模块,配置为依据客户端的配置参数生成客户端列表;客户端列表是按照客户端的优先级排列的列表。In some embodiments, the screen projection device further includes: a list generation module configured to generate a client list according to configuration parameters of the client; the client list is a list arranged according to the priority of the clients.
在一些实施方式中,所述投屏装置还包括:存储模块,配置为保存客户端的设备标识。In some embodiments, the screen projection apparatus further includes: a storage module configured to store the device identifier of the client.
在一些实施例中,所述投屏装置还包括通讯确定模块,配置为依据配置参数和客户端返回的设备参数确定通讯方式。In some embodiments, the screen projection device further includes a communication determination module configured to determine the communication mode according to the configuration parameters and the device parameters returned by the client.
本公开实施例提供的投屏装置中,服务端发送模块发送搜索文件以搜索客户端;搜索文件包括服务端的设备标识和服务端的配置参数,服务端的配置参数可以包括用于选择目标客户端的参数;服务端接收模块接收客户端返回的搜索响应消息;搜索响应消息包括客户端的设备标识和设备参数;服务端确定模块依据搜索响应消息确定满足投屏条件的目标客户端;服务端发送模块向目标客户端确定发送投屏文件,服务端在搜索阶段将设备标识和配置参数传输给客户端,通过配置参数可以选择要投屏的目标客户端,即只有选定的客户端能够接受投屏,实现了差异化配置参数,而且,传输参数在搜索阶段实现预解码,不需要在视频文件传输过程中解析传输参数,提升了投屏的稳定性和用户体验。In the screen projection device provided by the embodiment of the present disclosure, the server sending module sends a search file to search for the client; the search file includes the device identifier of the server and the configuration parameters of the server, and the configuration parameters of the server may include parameters for selecting the target client; The server receiving module receives the search response message returned by the client; the search response message includes the device identifier and device parameters of the client; the server determination module determines the target client that meets the screen projection conditions according to the search response message; the server sends the module to the target client. The client decides to send the screencast file, and the server transmits the device identification and configuration parameters to the client during the search phase. Through the configuration parameters, the target client to be casted can be selected, that is, only the selected client can accept the screencast, which realizes Differentiated configuration parameters, and the transmission parameters are pre-decoded in the search stage, and there is no need to parse the transmission parameters during the video file transmission process, which improves the screen projection stability and user experience.
第四方面,提供的一种投屏装置,应用于客户端。图4是本公开实施例提供的一种投屏装置的原理框图。如图4所示,所述投屏装置包括客户端接收模块401、搜索响应生成模块402和客户端发送模块403。In a fourth aspect, a screen projection device is provided, which is applied to a client. FIG. 4 is a schematic block diagram of a screen projection device provided by an embodiment of the present disclosure. As shown in FIG. 4 , the screen projection device includes a client receiving module 401 , a search response generating module 402 and a client sending module 403 .
客户端接收模块401配置为监听服务端发出的搜索文件。The client receiving module 401 is configured to monitor the search file sent by the server.
搜索文件可以包括随机产生的服务端的设备标识和服务端的配置参数,服务端的配置参数是用于选择满足条件的客户端。The search file may include a randomly generated device identifier of the server and configuration parameters of the server, and the configuration parameters of the server are used to select a client that meets the conditions.
在一些实施方式中,服务端和客户端为了实现彼此连接,需要各自生成一个设备标识ip,并在搜索阶段将设备标识传输给对方。In some implementations, in order to realize the connection between the server and the client, each needs to generate a device identifier ip, and transmit the device identifier to each other in the search phase.
搜索响应生成模块402配置为当监听到搜索文件时,基于搜索文件生成搜索响应消息。The search response generating module 402 is configured to generate a search response message based on the search file when the search file is monitored.
搜索响应消息包括客户端的设备标识和客户端的设备参数。The search response message includes the client's device identification and the client's device parameters.
在一些实施方式中,客户端接收到服务端发出的搜索文件后,识别服务端的设备标识和配置参数,判断自身设备是否支持显示,并返回搜索响应消息。In some embodiments, after receiving the search file sent by the server, the client identifies the device identifier and configuration parameters of the server, determines whether its own device supports display, and returns a search response message.
在一些实施方式中,客户端接收到服务端发出的搜索文件后,判断自身设备是否支持投屏文件的显示,如果支持显示,则向服务端返回ip匹配成功的消息;如果不支持显示,则向服务端返回ip匹配失败的消息。In some embodiments, after the client receives the search file sent by the server, it determines whether its own device supports the display of the screencast file, and if it supports the display, it returns a message indicating that the ip match is successful to the server; if it does not support the display, then Returns a message that IP matching failed to the server.
客户端发送模块403配置为向服务端返回搜索响应消息。The client sending module 403 is configured to return a search response message to the server.
客户端接收模块401还配置为接收服务端发送的投屏文件。The client receiving module 401 is further configured to receive the screen projection file sent by the server.
在一些实施方式中,投屏文件包括视频或图片,当服务端需要在客户端投屏图片时,向客户端发送图片流。当服务端需要在客户端投屏视频时,向客户端发送视频流。In some embodiments, the screen projection file includes a video or a picture, and when the server needs to project a picture on the client side, it sends a picture stream to the client. When the server needs to project video on the client, it sends a video stream to the client.
在一些实施方式中,所述投屏装置还可包括:显示模块,配置为在客户端显示投屏文件。In some embodiments, the screen projection device may further include: a display module configured to display the screen projection file on the client.
本公开实施例提供的投屏装置中,客户端接收模块配置为接收服务端发出的搜索文件;搜索文件包括随机产生的服务端的设备标识和配置参数,配置参数可以包括用于选择目标客户端的参数;客户端发送模块用配置为向服务端返回搜索响应消息;搜索响应消息包括客户端的设备标识和设备参数,服务端在搜索阶段将设备标识和配置参数传输给客户端,通过配置参数可以选择要投屏的目标客户端,即只有选定的客户端能够接受投屏,实现了差异化配置参数,而且,传输参数在搜索阶段实现预解码,不需要在视频文件传输过程中解析传输参数,提升了投屏的稳定性和用户体验。In the screen projection device provided by the embodiment of the present disclosure, the client receiving module is configured to receive a search file sent by the server; the search file includes a randomly generated device identifier and configuration parameters of the server, and the configuration parameters may include parameters for selecting a target client ; The client sending module is configured to return a search response message to the server; the search response message includes the client's device identification and device parameters, and the server transmits the device identification and configuration parameters to the client during the search phase. The target client of the screen projection, that is, only the selected client can accept the screen projection, which realizes the differentiated configuration parameters. Moreover, the transmission parameters are pre-decoded in the search stage, and there is no need to parse the transmission parameters during the video file transmission process. The stability and user experience of the projection screen are improved.
为了更好地理解上述实施例提供的投屏方法和投屏装置,下面 对投屏方法和投屏装置进行进一步说明。如图5所示,为实现投屏,服务端和客户端进行如下步骤501至504。In order to better understand the screen projection method and the screen projection device provided by the above embodiments, the screen projection method and the screen projection device are further described below. As shown in FIG. 5 , in order to realize screen projection, the server and the client perform the following steps 501 to 504 .
步骤501,服务端向客户端发送搜索文件。 Step 501, the server sends the search file to the client.
服务端向客户端发送搜索文件,以搜索可接受投屏的客户端。搜索文件包括随机产生的服务端的设备标识和配置参数,配置参数可以包括用于选择目标客户端的参数。The server sends a search file to the client to search for clients that accept screencasting. The search file includes the randomly generated device identification and configuration parameters of the server, and the configuration parameters may include parameters for selecting a target client.
步骤502,客户端向服务端返回搜索响应。 Step 502, the client returns a search response to the server.
客户端向述服务端返回搜索响应消息;搜索响应消息包括客户端的设备标识和设备参数。The client returns a search response message to the server; the search response message includes the client's device identification and device parameters.
步骤503,服务端保存客户端的设备标识。 Step 503, the server saves the device identifier of the client.
步骤504,服务端向客户端发送投屏文件。 Step 504, the server sends the screen projection file to the client.
服务端向满足要求的客户端发送投屏文件,投屏文件包括视频或图片。当服务端需要在客户端投屏图片时,向客户端发送图片流。当服务端需要在客户端投屏视频时,向客户端发送视频流。The server sends a screen projection file to the client that meets the requirements, and the screen projection file includes video or pictures. When the server needs to project a picture on the client, it sends a picture stream to the client. When the server needs to project video on the client, it sends a video stream to the client.
当服务端和客户端建立连接后,服务端和客户端可以进行数据交互,进行投屏。若在搜索ip阶段没有将服务端的屏幕尺寸发送给客户端,那么在数据交互时,客户端可以向服务端发送获取屏幕尺寸的请求,服务端基于该请求将实际屏幕尺寸发送给客户端。之后进行投屏数据的传输。After the server and the client establish a connection, the server and the client can exchange data and perform screen projection. If the screen size of the server is not sent to the client during the search ip phase, then during data interaction, the client can send a request to the server to obtain the screen size, and the server sends the actual screen size to the client based on the request. After that, transfer the screen projection data.
在一些实施方式中,投屏文件可以为图像或视频。服务端可以通过安卓调试桥(ADB)命令去截取图像或者通过安卓系统的ImageReader获取图像,然后通过网络传输图像。视频流可以通过操作系统获得屏幕流而获得,经过H264编码后传输,这样可以让画面显示的更加流畅。在操作系统中,系统代理有一个MediaCodec类以及VirtualDisplay类可以去读取屏幕流,然后将其转化为H264编码视频流。In some implementations, the screencast file can be an image or a video. The server can use the Android Debug Bridge (ADB) command to capture the image or obtain the image through the ImageReader of the Android system, and then transmit the image through the network. The video stream can be obtained by obtaining the screen stream through the operating system, and then transmitted after H264 encoding, which can make the screen display smoother. In the operating system, the system agent has a MediaCodec class and a VirtualDisplay class that can read the screen stream and convert it into an H264 encoded video stream.
在一些实施方式中,服务端通过套接字(Socket)接收客户端传过来的图像数据信息,解压缩显示到用户界面(UI)上,当用户点击UI时可获取点击位置的坐标,通过比例换算转化成实际坐标,之后通过ADB执行对应的命令,然后图像的变化再通过Socket实时传 输给客户端。In some implementations, the server receives the image data information sent by the client through a socket (Socket), decompresses it and displays it on the user interface (UI), and when the user clicks on the UI, the coordinates of the clicked position can be obtained by scaling The conversion is converted into actual coordinates, and then the corresponding command is executed through ADB, and then the change of the image is transmitted to the client through Socket in real time.
对于PC端,也可以读取安卓代理端的H264编码视频流,然后使用FFMPEG库去解码。For the PC side, you can also read the H264 encoded video stream of the Android agent, and then use the FFMPEG library to decode it.
Web应用程序首先在node.js上通过socket获取安卓代理端的H264编码视频流,然后通过WebSocket实时将字节数组传输给WebRtc,通过WebRtc的video标签去显示,之后通过js获取鼠标点击的坐标事件,之后的操作和PC很相似了,都是得到命令然后执行,然后AndroidClient再投屏图像,如此循环。The web application first obtains the H264 encoded video stream of the Android agent through socket on node.js, then transmits the byte array to WebRtc in real time through WebSocket, displays it through the video tag of WebRtc, and then obtains the coordinate event of the mouse click through js. The subsequent operations are very similar to those of the PC. They all get the command and execute it, and then the AndroidClient casts the screen image, and so on.
需要说明的是,本公开实施例中所涉及到的各模块均为逻辑模块,在实际应用中,一个逻辑模块/单元可以通过一个物理单元、或者一个物理单元的一部分、或者多个物理单元的组合实现。此外,为了突出本公开的创新部分,本公开实施例中并没有将与解决本公开所提出的技术问题关系不太密切的单元引入,但这并不表明本公开实施例中不存在其它的单元。It should be noted that the modules involved in the embodiments of the present disclosure are all logical modules, and in practical applications, a logical module/unit may pass through a physical unit, a part of a physical unit, or a combination of multiple physical units. Combination implementation. In addition, in order to highlight the innovative part of the present disclosure, the embodiments of the present disclosure do not introduce units that are not very closely related to solving the technical problems raised by the present disclosure, but this does not mean that there are no other units in the embodiments of the present disclosure. .
第五方面,参照图6,本公开实施例提供一种电子设备,其包括:In a fifth aspect, referring to FIG. 6 , an embodiment of the present disclosure provides an electronic device, which includes:
一个或多个处理器601;one or more processors 601;
存储器602,其上存储有一个或多个计算机程序,当所述一个或多个计算机程序被所述一个或多个处理器601执行时,使得所述一个或多个处理器601实现上述任意一项的投屏方法;以及The memory 602, on which one or more computer programs are stored, when the one or more computer programs are executed by the one or more processors 601, causes the one or more processors 601 to implement any one of the above the item's projection method; and
一个或多个I/O接口603,连接在所述处理器601与所述存储器602之间,配置为实现所述处理器601与所述存储器602的信息交互。One or more I/O interfaces 603 are connected between the processor 601 and the memory 602 , and are configured to implement information interaction between the processor 601 and the memory 602 .
处理器601为具有数据处理能力的器件,其包括但不限于中央处理器(CPU)等;存储器602为具有数据存储能力的器件,其包括但不限于随机存取存储器(RAM,更具体如SDRAM、DDR等)、只读存储器(ROM)、带电可擦可编程只读存储器(EEPROM)、闪存(FLASH);I/O接口(读写接口)603连接在处理器601与存储器602间,能实现处理器601与存储器602的信息交互,其包括但不限于数据总线(Bus)等。The processor 601 is a device with data processing capability, which includes but is not limited to a central processing unit (CPU), etc.; the memory 602 is a device with data storage capability, which includes but is not limited to random access memory (RAM, more specifically SDRAM) , DDR, etc.), read-only memory (ROM), electrified erasable programmable read-only memory (EEPROM), flash memory (FLASH); I/O interface (read-write interface) 603 is connected between the processor 601 and the memory 602, and can The information interaction between the processor 601 and the memory 602 is realized, which includes but is not limited to a data bus (Bus) and the like.
在一些实施方式中,处理器601、存储器602和I/O接口603 通过总线相互连接,进而与计算设备的其它组件连接。In some implementations, the processor 601, memory 602, and I/O interface 603 are interconnected by a bus and, in turn, connected to other components of the computing device.
第六方面,本公开实施例提供一种计算机可读存储介质,其上存储有计算机程序,所述计算机程序被处理器执行时实现上述任意一种投屏方法。In a sixth aspect, an embodiment of the present disclosure provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements any one of the above screen projection methods.
本领域普通技术人员可以理解,上文中所公开方法中的全部或某些步骤、系统、装置中的功能模块/单元可以被实施为软件、固件、硬件及其适当的组合。在硬件实施方式中,在以上描述中提及的功能模块/单元之间的划分不一定对应于物理组件的划分;例如,一个物理组件可以具有多个功能,或者一个功能或步骤可以由若干物理组件合作执行。某些物理组件或所有物理组件可以被实施为由处理器(如中央处理器、数字信号处理器或微处理器)执行的软件,或者被实施为硬件,或者被实施为集成电路,如专用集成电路。这样的软件可以分布在计算机可读介质上,计算机可读介质可以包括计算机存储介质(或非暂时性介质)和通信介质(或暂时性介质)。如本领域普通技术人员公知的,术语计算机存储介质包括在用于存储信息(诸如计算机可读指令、数据结构、程序模块或其它数据)的任何方法或技术中实施的易失性和非易失性、可移除和不可移除介质。计算机存储介质包括但不限于RAM、ROM、EEPROM、闪存或其它存储器技术、CD-ROM、数字多功能盘(DVD)或其它光盘存储、磁盒、磁带、磁盘存储或其它磁存储器、或者可以用于存储期望的信息并且可以被计算机访问的任何其它的介质。此外,本领域普通技术人员公知的是,通信介质通常包含计算机可读指令、数据结构、程序模块或者诸如载波或其它传输机制之类的调制数据信号中的其它数据,并且可包括任何信息递送介质。Those of ordinary skill in the art can understand that all or some of the steps in the methods disclosed above, functional modules/units in the systems, and devices can be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be composed of several physical components Components execute cooperatively. Some or all physical components may be implemented as software executed by a processor such as a central processing unit, digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit circuit. Such software may be distributed on computer-readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). As is known to those of ordinary skill in the art, the term computer storage media includes both volatile and nonvolatile implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data flexible, removable and non-removable media. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cartridges, magnetic tape, magnetic disk storage or other magnetic storage, or available with Any other medium that stores the desired information and can be accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and can include any information delivery media, as is well known to those of ordinary skill in the art .
本文已经公开了示例实施例,并且虽然采用了具体术语,但它们仅用于并仅应当被解释为一般说明性含义,并且不用于限制的目的。在一些实例中,对本领域技术人员显而易见的是,除非另外明确指出,否则与特定实施例相结合描述的特征、特性和/或元素可单独使用,或可与结合其它实施例描述的特征、特性和/或元件组合使用。因此, 本领域技术人员将理解,在不脱离由所附的权利要求阐明的本公开的范围的情况下,可进行各种形式和细节上的改变。Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should only be construed in a general descriptive sense and not for purposes of limitation. In some instances, it will be apparent to those skilled in the art that features, characteristics and/or elements described in connection with a particular embodiment may be used alone or in combination with features, characteristics described in connection with other embodiments unless expressly stated otherwise and/or components used in combination. Accordingly, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the scope of the present disclosure as set forth in the appended claims.

Claims (12)

  1. 一种投屏方法,包括:A screen projection method, including:
    发送搜索文件以搜索客户端;其中,所述搜索文件包括服务端的设备标识和服务端的配置参数,所述服务端的配置参数包括用于选择目标客户端的参数;Send a search file to search for a client; wherein, the search file includes the device identifier of the server and the configuration parameter of the server, and the configuration parameter of the server includes the parameter for selecting the target client;
    接收所述客户端返回的搜索响应消息;其中,所述搜索响应消息包括所述客户端的设备标识和设备参数;Receive a search response message returned by the client; wherein, the search response message includes the device identifier and device parameters of the client;
    依据所述搜索响应消息确定满足投屏条件的目标客户端;以及Determine a target client that satisfies the screen projection condition according to the search response message; and
    向所述目标客户端发送投屏文件。Send the screen projection file to the target client.
  2. 根据权利要求1所述的方法,其中,所述依据所述搜索响应消息确定满足投屏条件的目标客户端,包括:The method according to claim 1, wherein the determining, according to the search response message, a target client that satisfies the screen projection condition comprises:
    依据所述客户端的设备参数和设备标识生成客户端列表;其中,所述客户端列表包括按照预设的优先级顺序排列的至少一个客户端;以及generating a client list according to the device parameters and device identifiers of the client; wherein, the client list includes at least one client arranged in a preset priority order; and
    依据所述服务端的配置参数从所述客户端列表中确定所述目标客户端。The target client is determined from the client list according to the configuration parameters of the server.
  3. 根据权利要求1所述的方法,还包括:The method of claim 1, further comprising:
    接收所述客户端返回的搜索响应消息之后,依据所述服务端的配置参数和所述客户端返回的设备参数确定通讯方式,以供所述服务端与所述客户端按照所述通讯方式进行通信。After receiving the search response message returned by the client, determine the communication mode according to the configuration parameters of the server and the device parameters returned by the client, so that the server and the client can communicate according to the communication mode .
  4. 根据权利要求1所述的方法,其中,所述依据所述搜索响应消息确定满足投屏条件的所述目标客户端,包括:The method according to claim 1, wherein the determining, according to the search response message, the target client that satisfies the screen projection condition comprises:
    在所述搜索响应消息为匹配成功的消息时,确定所述搜索响应消息对应的所述客户端为满足投屏条件的目标客户端。When the search response message is a successfully matched message, it is determined that the client corresponding to the search response message is a target client that satisfies the screen projection condition.
  5. 根据权利要求1至4中任意一项所述的方法,其中,所述配 置参数包括下述一种或多种参数:屏幕大小、分辨率、系统版本、地域特性。The method according to any one of claims 1 to 4, wherein the configuration parameters include one or more of the following parameters: screen size, resolution, system version, regional characteristics.
  6. 一种投屏方法,包括:A screen projection method, including:
    监听服务端发出的搜索文件;其中,所述搜索文件包括服务端的设备标识和服务端的配置参数,所述服务端的配置参数包括用于选择目标客户端的参数;Monitor the search file sent by the server; wherein, the search file includes the device identifier of the server and the configuration parameters of the server, and the configuration parameters of the server include parameters for selecting the target client;
    当监听到所述搜索文件时,基于所述搜索文件生成并向所述服务端返回搜索响应消息,其中,所述搜索响应消息包括所述客户端的设备标识和设备参数;以及When the search file is monitored, a search response message is generated based on the search file and returned to the server, wherein the search response message includes the device identification and device parameters of the client; and
    接收所述服务端发送的投屏文件。Receive the screen projection file sent by the server.
  7. 根据权利要求6所述的方法,其中,所述配置参数包括下述一种或多种参数:屏幕大小、投屏分辨率、系统版本、地域特性。The method according to claim 6, wherein the configuration parameters include one or more of the following parameters: screen size, screen projection resolution, system version, and regional characteristics.
  8. 根据权利要求6所述的方法,其中,所述基于所述搜索文件生成并向所述服务端返回搜索响应消息,包括:The method according to claim 6, wherein the generating and returning a search response message to the server based on the search file comprises:
    基于所述搜索文件判断所述客户端是否支持显示,当不支持显示时,向所述服务端返回匹配失败的响应消息;或当支持显示时,向所述服务端返回匹配成功的响应消息。It is judged based on the search file whether the client supports display, and when the display is not supported, a response message of matching failure is returned to the server; or when display is supported, a response message of successful matching is returned to the server.
  9. 一种投屏装置,包括:A screen projection device, comprising:
    服务端发送模块,配置为向外发送搜索文件以搜索客户端;其中,所述搜索文件包括随机产生的服务端的设备标识和服务端的配置参数,所述服务端的配置参数包括用于选择目标客户端的参数;The server sending module is configured to send a search file to search for the client; wherein, the search file includes a randomly generated device identifier of the server and a configuration parameter of the server, and the configuration parameter of the server includes a configuration parameter for selecting a target client. parameter;
    服务端接收模块,配置为接收所述客户端返回的搜索响应消息;其中,所述搜索响应消息包括所述客户端的设备标识和设备参数;以及a server-side receiving module, configured to receive a search response message returned by the client; wherein the search response message includes a device identifier and device parameters of the client; and
    服务端确定模块,配置为依据所述搜索响应消息确定满足投屏条件的目标客户端;a server determination module, configured to determine a target client that satisfies the screen projection condition according to the search response message;
    其中,服务端发送模块还配置为向所述目标客户端发送投屏文件。Wherein, the server sending module is further configured to send the screen projection file to the target client.
  10. 一种投屏装置,包括:A screen projection device, comprising:
    客户端接收模块,配置为接收服务端发出的搜索文件;其中,所述搜索文件包括随机产生的服务端的设备标识和服务端的配置参数,所述服务端的配置参数包括用于选择目标客户端的参数;A client receiving module, configured to receive a search file sent by a server; wherein, the search file includes a randomly generated device identifier of the server and a configuration parameter of the server, and the configuration parameter of the server includes a parameter for selecting a target client;
    搜索响应生成模块,配置为当监听到所述搜索文件时,基于搜索文件生成搜索响应消息,其中,所述搜索响应消息包括所述客户端的设备标识和设备参数;以及a search response generation module, configured to generate a search response message based on the search file when the search file is monitored, wherein the search response message includes the device identification and device parameters of the client; and
    客户端发送模块,配置为向所述服务端返回搜索响应消息;a client sending module, configured to return a search response message to the server;
    其中,所述客户端接收模块还配置为接收所述服务端发送的投屏文件。Wherein, the client receiving module is further configured to receive the screen projection file sent by the server.
  11. 一种电子设备,包括:An electronic device comprising:
    一个或多个处理器;one or more processors;
    存储器,其上存储有一个或多个计算机程序,当所述一个或多个计算机程序被所述一个或多个处理器执行时,使得所述一个或多个处理器实现根据权利要求1至8中任意一项所述的方法;以及a memory having stored thereon one or more computer programs which, when executed by the one or more processors, cause the one or more processors to implement the one or more processors according to claims 1 to 8 The method of any one of; and
    一个或多个I/O接口,连接在所述处理器与所述存储器之间,配置为实现所述处理器与所述存储器的信息交互。One or more I/O interfaces, connected between the processor and the memory, are configured to realize the information interaction between the processor and the memory.
  12. 一种计算机可读存储介质,其上存储有计算机程序,所述计算机程序被处理器执行时实现根据权利要求1至8中任意一项所述的方法。A computer-readable storage medium having a computer program stored thereon, the computer program implementing the method according to any one of claims 1 to 8 when executed by a processor.
PCT/CN2021/104752 2020-08-05 2021-07-06 Screen projection method and apparatus, electronic device, and computer readable storage medium WO2022028189A1 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
JP2022572724A JP2023527393A (en) 2020-08-05 2021-07-06 Screen mirroring method and apparatus, electronic device and computer readable storage medium
US18/019,853 US20230333800A1 (en) 2020-08-05 2021-07-06 Screen projection method and apparatus, electronic device and computer readable storage medium
EP21853925.2A EP4180942A1 (en) 2020-08-05 2021-07-06 Screen projection method and apparatus, electronic device, and computer readable storage medium
KR1020227038309A KR20220162775A (en) 2020-08-05 2021-07-06 Screen projection method and device, electronic device and computer readable storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010777220.0A CN114077409A (en) 2020-08-05 2020-08-05 Screen projection method and device, electronic equipment and computer readable medium
CN202010777220.0 2020-08-05

Publications (1)

Publication Number Publication Date
WO2022028189A1 true WO2022028189A1 (en) 2022-02-10

Family

ID=80116887

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/104752 WO2022028189A1 (en) 2020-08-05 2021-07-06 Screen projection method and apparatus, electronic device, and computer readable storage medium

Country Status (6)

Country Link
US (1) US20230333800A1 (en)
EP (1) EP4180942A1 (en)
JP (1) JP2023527393A (en)
KR (1) KR20220162775A (en)
CN (1) CN114077409A (en)
WO (1) WO2022028189A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114827690A (en) * 2022-03-30 2022-07-29 北京奇艺世纪科技有限公司 Network resource display method, device and system

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180241609A1 (en) * 2017-02-17 2018-08-23 Wyse Technology L.L.C. Remote casting of media content
CN108810594A (en) * 2018-05-04 2018-11-13 新华三技术有限公司 Remote screen projection method, device and system
CN108989879A (en) * 2018-08-28 2018-12-11 广州视源电子科技股份有限公司 Throw control method, the device and system of screen
CN110602087A (en) * 2019-09-10 2019-12-20 腾讯科技(深圳)有限公司 Intelligent screen projection method and device, intelligent terminal and server
CN111427527A (en) * 2020-03-20 2020-07-17 海信视像科技股份有限公司 Screen projection method, device, equipment and computer readable storage medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180241609A1 (en) * 2017-02-17 2018-08-23 Wyse Technology L.L.C. Remote casting of media content
CN108810594A (en) * 2018-05-04 2018-11-13 新华三技术有限公司 Remote screen projection method, device and system
CN108989879A (en) * 2018-08-28 2018-12-11 广州视源电子科技股份有限公司 Throw control method, the device and system of screen
CN110602087A (en) * 2019-09-10 2019-12-20 腾讯科技(深圳)有限公司 Intelligent screen projection method and device, intelligent terminal and server
CN111427527A (en) * 2020-03-20 2020-07-17 海信视像科技股份有限公司 Screen projection method, device, equipment and computer readable storage medium

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114827690A (en) * 2022-03-30 2022-07-29 北京奇艺世纪科技有限公司 Network resource display method, device and system
CN114827690B (en) * 2022-03-30 2023-07-25 北京奇艺世纪科技有限公司 Network resource display method, device and system

Also Published As

Publication number Publication date
EP4180942A1 (en) 2023-05-17
CN114077409A (en) 2022-02-22
JP2023527393A (en) 2023-06-28
US20230333800A1 (en) 2023-10-19
KR20220162775A (en) 2022-12-08

Similar Documents

Publication Publication Date Title
KR101467430B1 (en) Method and system for providing application based on cloud computing
US9432672B2 (en) Image compression method and system with image compression time information
US9277030B2 (en) Stream processing using a client-server architecture
CN111309279B (en) Multi-screen interactive display method and device
US20220053244A1 (en) Method and device for managing processing of media content, and computer-readable recording medium
US10296514B2 (en) Information processing system, method for acquiring information, information processing apparatus and recording medium
CN112799825A (en) Task processing method and network equipment
EP3059945A1 (en) Method and system for video surveillance content adaptation, and central server and device
CN111291292A (en) Page rendering method and device, terminal equipment and storage medium
CN104349177A (en) Method for turning to play multimedia file under desktop cloud, virtual machine and system
US10277652B2 (en) Transmission apparatus, transmission method, and program
CN110944220A (en) Video screen projection playing method, device, equipment and system
WO2022028189A1 (en) Screen projection method and apparatus, electronic device, and computer readable storage medium
CN114040232B (en) Screen projection system, screen projection method, electronic equipment and storage medium
CN104639501A (en) Data stream transmission method, equipment and system
WO2016197783A2 (en) Method and apparatus for controlling message transmission
CN115865884A (en) Network camera data access device and method, network camera and medium
CN110677443A (en) Data transmitting and receiving method, transmitting end, receiving end, system and storage medium
CN106899652B (en) Method and device for pushing service processing result
US11303692B2 (en) Communication apparatus for transmitting response, communication method, and storage medium
CN115349248A (en) Network-based media processing (NBMP) deployment with real-time uplink streaming framework (plus) and 5G Application Function (AF)
CN110392017B (en) Method and related device for processing RPC message
CN112511884A (en) Mixed flow control method and system of audio and video stream and storage medium
EP3618349A1 (en) Conference cascading method based on software media server, and server and storage medium
WO2018227853A1 (en) Document printing apparatus, method and system and computer readable storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21853925

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 20227038309

Country of ref document: KR

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2022572724

Country of ref document: JP

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2021853925

Country of ref document: EP

Effective date: 20230213

NENP Non-entry into the national phase

Ref country code: DE