WO2022252977A1 - 投屏方法、存储介质及通信设备 - Google Patents

投屏方法、存储介质及通信设备 Download PDF

Info

Publication number
WO2022252977A1
WO2022252977A1 PCT/CN2022/093260 CN2022093260W WO2022252977A1 WO 2022252977 A1 WO2022252977 A1 WO 2022252977A1 CN 2022093260 W CN2022093260 W CN 2022093260W WO 2022252977 A1 WO2022252977 A1 WO 2022252977A1
Authority
WO
WIPO (PCT)
Prior art keywords
screen projection
target object
state
data
sending end
Prior art date
Application number
PCT/CN2022/093260
Other languages
English (en)
French (fr)
Inventor
张帆
Original Assignee
广州视源电子科技股份有限公司
广州视睿电子科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 广州视源电子科技股份有限公司, 广州视睿电子科技有限公司 filed Critical 广州视源电子科技股份有限公司
Publication of WO2022252977A1 publication Critical patent/WO2022252977A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units

Definitions

  • the present application relates to the communication field, for example, to a screen projection method, a storage medium and a communication device.
  • Screen projection refers to the technology of transmitting the specific screen, sound and other information of one device to another device through wireless transmission or wired transmission and other data transmission methods and displaying it in real time.
  • the most common projection scenario is to project the specific screen or/and sound of user equipment such as a user's computer or mobile phone to a receiver such as a TV, monitor, smart interactive tablet or projection box.
  • the sending end is an operating system that supports multiple windows
  • the user can also select one of the open application windows, such as the text compilation window, and only project the content of this window to the receiving end, so as to avoid displacing the entire screen.
  • the content is projected to the receiving end, which leads to problems such as leakage of user privacy.
  • the sending software needs to collect the window image of the target object through the system application program interface (Application Programming Interface, API), which is limited by the interface of most systems. If the window being projected is minimized, The screen projection sender software will not be able to capture the screen content of the target object window. At this time, the screen projection display device may display the last frame of the screen before the window was minimized, and the screen may freeze, making the screen projection impossible.
  • API Application Programming Interface
  • the embodiment of the present application provides a screen projection method, a storage medium, and a communication device, which can solve the problem that the screen is still and the screen projection cannot be performed normally. Described technical scheme is as follows:
  • the embodiment of the present application provides a screen projection method, the method is applied to the sending end, the sending end is connected to the receiving end, and the receiving end is used to display the screen projection data of the sending end, including: the sending end obtains the user Triggered window projection operation for the target object; the sending end sends the first screen projection data of the target object to the receiving end, wherein the first screen projection data includes the content of the target object displayed on the display interface of the sending end ; The sending end detects the state of the target object; if the target object is in a preset state, then the sending end sends the second screen projection data to the receiving end; wherein, the second screen projection data is used to prompt that the target object is in the A preset state, the preset state includes a minimized state or a closed state.
  • the method further includes: the sending end periodically collects the screen projection of the target object, wherein the first screen projection data is generated by the sending end according to the screen projection; and the sending end detects The state of the target object includes: the sending end detects the state of the target object before each period of collecting the projected screen image.
  • the second screen projection data includes preset image data or preset text data.
  • the second screen projection data includes data of a preset image
  • the sending end sends the second screen projection data to the receiving end, including: the sending end reads the preset image, and decodes the preset image to obtain the preset image.
  • the data of the image is set; the sending end encodes the data of the preset image to obtain the second screen projection data; the sending end sends the second screen projection data to the receiving end.
  • the method further includes: if the target object is not in the preset state, then the sending end continues to send the first screen projection data to the receiving end.
  • an embodiment of the present application provides a communication device, the screen projection device is connected to a receiving end, and the receiving end is used to display the screen projection data of the screen projection device, including: an acquisition unit, configured to acquire user-triggered The window projection operation for the target object; the transceiver unit, configured to send the first screen projection data of the target object to the receiving end, wherein the first screen projection data includes the target displayed on the display interface of the screen projection device The content of the object; the processing unit is used to detect the state of the target object; the transceiver unit is also used to send the second screen projection data to the receiving end when the target object is in a preset state; wherein, the second screen projection The data is used to prompt that the target object is in the preset state, and the preset state includes a minimized state or a closed state.
  • the processing unit is further configured to periodically collect the screen projection of the target object, wherein the first screen projection data is generated by the processing unit according to the screen projection; and the processing unit is configured to The state of the target object is detected before each cycle of capturing the projected screen image.
  • the second screen projection data includes preset image data or preset text data.
  • the second screen projection data includes data of a preset image
  • the processing unit is also used to read the preset image, decode the preset image to obtain the data of the preset image; the processing unit is also used to The data of the preset image is encoded to obtain the second screen projection data.
  • the transceiving unit is further configured to continue sending the first screen projection data to the receiving end when the target object is not in the preset state.
  • an embodiment of the present application provides a computer storage medium, where a plurality of instructions are stored in the computer storage medium, and the instructions are suitable for being loaded by a processor to execute the above method steps.
  • the embodiment of the present application provides an electronic device, which may include: a processor and a memory; wherein, the memory stores a computer program, and the computer program is adapted to be loaded by the processor and execute the above method steps.
  • the sending end detects the state of the target object during the process of sending the projection data corresponding to the window screen of the target object to the receiving end. Closed state), the sender sends the preset projection data to the receiver to remind the user why the projection data cannot be obtained.
  • the projection display terminal can display the preset projection data on the display interface, and can prompt the user why the projection data cannot be obtained, so that the user can restore the projection according to the prompt.
  • FIG. 1 is a schematic diagram of an application scenario of a screen projection control method provided by an embodiment of the present application
  • Fig. 2 is another schematic diagram of the application scenario of the screen projection control method provided by the embodiment of the present application.
  • Fig. 3 is a schematic flowchart of window projection provided by the embodiment of the present application.
  • FIG. 4 is a schematic flowchart of a screen projection method provided by an embodiment of the present application.
  • FIG. 5 is a schematic diagram of a conference scene provided by an embodiment of the present application.
  • Fig. 6 is another schematic flowchart of the screen projection method provided by the embodiment of the present application.
  • FIG. 7 is a schematic diagram of a screen projection method provided by an embodiment of the present application.
  • FIG. 8 is a schematic structural diagram of a screen projection device provided by an embodiment of the present application.
  • FIG. 9 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • FIG. 1 is a schematic diagram of an application scenario 100 of a screen projection control method provided by an embodiment of the present application.
  • the embodiment of the present application can be applied in a conference scene 100, which includes at least one screen projection receiving end (the screen projection receiving end can also be called a screen projection receiving end device or a receiving end), such as The screen projection receiving end 101 shown in FIG. 1 .
  • the conference scene 100 may also include at least two screen projection sending ends (the screen projection sending end may also be referred to as a screen projection sending end device or a sending end), and the screen projection sending end may be a screen projection sending end as shown in FIG. 1 end 102, screen projection sending end 103.
  • the screen projection receiver 101 can establish a communication connection with a plurality of screen projection transmitters. As shown in FIG.
  • the communication connection may also be a wireless communication connection, which is not limited in this application.
  • the screen projection receiving end 101 can receive the screen projection data from the screen projection sending end 102 and the screen projection data from the screen projection sending end 103 .
  • the screen projection data may include screen projection image information and/or screen projection sound information.
  • the screen projection receiving end 101 is equipped with a screen, and the screen projection receiving end 101 can simultaneously display images and sounds from the screen projection sending end 102 and the screen projection sending end 103 on the screen. But the present application is not limited thereto.
  • FIG. 2 is a schematic diagram of an application scenario 200 of the screen projection control method provided by the embodiment of the present application.
  • the screen projection receiving end 202 may be a screen projection processing device, for example, the screen projection receiving end 202 may be a screen transmission box, but the present application is not limited thereto.
  • the screen projection receiver 202 can receive the screen projection data from the screen projection sender 203 and the screen projection sender 204, and forward the received screen projection data to the connected display 201, and control the display 201 to display the data from the screen projection sender 203 and the screen projection sender 204.
  • Screen projection sending end 204 screen, and play sound, wherein, the communication connection between projection screen receiving end 202 and screen projection sending end 203 and 204, display 201 can be wireless communication connection, also can be wired communication connection, this application There is no limit to this.
  • the screen projection sending end provided in the embodiment of the present application may be a user terminal, and the user terminal may be a terminal device such as a mobile phone, a tablet, or a notebook computer.
  • the screen projection receiving end provided in the embodiment of the present application may be a communication device with a screen transmission function such as a TV, a projector, an intelligent interactive tablet, a computer device, or a screen transmission box.
  • the screen projection receiving end may use wired communication or wireless communication The way to receive the projection data from the sender.
  • Figure 3 is an example of window projection. As shown in Figure 3, the process of window projection may include but not limited to the following steps:
  • the sender starts the screen projection sender software
  • the screen projection sender software establishes a communication connection with the screen projection receiver software
  • the user selects the target object for window projection in the projection sending software
  • the sending software collects the window screen of the target object through the system API
  • the screen projection sender software performs video encoding on the collected window images to obtain projection data
  • the screen projection sender software sends the screen projection data to the receiver through the network, etc., and executes the subsequent steps from step 3 again.
  • the sender software needs to collect the window screen of the target object through the system API.
  • the The screen sender software will not be able to collect the screen content of the target object window.
  • the projection display device may display the last frame of the window before the window was minimized, and the screen will remain still. It will be confusing for users who don't understand the situation, or mistakenly think that the screen is still due to connection failure or poor network, which makes the screen projection impossible.
  • the embodiment of the present application proposes that the user can be prompted about the reason for the abnormal screen projection, so that the user can restore the screen projection according to the prompt.
  • FIG. 4 is a schematic flowchart of a screen projection method provided by an embodiment of the present application.
  • the sending end is connected to the receiving end, and the connection may be a wired communication connection or a wireless communication connection.
  • the sending end In response to the window projection operation of the target object triggered by the user, the sending end sends the first screen projection data of the target object to the receiving end, where the first screen projection data includes the target object displayed on the display interface of the sending end. content.
  • the user can select the target object that needs to perform window projection in the screen projection sender software, and execute the window projection operation to trigger the window projection of the target object.
  • the sending end obtains the user's operation through the screen projection sending end software, in response to the user's window projection operation, the sending end sends the first screen projection data of the target object to the receiving end.
  • the sending end obtains the user's window projection operation, and can send request information for requesting screen projection to the receiving end.
  • the receiving end can send a message to the sending end
  • the sending end may send the first screen projection data of the target object to the receiving end after receiving the response information from the receiving end.
  • the target object may be a window of an application.
  • the windows of multiple applications on the display interface of the sending end are open and displayed on the display interface, and the user can select the target application that needs to perform window projection among the multiple applications in the screen projection sending software. Window mirroring operation.
  • the present application is not limited thereto.
  • the windows of the multiple application programs in the open state may overlap each other or be covered (or blocked) by other windows, but the windows are in the open state, that is, the non-minimized state or the closed state.
  • the sender can use the system API to collect the content displayed on the display interface of the target object (that is, the screen projection screen).
  • the API includes but is not limited to "BitBlt”, “PrintWindow”, or “StretchBlt”.
  • the API includes but not limited to "XGetImage” or "xcb_shm_get_image”.
  • this API includes but is not limited to "CGWindowListCreateImage”.
  • the sending end collects the projection screen of the target object, performs video encoding on the projection screen, obtains first projection data, and sends the first projection data to the receiving end. After receiving the first screen projection data, the receiving end performs video decoding and other operations, and then displays the screen projection image of the target object on the display interface.
  • the sending end detects the state of the target object, and determines whether the state of the target object is in a preset state.
  • the sending end detects the state of the target object to determine whether the target object is in a preset state, and the preset state includes a minimized state or a closed state.
  • the operating system of the sending end is a Windows system
  • the functions used by the sending end to judge the state of the target object include but not limited to the IsWindowVisible function and/or the IsIconic function.
  • the screen projection sender software can call the IsWindowVisible function to determine whether the state of the target object is displayed or hidden. If the screen projection sending end software determines that the state of the target object is hidden through the IsWindowVisble function, the target object is in the default state.
  • the screen projection sender software can also call the IsIconic function. If the return value of the IsIconic function is "true”, the screen projection sender software can determine that the target object is in a minimized state. If the return value of the IsIconic function is is "false", then the screen projection sending software can determine that the target object is in the closed state. But the present application is not limited thereto.
  • the preset state includes a minimized state
  • the screen projection sender software can call the IsIconic function. If the return value of the IsIconic function is "true”, the screen projection sender software can determine that the target object is in the preset state. That is, the minimized state. If the return value of the IsIconic function is "false", the screen projection sending end software may determine that the target object is not in a minimized state, but the present application is not limited thereto.
  • the operating system of the sending end is a macOS system
  • the functions used by the sending end to judge the state of the target object include but not limited to CGWindowListCopyWindowInfo function and/or CGWindowListCreate function.
  • the screen projection sender software can call the CGWindowListCreate function to obtain a list of all visible windows. If the target object is not included in the visible window list, the screen projection sender software can determine that the target object is not visible, that is, the target object is in the preset state. Optionally, the screen projection sender software can also call CGWindowListCopyWindowInfo to obtain a list of all open windows. If the target object is not included in the open window list, the screen projection sender software can determine that the target object is in a closed state. , if the target object is included in the window list in the open state, then the screen projection sending software may determine that the target object is in a minimized state, but the present application is not limited thereto.
  • the preset state includes the closed state
  • the screen projection sender software can call the CGWindowListCopyWindowInfo function to determine whether the target object is in the closed state according to the window list in the open state, but the application is not limited thereto.
  • the operating system of the sending end is the GUN/Linux system
  • the function used by the sending end to judge the state of the target object includes but not limited to the XGetWindowAttributes function and/or the XGetWindowProperty function.
  • the screen projection sender software can call the XGetWindowAttributes function of the target object. If the XGetWindowAttributes function is successfully called, it is determined that the target object exists. If the call of the XGetWindowAttributes function fails, it is determined that the target object does not exist, or that the target object is in a preset state.
  • the screen projection sending end software can also call the XGetWindowProperty function of the target object to obtain the _NET_WM_STATE attribute, and use this attribute to determine whether the target object is hidden, thereby determining the target object Whether the object is minimized. But the present application is not limited thereto.
  • the default state includes the minimized state.
  • the screen projection sender software can call the XGetWindowProperty function of the target object to obtain the _NET_WM_STATE property, and use this property to determine whether the target object is hidden, thereby determining whether the target object is in the default state, that is, minimum status. But the present application is not limited thereto.
  • the target object is in a preset state
  • send second screen projection data to the receiving end the second screen projection data is used to remind the target object that it is in a preset state, where the preset state includes a minimized state or a closed state.
  • the sending end can send the second screen projection data to the receiving end, so that the receiving end can display the second screen projection data on the display interface after receiving the second screen projection data
  • the decoding content of the data achieves the purpose of prompting the user that the target object is in a preset state. This enables the user to know the status of the target object according to the content displayed on the receiving end, and to process it in time. Realizing that the sender actively informs the user of the status of the screen projection window can improve the user experience.
  • the second screen projection data includes preset image data or preset text data.
  • the default state is the minimized state, and the user is in the process of projecting a video application.
  • the video application may be minimized due to misoperation by other users or the influence of other software or operating systems.
  • the sender detects that the video application is minimized, it can read the preset image.
  • the preset image includes text such as "The projection window has been minimized and the screen cannot be captured. Please cancel the minimization" .
  • the sending end After reading the preset image, the sending end decodes the preset image to obtain the data of the preset image, and then performs video encoding on the data of the preset image to obtain the second projection data, and sends it to the receiving end, and the receiving end decodes it
  • the preset image can be displayed on the display interface of the receiving end, so that the user can know that the sending end cannot collect the screen projection image due to the minimization of the screen projection window. Therefore, perform corresponding operations to restore the screen projection status of the screen projection window.
  • the present application is not limited thereto.
  • the default state is closed, and the user is in the process of projecting the target object, and the sender detects that the target object is not turned on or has been turned off, that is, the state of the target object is turned off, and the sender can Read the preset image, for example, the preset image includes text such as "the screen projection window has been closed and the screen cannot be captured".
  • the sending end generates second screen projection data according to the preset image, and sends it to the receiving end.
  • the receiver decodes and displays the preset image on the display interface, so that the user can know that the screen projection window cannot be collected by the sender because the screen projection window is closed. Therefore, perform corresponding operations to restore the screen projection status of the screen projection window.
  • the present application is not limited thereto.
  • the format of the preset image includes but not limited to PNG, JPG and so on.
  • the format of the data of the preset image obtained by decoding the preset image (which may be referred to as the original data of the preset image) at the sending end may be RGBA, YUV and the like.
  • the sending end may periodically collect the projected screen image of the target object, and the sending end may detect the state of the target object before each cycle of collecting the projected screen image.
  • the sending end detects the status of the screen projection object during the screen projection process, and sends the preset projection data to the receiver when it is in the preset state, so that the receiving end displays a prompt message, indicating that the screen projection cannot be collected
  • the reason is that the sender can actively inform the user of the status of the screen projection window, which can improve the user experience.
  • FIG. 5 is a schematic diagram of a conference scene provided by an embodiment of the present application. As shown in Figure 5, the user chooses to perform window projection of the video application program on the screen projection software of the sending end, and can share the video with the participants through window projection.
  • the sender software can use the screen projection method provided by the embodiment of the present application during the window projection process of the video application program, as shown in Figure 6, the method includes but is not limited to the following steps:
  • the sending end software detects the status of the video application program.
  • the software at the sending end determines whether the state of the video application program is a preset state.
  • the sending software executes S603. Then execute S606 and S607. If the state of the video application program is the default state, the screen projection sender software executes S604.
  • the sending end software collects the window image of the video application program through the system API.
  • the software at the sending end executes S606 and S607 after executing S603.
  • the sending software reads the preset image data.
  • the sending end may store the original data of the preset picture, that is, the picture data that has not been encoded and compressed, or the sending end may store the encoded and compressed data of the original data of the preset picture. This application does not limit this.
  • the sending end stores the original data of the preset picture, that is, the preset picture data is the original data of the preset picture, then the software of the sending end reads the preset picture data and executes S606.
  • the sender stores the coded and compressed data of the preset picture, that is, the preset picture data is the encoded and compressed data of the original data of the preset picture, and the software of the sender reads the preset picture. After the image data is set, S605 and subsequent steps are executed.
  • the sending-end software decodes the preset picture data to obtain the original data of the picture.
  • the format of the preset picture includes but not limited to PNG, JPG and so on.
  • the format of the original data of the picture may be RGBA, YUV, etc. But the present application is not limited thereto.
  • the software at the sending end performs video encoding on the original data of the window picture or picture to obtain screen projection data.
  • the default state is the minimized state. If the sending-end software detects that the video application is not in the minimized state, the sending-end software performs video encoding on the window screen to obtain screen projection data including the window screen. If the software at the sending end detects that the video application program is in a minimized state, the software at the sending end performs video encoding on the original data of the picture to obtain projection data including the original data of the picture.
  • the default state is the closed state. If the sending-end software detects that the video application is not in the closed state, the sending-end software performs video encoding on the window screen to obtain screen projection data including the window screen. If the software at the sending end detects that the video application program is in a closed state, the software at the sending end performs video encoding on the original data of the picture to obtain projection data including the original data of the picture.
  • the software at the sending end sends the screen projection data to the software at the receiving end.
  • the software at the receiving end receives the screen projection data from the software at the sending end.
  • the software at the receiving end performs video decoding on the projection data and displays it on the display interface. If the decoded window picture is obtained, the receiving-end software displays the window picture; if the decoded original picture data is obtained, the receiving-end software displays the preset picture.
  • the default state is the minimized state
  • the preset picture includes prompt information, such as "the screen projection window has been minimized, and the screen cannot be collected, please cancel the minimization", and the receiving end can display the preset picture on the display interface. content.
  • the participants can learn from the prompt information displayed on the receiving end that the sending end cannot capture the screen because the screen projection window is minimized.
  • the software at the sending end actively informs the user of the reason why the screen cannot be collected, so that the user can restore the screen projection status of the window in time, which improves the user experience.
  • the default state is the closed state
  • the preset picture includes prompt information, such as "the screen projection window is closed, and the picture cannot be collected", and the receiving end can display the content of the preset picture on the display interface. Participants can use the prompt information displayed on the receiving end to know that the sending end cannot capture the screen because the screen projection window is closed.
  • the software at the sending end actively informs the user of the reason why the screen cannot be collected, so that the user can restore the screen projection status of the window in time, which improves the user experience.
  • the sending end software executes the screen projection method provided by the embodiment of the present application as an example. It should be understood that the operation performed by the sending end software can be understood as the sending end running the sending end software. implemented operations.
  • FIG. 8 shows a schematic structural diagram of a screen projection device provided by an exemplary embodiment of the present application.
  • the screen projection device can be implemented as all or a part of the intelligent interactive panel through software, hardware or a combination of the two.
  • the device includes an acquisition unit 801 , a processing unit 802 and a transceiver unit 803 .
  • the acquiring unit is configured to acquire a window mirroring operation triggered by a user for a target object.
  • the transceiving unit is configured to send the first screen projection data of the target object to the receiving end, wherein the first screen projection data includes the content of the target object displayed on the display interface of the sending end.
  • a processing unit configured to detect the state of the target object.
  • the transceiver unit is also used to send the second screen projection data to the receiving end when the target object is in a preset state; wherein, the second screen projection data is used to prompt that the target object is in the preset state, and the preset States include minimized or closed states.
  • the screen projection device further includes a storage unit 804, which is used for storing a screen projection application program.
  • the processing unit is further configured to periodically collect the screen projection of the target object, wherein the first screen projection data is generated by the processing unit according to the screen projection; and the processing unit is configured to The state of the target object is detected before each cycle of capturing the projected screen image.
  • the second screen projection data includes preset image data or preset text data.
  • the second screen projection data includes data of a preset image
  • the processing unit is also used to read the preset image, decode the preset image to obtain the data of the preset image; the processing unit is also used to The data of the preset image is encoded to obtain the second screen projection data.
  • the transceiver unit is further configured to continue sending the first screen projection data to the receiving end when the target object is not in the preset state.
  • the electronic device 900 may include: at least one processor 901 , at least one network interface 904 , a user interface 903 , a memory 905 and at least one communication bus 902 .
  • the communication bus 902 is used to realize connection and communication between these components.
  • the user interface 903 may include a display screen (Display) and a camera (Camera), and the optional user interface 903 may also include a standard wired interface and a wireless interface.
  • Display display screen
  • Camera Camera
  • the optional user interface 903 may also include a standard wired interface and a wireless interface.
  • the network interface 904 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).
  • the processor 901 may include one or more processing cores.
  • the processor 901 uses various interfaces and lines to connect various parts in the entire electronic device 900, and executes or executes by running or executing instructions, programs, code sets or instruction sets stored in the memory 905, and calling data stored in the memory 905.
  • the processor 901 may adopt at least one of Digital Signal Processing (Digital Signal Processing, DSP), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA), and Programmable Logic Array (Programmable Logic Array, PLA). implemented in the form of hardware.
  • DSP Digital Signal Processing
  • FPGA Field-Programmable Gate Array
  • PLA Programmable Logic Array
  • the processor 901 may integrate one or a combination of a central processing unit (Central Processing Unit, CPU), an image processor (Graphics Processing Unit, GPU), a modem, and the like.
  • CPU Central Processing Unit
  • GPU Graphics Processing Unit
  • the CPU mainly handles the operating system, user interface and application programs, etc.
  • the GPU is used to render and draw the content that needs to be displayed on the display screen
  • the modem is used to handle wireless communication. It can be understood that the foregoing modem may not be integrated into the processor 901, but may be implemented by a single chip.
  • the memory 905 may include a random access memory (Random Access Memory, RAM), and may also include a read-only memory (Read-Only Memory).
  • the memory 905 includes a non-transitory computer-readable storage medium (non-transitory computer-readable storage medium). The memory 905 may be used to store instructions, programs, codes, sets of codes, or sets of instructions.
  • the memory 905 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), Instructions and the like for implementing the above method embodiments; the storage data area can store the data and the like involved in the above method embodiments.
  • the memory 905 may also be at least one storage device located away from the aforementioned processor 901 .
  • the memory 905 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a screen-casting application program.
  • the user interface 903 is mainly used to provide the user with an input interface to obtain the data input by the user; and the processor 901 can be used to call the screen projection application program stored in the memory 905 and execute Do the following:
  • Obtain a window projection operation triggered by the user for the target object send the first projection data of the target object to the receiving end, wherein the first projection data includes the content of the target object displayed on the display interface; detect the The state of the target object; if the target object is in the preset state, then send the second screen projection data to the receiving end; wherein, the second screen projection data is used to prompt the target object to be in the preset state, and the preset state includes Minimized state or closed state.
  • periodically collect the screen projection of the target object, wherein the first screen projection data is generated according to the screen projection; and detecting the state of the target object includes: collecting the screen projection Before each cycle of , check the state of the target object.
  • the second screen projection data includes preset image data or preset text data.
  • the second screen projection data includes data of a preset image
  • sending the second screen projection data to the receiving end includes: reading the preset image, decoding the preset image to obtain the data of the preset image; Encoding the data of the preset image to obtain the second screen projection data; sending the second screen projection data to the receiving end.
  • the method further includes: if the target object is not in the preset state, continuing to send the first screen projection data to the receiving end.
  • the embodiment of the present application also provides a computer storage medium, the computer storage medium can store a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the method steps of the embodiments shown in the above-mentioned Fig. 4 to Fig. 7
  • the execution process reference may be made to the descriptions of the embodiments shown in FIG. 4 to FIG. 7 , and details are not described here.
  • the device where the storage medium is located may be a communication device.
  • the hardware part of the communication device can be composed of a touch display module, an intelligent processing system (including a controller), etc., which are combined by integral structural parts and supported by a dedicated software system.
  • the touch display module It includes a display screen, a touch component and a backlight component.
  • the backlight component is used to provide a backlight source for the display screen.
  • the display screen generally uses a liquid crystal display device for displaying images.
  • the touch component is set on the display screen or on the display The front end of the screen is used to collect the user's touch operation data, and send the collected touch operation data to the intelligent processing system for processing.
  • the display screen of the communication device displays picture data.
  • a touch object such as a finger or a stylus pen
  • the touch display mode The touch component of the group will collect the touch data, so that the touch component converts the touch data into the coordinate data of the touch point and then sends it to the intelligent processing system, or sends it to the intelligent processing system, and the intelligent processing system converts it into a touch point coordinate data.
  • the coordinate data of the control point after the intelligent processing system obtains the coordinate data of the touch point, realizes the corresponding control operation according to the preset program, drives the display content to change, and realizes various display and operation effects.
  • touch components can be divided into five basic categories: vector pressure sensing technology touch components, resistive technology touch components, capacitive technology touch components, electromagnetic technology touch screens, infrared technology touch components, and surface acoustic wave technology touch components.
  • the touch component can be divided into four types: resistive, capacitive induction, electromagnetic induction, infrared and surface acoustic wave.
  • the touch component can collect the data of the touch point and send it to the intelligent processing system, and then implement different functional applications with the built-in software of the intelligent processing system, thereby realizing the control of the intelligent processing system Touch controls.
  • the embodiments of the present application may be provided as methods, systems, or computer program products. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
  • computer-usable storage media including but not limited to disk storage, CD-ROM, optical storage, etc.
  • These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions
  • the device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.
  • a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
  • processors CPUs
  • input/output interfaces network interfaces
  • memory volatile and non-volatile memory
  • Memory may include non-permanent storage in computer readable media, in the form of random access memory (RAM) and/or nonvolatile memory such as read only memory (ROM) or flash RAM.
  • RAM random access memory
  • ROM read only memory
  • flash RAM flash random access memory
  • Computer-readable media including both permanent and non-permanent, removable and non-removable media, can be implemented by any method or technology for storage of information.
  • Information may be computer readable instructions, data structures, modules of a program, or other data.
  • Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Flash memory or other memory technology, Compact Disc Read-Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cartridge, tape magnetic disk storage or other magnetic storage device or any other non-transmission medium that can be used to store information that can be accessed by a computing device.
  • computer-readable media excludes transitory computer-readable media, such as modulated data signals and carrier waves.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Controls And Circuits For Display Device (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

本申请实施例提供了一种投屏方法、存储介质及通信设备，属于通信技术领域。该方法应用于发送端，该发送端与接收端连接，该接收端用于显示该发送端的投屏数据，该方法包括：该发送端获取用户触发的针对目标对象的窗口投屏操作；该发送端向该接收端发送该目标对象的第一投屏数据，其中，该第一投屏数据包括该发送端的显示界面上显示的该目标对象的内容；该发送端检测该目标对象的状态；若该目标对象处于预设状态，则该发送端向该接收端发送第二投屏数据；其中，该第二投屏数据用于提示该目标对象处于该预设状态，该预设状态包括最小化状态或关闭状态。

Description

投屏方法、存储介质及通信设备
本申请要求于2021年05月31日提交国家知识产权局、申请号为202110604190.8、发明名称为“投屏方法、存储介质及通信设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请。
技术领域
本申请涉及通信领域，例如涉及一种投屏方法、存储介质及通信设备。
背景技术
投屏是指将一台设备的特定画面、声音等信息通过无线传输或有线传输等数据传输方式传输到另一台设备并实时显示出来的技术。目前最常见的投屏场景是将用户电脑或手机等用户设备的特定画面或/和声音投屏到电视、显示器、智能交互平板或投屏盒子等接收端。
当发送端为支持多窗口的操作系统时，用户还可以选择已打开的其中一个应用程序的窗口，如文本编译窗口，仅将该窗口的内容投屏至接收端，以避免在将整个屏幕的内容投屏至接收端，而导致用户隐私泄露等问题。
然而，在进行窗口投屏时，发送端软件需要通过系统应用程序接口(Application Programming Interface，API)采集目标对象的窗口画面，受到大部分系统的接口限制，若正在投屏的窗口最小化时，投屏发送端软件将无法采集到该目标对象窗口画面内容。此时投屏显示设备可能显示该窗口最小化前的最后一帧画面，出现画面静止不动的情况，使得投屏不能正常进行。
发明内容
本申请实施例提供了一种投屏方法、存储介质及通信设备，可以解决画面静止不动、投屏不能正常进行的问题。所述技术方案如下：
第一方面，本申请实施例提供了一种投屏方法，该方法应用于发送端，该发送端与接收端连接，该接收端用于显示该发送端的投屏数据，包括：发送端获取用户触发的针对目标对象的窗口投屏操作；该发送端向接收端发送该目标对象的第一投屏数据，其中，该第一投屏数据包括该发送端的显示界面上显示的该目标对象的内容；该发送端检测该目标对象的状态；若该目标对象处于预设状态，则该发送端向接收端发送第二投屏数据；其中，该第二投屏数据用于提示该目标对象处于该预设状态，该预设状态包括最小化状态或关闭状态。
可选的，该方法还包括：该发送端周期性地采集该目标对象的投屏画面，其中，该第一投屏数据为该发送端根据该投屏画面生成的；以及，该发送端检测该目标对象的状态，包括：该发送端在采集该投屏画面的每个周期之前，检测该目标对象的状态。
可选的，该第二投屏数据包括预设图像的数据或预设文本的数据。
可选的，该第二投屏数据包括预设图像的数据，该发送端向接收端发送第二投屏数据，包括：该发送端读取预设图像，对该预设图像解码得到该预设图像的数据；该发送端对该预设图像的数据进行编码，得到该第二投屏数据；该发送端向接收端发送该第二投屏数据。
可选的，该发送端检测该目标对象的状态之后，该方法还包括：若该目标对象未处于该预设状态，则该发送端继续向该接收端发送该第一投屏数据。
第二方面，本申请实施例提供了一种通信装置，该投屏装置与接收端连接，该接收端用于显示该投屏装置的投屏数据，包括：获取单元，用于获取用户触发的针对目标对象的窗口投屏操作；收发单元，用于向该接收端发送该目标对象的第一投屏数据，其中，该第一投屏数据包括该投屏装置的显示界面上显示的该目标对象的内容；处理单元，用于检测该目标对象的状态；该收发单元还用于在该目标对象处于预设状态的情况下向接收端发送第二投屏数据；其中，该第二投屏数据用于提示该目标对象处于该预设状态，该预设状态包括最小化状态或关闭状态。
可选的，该处理单元还用于周期性地采集该目标对象的投屏画面，其中，该第一投屏数据为该处理单元根据该投屏画面生成的；以及，该处理单元用于在采集该投屏画面的每个周期之前，检测该目标对象的状态。
可选的，该第二投屏数据包括预设图像的数据或预设文本的数据。
可选的，该第二投屏数据包括预设图像的数据，该处理单元还用于读取预设图像，对该预设图像解码得到该预设图像的数据；该处理单元还用于对该预设图像的数据进行编码，得到该第二投屏数据。
可选的，该发送端检测该目标对象的状态之后，该收发单元还用于在该目标对象未处于该预设状态的情况下继续向该接收端发送该第一投屏数据。
第三方面，本申请实施例提供一种计算机存储介质，该计算机存储介质存储有多条指令，该指令适于由处理器加载并执行上述的方法步骤。
第四方面，本申请实施例提供一种电子设备，可包括：处理器和存储器；其中，该存储器存储有计算机程序，该计算机程序适于由该处理器加载并执行上述的方法步骤。
在本申请实施例中，发送端在向接收端发送目标对象的窗口画面对应的投屏数据的过 程中，检测目标对象的状态，当目标对象的状态处于预设状态(可以包括最小化状态或关闭状态)时，发送端向接收端发送预设投屏数据，以提示用户无法获取投屏数据的原因。使得投屏显示端可以将预设投屏数据显示在显示界面，能够提示用户无法获取投屏数据的原因，以便用户能够根据提示恢复投屏。
附图说明
为了更清楚地说明本申请实施例或一些技术中的技术方案，下面将对实施例或一些技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本申请实施例提供的投屏控制方法的应用场景的一个示意图；
图2是本申请实施例提供的投屏控制方法的应用场景的另一个示意图；
图3是本申请实施例提供的窗口投屏的一个示意性流程图；
图4是本申请实施例提供的投屏方法的一个示意性流程图；
图5是本申请实施例提供的会议场景的一个示意图；
图6是本申请实施例提供的投屏方法的另一个示意性流程图；
图7是本申请实施例提供的投屏方法的示意图；
图8是本申请实施例提供的投屏装置的结构示意图；
图9是本申请实施例提供的电子设备的结构示意图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚，下面将结合附图对本申请实施例方式作详细描述。
应当明确，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其它实施例，都属于本申请保护的范围。
下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本申请相一致的所有实施方式。相反，它们仅是如所附权利要求书中所详述的、本申请的一些方面相一致的装置和方法的例子。
在本申请的描述中，需要理解的是，术语“第一”、“第二”、“第三”等用于区别类似的对象，而不必用于描述特定的顺序或先后次序，也不能理解为指示或暗示相对重要性。对 于本领域的普通技术人员而言，可以根据具体情况理解上述术语在本申请中的具体含义。此外，在本申请的描述中，除非另有说明，“多个”是指两个或两个以上。“和/或”，描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。字符“/”一般表示前后关联对象是一种“或”的关系。
图1是本申请实施例提供的投屏控制方法的应用场景100的一个示意图。
如图1所示，本申请实施例可以应用于会议场景100中，该会议场景100中包括至少一个投屏接收端(投屏接收端还可以称为投屏接收端设备或接收端)，如图1所示的投屏接收端101。以及该会议场景100中还可以包括至少两个投屏发送端(投屏发送端还可以称为投屏发送端设备或发送端)，投屏发送端可以是如图1所示的投屏发送端102、投屏发送端103。投屏接收端101可以与多个投屏发送端建立通信连接，如图1所示投屏接收端101可以与投屏发送端102、投屏发送端103建立通信连接，该通信连接可以是有线通信连接、也可以是无线通信连接，本申请对此不做限定。投屏接收端101可以接收来自投屏发送端102的投屏数据和来自投屏发送端103的投屏数据。投屏数据可以包括投屏画面信息和/或投屏声音信息。如图1所示，投屏接收端101配置有屏幕，则投屏接收端101可以在屏幕上同时显示来自投屏发送端102和来自投屏发送端103的画面、声音。但本申请不限于此。
图2是本申请实施例提供的投屏控制方法的应用场景200的一个示意图。
如图2所示，投屏接收端202可以是投屏处理装置，例如，该投屏接收端202可以是传屏盒子，但本申请不限于此。投屏接收端202可以接收来自投屏发送端203和投屏发送端204的投屏数据，并转发所接收到的投屏数据至连接的显示器201，控制显示器201显示来自投屏发送端203和投屏发送端204的画面，并播放声音，其中，投屏接收端202与投屏发送端203和204、显示器201之间的通信连接可以是无线通信连接，也可以是有线通信连接，本申请对此不做限定。
本申请实施例提供的投屏发送端可以是用户终端，该用户终端可以是手机、平板或笔记本电脑等终端设备。
本申请实施例提供的投屏接收端可以是电视、投影仪、智能交互平板、计算机设备或传屏盒子等具有传屏功能的通信设备，该投屏接收端可以通过有线通信的方式或无线通信的方式接收来自发送端的投屏数据。
图3为窗口投屏的一个示例，如图3所示，窗口投屏过程可以包括但不限于如下步骤：
1.响应于用户启动投屏软件的操作，发送端启动投屏发送端软件；
2.投屏发送端软件与投屏接收端软件建立通信连接；
3.用户在投屏发送端软件选择进行窗口投屏的目标对象；
4.发送端软件通过系统API采集目标对象的窗口画面；
5.投屏发送端软件将采集到的窗口画面进行视频编码得到投屏数据；
6.投屏发送端软件将投屏数据通过网络等方式发送至接收端，并再次从步骤3开始执行之后的步骤。
根据上述步骤4，发送端软件需要通过系统API采集目标对象的窗口画面，然而，受到大部分系统的接口限制，一些技术中，进行窗口投屏时，若正在投屏的窗口最小化时，投屏发送端软件将无法采集到该目标对象窗口画面内容。此时投屏显示设备可能将显示该窗口最小化前的最后一帧画面，出现画面静止不动的情况。对于不了解情况的用户来说将感到困惑，或者误认为是由于连接故障或网络不佳等原因造成的画面静止不动的情况，使得投屏无法正常进行。本申请实施例提出可以提示用户投屏异常的原因，以便用户能够根据提示恢复投屏。
下面结合附图对本申请实施例提供的投屏方法进行说明。
图4为本申请实施例提供的投屏方法的一个示意性流程图。如图4所示的方法中，发送端与接收端连接，该连接可以是有线通信连接、也可以是无线通信连接。
S401，响应于用户触发的针对目标对象的窗口投屏操作，发送端向接收端发送目标对象的第一投屏数据，该第一投屏数据包括在发送端的显示界面上显示的该目标对象的内容。
在用户需要进行窗口投屏时，用户可以在投屏发送端软件中选择需要进行窗口投屏的目标对象，执行窗口投屏操作以触发目标对象的窗口投屏。发送端通过投屏发送端软件获取到用户的该操作后，响应于用户的窗口投屏操作，发送端向接收端发送目标对象的第一投屏数据。
可选地，发送端获取到用户的窗口投屏操作，可以向接收端发送用于请求投屏的请求信息，接收端接收到来自发送端的该请求信息后，可以向发送端发送接收投屏的响应信息，发送端可以在接收到接收端的该响应信息后，向接收端发送目标对象的第一投屏数据。
可选地，该目标对象可以是应用程序的窗口。
例如，发送端的显示界面上多个应用程序的窗口处于打开状态，显示在该显示界面上，用户可以在投屏发送端软件中选择该多个应用程序中需要进行窗口投屏的目标应用程序执行窗口投屏操作。但本申请不限于此。
应理解，处于打开状态的该多个应用程序的窗口显示在显示界面上时，可以相互重叠或被其他窗口覆盖(或遮挡)，但窗口处于打开状态，即非最小化状态或关闭状态。
发送端运行投屏发送端软件时，发送端可以通过系统API采集目标对象在显示界面上显示的内容(即投屏画面)，例如，发送端的操作系统为Windows系统时，该API包括但不限于“BitBlt”、“PrintWindow”或“StretchBlt”。或者，发送端的操作系统为GUN/Linux系统时，该API包括但不限于“XGetImage”或“xcb_shm_get_image”。或者，发送端的操作系统为macOS系统时，该API包括但不限于“CGWindowListCreateImage”。
发送端采集目标对象的投屏画面，对投屏画面进行视频编码，得到第一投屏数据，并向接收端发送该第一投屏数据。接收端在接收到该第一投屏数据后进行视频解码等操作后在显示界面显示目标对象的投屏画面。
S402，发送端检测目标对象的状态，判断目标对象的状态是否处于预设状态。
发送端在发送第一投屏数据的过程中，检测目标对象的状态以判断目标对象是否处于预设状态，预设状态包括最小化状态或关闭状态。
一种实施方式中，发送端的操作系统为Windows系统，发送端判断目标对象的状态采用的函数包括但不限于IsWindowVisible函数和/或IsIconic函数。
例如，投屏发送端软件可以调用IsWindowVisible函数，判断目标对象的状态为显示或隐藏。若投屏发送端软件通过IsWindowVisble函数确定目标对象的状态为隐藏，则目标对象处于预设状态。可选的，投屏发送端软件还可以调用IsIconic函数，若IsIconic函数的返回值为“真(true)”，则投屏发送端软件可以确定目标对象处于最小化状态，若IsIconic函数的返回值为“假(false)”，则投屏发送端软件可以确定目标对象处于关闭状态。但本申请不限于此。
再例如，预设状态包括最小化状态，投屏发送端软件可以调用IsIconic函数，若IsIconic函数的返回值为“真(true)”，则投屏发送端软件可以确定目标对象处于预设状态，即最小化状态。若IsIconic函数的返回值为“假(false)”，则投屏发送端软件可以确定目标对象未处于最小化状态，但本申请不限于此。
另一种实施方式中，发送端的操作系统为macOS系统，发送端判断目标对象的状态采用的函数包括但不限于CGWindowListCopyWindowInfo函数和/或CGWindowListCreate函数。
例如，投屏发送端软件可以调用CGWindowListCreate函数，获取所有可见的窗口的列表，若该可见窗口列表中不包括目标对象，则投屏发送端软件可以确定目标对象不可见， 即目标对象处于预设状态。可选的，投屏发送端软件还可以再调用CGWindowListCopyWindowInfo获取所有处于开启状态的窗口列表，若该处于开启状态的窗口列表中不包括目标对象，则投屏发送端软件可以确定目标对象处于关闭状态，若该处于开启状态的窗口列表中包括目标对象，则投屏发送端软件可以确定目标对象处于最小化状态，但本申请不限于此。
再例如，预设状态包括关闭状态，投屏发送端软件可以调用CGWindowListCopyWindowInfo函数，根据处于开启状态的窗口列表，确定目标对象是否处于关闭状态，但本申请不限于此。
另一种实施方式中，发送端的操作系统为GUN/Linux系统，发送端判断目标对象的状态采用的函数包括但不限于XGetWindowAttributes函数和/或XGetWindowProperty函数。
例如，投屏发送端软件可以调用目标对象的XGetWindowAttributes函数，若成功调取XGetWindowAttributes函数则确定存在该目标对象，若调用XGetWindowAttributes函数失败则确定不存在目标对象，或者说确定目标对象处于预设状态。可选的，在投屏发送端软件确定存在该目标对象的情况下，投屏发送端软件还可以调用目标对象的XGetWindowProperty函数，获取_NET_WM_STATE属性，通过该属性确定目标对象是否隐藏，从而确定目标对象是否为最小化状态。但本申请不限于此。
再例如，预设状态包括最小化状态，投屏发送端软件可以调用目标对象的XGetWindowProperty函数，获取_NET_WM_STATE属性，通过该属性确定目标对象是否隐藏，从而确定目标对象是否为预设状态，即最小化状态。但本申请不限于此。
S403，若目标对象处于预设状态，向接收端发送第二投屏数据，该第二投屏数据用于提示目标对象处于预设状态，其中，预设状态包括最小化状态或关闭状态。
若发送端在S402检测到目标对象的状态为预设状态，发送端可以向接收端发送第二投屏数据，以便接收端接收到该第二投屏数据后在显示界面显示该第二投屏数据的解码内容，达到提示用户目标对象处于预设状态的目的。使得用户能够根据接收端显示的内容，获知目标对象的状态，及时进行处理。实现发送端主动告知用户投屏窗口的状态，能够提升用户体验。
可选的，该第二投屏数据包括预设图像的数据或预设文本的数据。
例如，预设状态为最小化状态，用户正在进行视频应用程序投屏展示过程中，在发送端上，可能受到其他用户的误操作或受到其他软件、操作系统的影响，该视频应用程序被最小化，发送端检测到该视频应用程序被最小化后，可以读取预设图像，比如，该预设图 像中包括“投屏窗口已最小化，无法采集到画面，请取消最小化”等文字。发送端读取到该预设图像后对预设图像解码得到预设图像的数据，再对预设图像的数据进行视频编码后，得到第二投屏数据，并发送给接收端，接收端解码该第二投屏数据后，可以将预设图像显示在接收端显示界面上，使得用户可以获知因投屏窗口最小化导致发送端无法采集到投屏画面。从而执行相应操作恢复投屏窗口的投屏状态。但本申请不限于此。
再例如，预设状态为关闭状态，用户正在进行对目标对象的投屏展示过程中，发送端检测到该目标对象未开启或已被关闭，即该目标对象的状态为关闭状态，发送端可以读取预设图像，比如，该预设图像中包括“投屏窗口已被关闭，无法采集到画面”等文字。发送端根据该预设图像生成第二投屏数据，并发送给接收端。接收端接收到第二投屏数据后，解码并在显示界面上该预设图像，使得用户可以获知因投屏窗口被关闭导致发送端无法采集到投屏画面。从而执行相应操作恢复投屏窗口的投屏状态。但本申请不限于此。
作为示例非限定，预设图像的格式包括但不限于PNG，JPG等。发送端对预设图像解码得到的预设图像的数据(可以称为预设图像的原始数据)的格式可能为RGBA，YUV等。
可选的，该发送端可以是周期性地采集该目标对象的投屏画面，该发送端在采集该投屏画面的每个周期之前，可以检测该目标对象的状态。
S404，若发送端在S402中检测到目标对象未处于预设状态，则发送端继续向接收端发送第一投屏数据
根据上述方案，发送端检测在投屏过程中检测投屏对象的状态，在处于预设状态时向接收端发送预设投屏数据，以便通过接收端显示提示信息，提示无法采集到投屏画面的原因，实现发送端主动告知用户投屏窗口的状态，能够提升用户体验。
图5为本申请实施例提供的会议场景的示意图。如图5所示，用户在发送端的投屏发送端软件上选择进行视频应用程序的窗口投屏，可以通过窗口投屏的方式与参会者分享视频。
在该场景中发送端软件可以在执行该视频应用程序的窗口投屏过程可以采用本申请实施例提供的投屏方法，如图6所示，该方法包括但不限于如下步骤：
S601，发送端软件检测该视频应用程序的状态。
S602，发送端软件判断该视频应用程序的状态是否为预设状态。
若该视频应用程序的状态不是预设状态，该发送端软件执行S603。后执行S606、S607。若该视频应用程序的状态为预设状态，该投屏发送端软件执行S604。
S603，发送端软件通过系统API采集该视频应用程序的窗口画面。
发送端软件执行S603之后执行S606、S607。
S604，发送端软件读取预设图片数据。
发送端可以存储预设图片的原始数据即未经过编码压缩的图片数据，或者，发送端存储预设图片的原始数据经过编码压缩后的数据。本申请对此不做限定。
一种实施方式中，发送端存储有预设图片的原始数据，即预设图片数据为该预设图片的原始数据，则发送端软件读取该预设图片数据后执行S606。
另一种实施方式中，发送端存储有预设图片的已编码压缩后的数据，即预设图片数据为该预设图片的原始数据经过编码压缩后的数据，则发送端软件读取该预设图片数据后执行S605以及之后的步骤。
可选的，S605，发送端软件解码预设图片数据得到图片的原始数据。
例如，预设图片的格式包括但不限于PNG，JPG等。图片解码后会得到图片的原始数据的格式可能为RGBA，YUV等。但本申请不限于此。
S606，发送端软件对窗口画面或图片的原始数据进行视频编码，得到投屏数据。
例如，预设状态为最小化状态，若发送端软件检测到该视频应用程序未处于最小化状态，发送端软件对窗口画面进行视频编码，得到包含窗口画面的投屏数据。若发送端软件检测到该视频应用程序处于最小化状态，发送端软件对图片的原始数据进行视频编码，得到包含图片原始数据的投屏数据。
再例如，预设状态为关闭状态，若发送端软件检测到该视频应用程序未处于关闭状态，发送端软件对窗口画面进行视频编码，得到包含窗口画面的投屏数据。若发送端软件检测到该视频应用程序处于关闭状态，发送端软件对图片的原始数据进行视频编码，得到包含图片原始数据的投屏数据。
S607，发送端软件向接收端软件发送该投屏数据。
相应地，接收端软件接收来自发送端软件的该投屏数据。接收端软件对该投屏数据进行视频解码后在显示界面进行显示。若解码得到窗口画面则接收端软件显示窗口画面，若解码得到图片原始数据，接收端软件显示预设图片。
例如，预设状态为最小化状态，该预设图片包括提示信息，如“投屏窗口已最小化，无法采集到画面，请取消最小化”，接收端可以在显示界面显示该预设图片的内容。如图7所示，参会者可以通过接收端显示的提示信息，获知发送端因投屏窗口最小化而无法采集到画面。发送端软件通过主动告知用户无法采集到画面的原因，使得用户能够及时恢复 窗口的投屏状态，提升了用户体验。
再例如，预设状态为关闭状态，该预设图片包括提示信息，如“投屏窗口已关闭，无法采集到画面”，接收端可以在显示界面显示该预设图片的内容。参会者可以通过接收端显示的提示信息，获知发送端因投屏窗口被关闭而无法采集到画面。发送端软件通过主动告知用户无法采集到画面的原因，使得用户能够及时恢复窗口的投屏状态，提升了用户体验。
需要说明的是，图6中以发送端软件执行本申请实施例提供的投屏方法为例进行说明，应理解，发送端软件执行操作可以理解为发送端在运行该发送端软件时由发送端实现的操作。
以上，结合图4至图7详细说明了本申请实施例提供的方法。以下说明本申请实施例提供的投屏装置。
请参见图8，其示出了本申请一个示例性实施例提供的投屏装置的结构示意图。该投屏装置可以通过软件、硬件或者两者的结合实现成为智能交互平板的全部或一部分。该装置包括获取单元801、处理单元802以及收发单元803。
获取单元，用于获取用户触发的针对目标对象的窗口投屏操作。
收发单元，用于向接收端发送该目标对象的第一投屏数据，其中，该第一投屏数据包括该发送端的显示界面上显示的该目标对象的内容。
处理单元，用于检测该目标对象的状态。
该收发单元还用于在该目标对象处于预设状态的情况下向接收端发送第二投屏数据；其中，该第二投屏数据用于提示该目标对象处于该预设状态，该预设状态包括最小化状态或关闭状态。
可选的，该投屏装置还包括存储单元804，该存储单元用于存储投屏应用程序。
可选的，该处理单元还用于周期性地采集该目标对象的投屏画面，其中，该第一投屏数据为该处理单元根据该投屏画面生成的；以及，该处理单元用于在采集该投屏画面的每个周期之前，检测该目标对象的状态。
可选的，该第二投屏数据包括预设图像的数据或预设文本的数据。
可选的，该第二投屏数据包括预设图像的数据，该处理单元还用于读取预设图像，对该预设图像解码得到该预设图像的数据；该处理单元还用于对该预设图像的数据进行编码，得到该第二投屏数据。
可选的，该发送端检测该目标对象的状态之后，该收发单元还用于在该目标对象未处 于该预设状态的情况下继续向该接收端发送该第一投屏数据。
请参见图9，为本申请实施例提供了一种电子设备的结构示意图。如图9所示，所述电子设备900可以包括：至少一个处理器901，至少一个网络接口904，用户接口903，存储器905以及至少一个通信总线902。
其中，通信总线902用于实现这些组件之间的连接通信。
其中，用户接口903可以包括显示屏(Display)、摄像头(Camera)，可选用户接口903还可以包括标准的有线接口、无线接口。
其中，网络接口904可选的可以包括标准的有线接口、无线接口(如WI-FI接口)。
其中，处理器901可以包括一个或者多个处理核心。处理器901利用各种接口和线路连接整个电子设备900内的各个部分，通过运行或执行存储在存储器905内的指令、程序、代码集或指令集，以及调用存储在存储器905内的数据，执行电子设备900的各种功能和处理数据。可选的，处理器901可以采用数字信号处理(Digital Signal Processing，DSP)、现场可编程门阵列(Field-Programmable Gate Array，FPGA)、可编程逻辑阵列(Programmable Logic Array，PLA)中的至少一种硬件形式来实现。处理器901可集成中央处理器(Central Processing Unit，CPU)、图像处理器(Graphics Processing Unit，GPU)和调制解调器等中的一种或几种的组合。其中，CPU主要处理操作系统、用户界面和应用程序等；GPU用于负责显示屏所需要显示的内容的渲染和绘制；调制解调器用于处理无线通信。可以理解的是，上述调制解调器也可以不集成到处理器901中，单独通过一块芯片进行实现。
其中，存储器905可以包括随机存储器(Random Access Memory，RAM)，也可以包括只读存储器(Read-Only Memory)。可选的，该存储器905包括非瞬时性计算机可读介质(non-transitory computer-readable storage medium)。存储器905可用于存储指令、程序、代码、代码集或指令集。存储器905可包括存储程序区和存储数据区，其中，存储程序区可存储用于实现操作系统的指令、用于至少一个功能的指令(比如触控功能、声音播放功能、图像播放功能等)、用于实现上述各个方法实施例的指令等；存储数据区可存储上面各个方法实施例中涉及到的数据等。存储器905可选的还可以是至少一个位于远离前述处理器901的存储装置。如图9所示，作为一种计算机存储介质的存储器905中可以包括操作系统、网络通信模块、用户接口模块以及投屏应用程序。
在图9所示的电子设备900中，用户接口903主要用于为用户提供输入的接口，获取用户输入的数据；而处理器901可以用于调用存储器905中存储的投屏应用程序，并执行 以下操作：
获取用户触发的针对目标对象的窗口投屏操作；向接收端发送该目标对象的第一投屏数据，其中，该第一投屏数据包括在显示界面上显示的该目标对象的内容；检测该目标对象的状态；若该目标对象处于预设状态，则向接收端发送第二投屏数据；其中，该第二投屏数据用于提示该目标对象处于该预设状态，该预设状态包括最小化状态或关闭状态。
可选的，周期性地采集该目标对象的投屏画面，其中，该第一投屏数据是根据该投屏画面生成的；以及，检测该目标对象的状态，包括：在采集该投屏画面的每个周期之前，检测该目标对象的状态。
可选的，该第二投屏数据包括预设图像的数据或预设文本的数据。
可选的，该第二投屏数据包括预设图像的数据，向接收端发送第二投屏数据，包括：读取预设图像，对该预设图像解码得到该预设图像的数据；对该预设图像的数据进行编码，得到该第二投屏数据；向接收端发送该第二投屏数据。
可选的，检测该目标对象的状态之后，该方法还包括：若该目标对象未处于该预设状态，则继续向该接收端发送该第一投屏数据。
本申请实施例还提供了一种计算机存储介质，所述计算机存储介质可以存储有多条指令，所述指令适于由处理器加载并执行如上述图4至图7所示实施例的方法步骤，执行过程可以参见图4至图7所示实施例的说明，在此不进行赘述。
存储介质所在设备可以是通信设备。
通信设备的硬件部分可以由触控显示模组、智能处理系统(包括控制器)等部分所构成，由整体结构件结合到一起，同时也由专用的软件系统作为支撑，其中触控显示模组包括显示屏、触控组件和背光灯组件，背光灯组件用于为显示屏提供背光光源，显示屏一般采用液晶显示装置，用于进行画面展示，触控组件设置在显示屏上或者设置在显示屏前端，用于采集用户的触控操作数据，并将采集的触控操作数据发送到智能处理系统进行处理。
在实际使用中，通信设备的显示屏上显示画面数据，当用户通过手指或者触控笔等触控物体点击显示屏上显示的内容，例如点击显示屏上显示的图形按钮时，触控显示模组的触控组件将采集到触控数据，从而触控组件将该触控数据转换为触控点的坐标数据后发送到智能处理系统，或者发送到智能处理系统处由智能处理系统转换为触控点的坐标数据，智能处理系统获得触控点的坐标数据后，根据预先设定的程序实现相应的控制操作，驱动显示屏显示内容发生变化，实现多样化的显示、操作效果。
按照技术原理，触摸组件可以分为五个基本种类；矢量压力传感技术触摸组件、电阻 技术触摸组件、电容技术触摸组件、电磁技术触摸屏、红外线技术触摸组件、表面声波技术触摸组件。按照触摸组件的工作原理和传输信息的介质，可以把触摸组件分为四个种类：电阻式、电容感应式、电磁感应式、红外线式以及表面声波式。
当用户用手指或笔触摸显示屏时，触摸组件可以采集触控点的数据并发送到智能处理系统，然后随着智能处理系统内置的软件来实现不同的功能应用，从而实现对智能处理系统的触控控制。
本领域内的技术人员应明白，本申请的实施例可提供为方法、系统、或计算机程序产品。因此，本申请可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本申请是参照根据本申请实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
在一个配置中，计算设备包括一个或多个处理器(CPU)、输入/输出接口、网络接口和内存。
存储器可能包括计算机可读介质中的非永久性存储器，随机存取存储器(RAM)和/或非易失性内存等形式，如只读存储器(ROM)或闪存(flash RAM)。存储器是计算机可读介质的 示例。
计算机可读介质包括永久性和非永久性、可移动和非可移动媒体可以由任何方法或技术来实现信息存储。信息可以是计算机可读指令、数据结构、程序的模块或其他数据。计算机的存储介质的例子包括，但不限于相变内存(PRAM)、静态随机存取存储器(SRAM)、动态随机存取存储器(DRAM)、其他类型的随机存取存储器(RAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、快闪记忆体或其他内存技术、只读光盘只读存储器(CD-ROM)、数字多功能光盘(DVD)或其他光学存储、磁盒式磁带，磁带磁磁盘存储或其他磁性存储设备或任何其他非传输介质，可用于存储可以被计算设备访问的信息。按照本文中的界定，计算机可读介质不包括暂存电脑可读媒体(transitory media)，如调制的数据信号和载波。
还需要说明的是，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、商品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、商品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括要素的过程、方法、商品或者设备中还存在另外的相同要素。
以上仅为本申请的实施例而已，并不用于限制本申请。对于本领域技术人员来说，本申请可以有各种更改和变化。凡在本申请的精神和原理之内所作的任何修改、等同替换、改进等，均应包含在本申请的权利要求范围之内。

Claims (13)

  1. 一种投屏方法，其中，所述方法应用于发送端，所述发送端与接收端连接，所述接收端用于显示所述发送端的投屏数据，所述方法包括：
    所述发送端获取用户触发的针对目标对象的窗口投屏操作；
    所述发送端向所述接收端发送所述目标对象的第一投屏数据，其中，所述第一投屏数据包括所述发送端的显示界面上显示的所述目标对象的内容；
    所述发送端检测所述目标对象的状态；
    若所述目标对象处于预设状态，则所述发送端向所述接收端发送第二投屏数据；其中，所述第二投屏数据用于提示所述目标对象处于所述预设状态，所述预设状态包括最小化状态或关闭状态。
  2. 根据权利要求1所述的方法，其中，所述方法还包括：
    所述发送端周期性地采集所述目标对象的投屏画面，其中，所述第一投屏数据为所述发送端根据所述投屏画面生成的；以及，
    所述发送端检测所述目标对象的状态，包括：
    所述发送端在采集所述投屏画面的每个周期之前，检测所述目标对象的状态。
  3. 根据权利要求1或2所述的方法，其中，所述第二投屏数据包括预设图像的数据或预设文本的数据。
  4. 根据权利要求3所述的方法，其中，所述第二投屏数据包括预设图像的数据，所述发送端向接收端发送第二投屏数据，包括：
    所述发送端读取预设图像，对所述预设图像解码得到所述预设图像的数据；
    所述发送端对所述预设图像的数据进行编码，得到所述第二投屏数据；
    所述发送端向接收端发送所述第二投屏数据。
  5. 根据权利要求1或2所述的方法，其中，所述发送端检测所述目标对象的状态之后，所述方法还包括：
    若所述目标对象未处于所述预设状态，则所述发送端继续向所述接收端发送所述第一投屏数据。
  6. 一种投屏装置，其中，所述投屏装置与接收端连接，所述接收端用于显示所述投屏装置的投屏数据，包括：
    获取单元，用于获取用户触发的针对目标对象的窗口投屏操作；
    收发单元，用于向所述接收端发送所述目标对象的第一投屏数据，其中，所述第一投 屏数据包括所述投屏装置的显示界面上显示的所述目标对象的内容；
    处理单元，用于检测所述目标对象的状态；
    所述收发单元还用于在所述目标对象处于预设状态的情况下向接收端发送第二投屏数据；其中，所述第二投屏数据用于提示所述目标对象处于所述预设状态，所述预设状态包括最小化状态或关闭状态。
  7. 根据权利要求6所述的装置，其中，
    所述处理单元还用于周期性地采集所述目标对象的投屏画面，其中，所述第一投屏数据为所述处理单元根据所述投屏画面生成的；以及，
    所述处理单元用于在采集所述投屏画面的每个周期之前，检测所述目标对象的状态。
  8. 根据权利要求6或7所述的装置，其中，所述第二投屏数据包括预设图像的数据或预设文本的数据。
  9. 根据权利要求8所述的装置，其中，所述第二投屏数据包括预设图像的数据，
    所述处理单元还用于读取预设图像，对所述预设图像解码得到所述预设图像的数据；
    所述处理单元还用于对所述预设图像的数据进行编码，得到所述第二投屏数据。
  10. 根据权利要求6或7所述的装置，其中，所述发送端检测所述目标对象的状态之后，
    所述收发单元还用于在所述目标对象未处于所述预设状态的情况下继续向所述接收端发送所述第一投屏数据。
  11. 一种计算机存储介质，其中，所述计算机存储介质存储有多条指令，所述指令适于由处理器加载并执行如权利要求1至5中任意一项的所述的投屏方法。
  12. 一种计算机程序产品，包括计算机程序，其中，所述计算机程序被处理器执行时实现如权利要求1至5中任一项所述的投屏方法。
  13. 一种电子设备，其中，包括：处理器和存储器；其中，所述存储器存储有计算机程序，所述计算机程序适于由所述处理器加载并执行如权利要求1至5中任一项所述的投屏方法。
PCT/CN2022/093260 2021-05-31 2022-05-17 投屏方法、存储介质及通信设备 WO2022252977A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110604190.8 2021-05-31
CN202110604190.8A CN115480720A (zh) 2021-05-31 2021-05-31 投屏方法、存储介质及通信设备

Publications (1)

Publication Number Publication Date
WO2022252977A1 true WO2022252977A1 (zh) 2022-12-08

Family

ID=84322774

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/093260 WO2022252977A1 (zh) 2021-05-31 2022-05-17 投屏方法、存储介质及通信设备

Country Status (2)

Country Link
CN (1) CN115480720A (zh)
WO (1) WO2022252977A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116828242A (zh) * 2023-08-30 2023-09-29 亿咖通(湖北)技术有限公司 长链路投屏方法、系统及存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106325666A (zh) * 2015-07-08 2017-01-11 腾讯科技（深圳）有限公司 一种屏幕分享方法、装置及服务器
US20170192736A1 (en) * 2013-08-29 2017-07-06 Samsung Electronics Co., Ltd. Method for sharing screen and electronic device thereof
CN111190558A (zh) * 2018-11-15 2020-05-22 腾讯科技（深圳）有限公司 投屏控制方法、装置、计算机可读存储介质和计算机设备
CN112612436A (zh) * 2020-12-24 2021-04-06 广州视源电子科技股份有限公司 一种投屏切换方法、装置、设备及存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170192736A1 (en) * 2013-08-29 2017-07-06 Samsung Electronics Co., Ltd. Method for sharing screen and electronic device thereof
CN106325666A (zh) * 2015-07-08 2017-01-11 腾讯科技（深圳）有限公司 一种屏幕分享方法、装置及服务器
CN111190558A (zh) * 2018-11-15 2020-05-22 腾讯科技（深圳）有限公司 投屏控制方法、装置、计算机可读存储介质和计算机设备
CN112612436A (zh) * 2020-12-24 2021-04-06 广州视源电子科技股份有限公司 一种投屏切换方法、装置、设备及存储介质

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116828242A (zh) * 2023-08-30 2023-09-29 亿咖通(湖北)技术有限公司 长链路投屏方法、系统及存储介质
CN116828242B (zh) * 2023-08-30 2023-12-05 亿咖通(湖北)技术有限公司 长链路投屏方法、系统及存储介质

Also Published As

Publication number Publication date
CN115480720A (zh) 2022-12-16

Similar Documents

Publication Publication Date Title
KR102425818B1 (ko) 스크린 미러링 서비스 제공장치 및 방법
KR102445699B1 (ko) 전자 장치 및 그의 동작 방법
US11720179B1 (en) System and method for redirecting content based on gestures
US11106420B2 (en) Method, device, system and storage medium for information transmission and data processing
US10135898B2 (en) Method, terminal, and system for reproducing content
WO2019001347A1 (zh) 移动设备的投屏方法、存储介质、终端及投屏系统
CN111629239B (zh) 投屏处理方法、装置、设备及计算机可读存储介质
US20120001856A1 (en) Responding to tactile inputs
KR20160135132A (ko) 네트워크 접속 채널 상태의 검출방법, 장치, 전자기기, 프로그램 및 기록매체
US11656831B2 (en) Implementing wireless displays that handle rotation and/or portrait-first sinks
RU2619089C2 (ru) Способ и устройство для воспроизведения множества видео
US20170083280A1 (en) Display apparatus and method for controlling display apparatus thereof
KR20170094745A (ko) 영상 인코딩 방법 및 이를 지원하는 전자 장치
WO2022252977A1 (zh) 投屏方法、存储介质及通信设备
US11928383B2 (en) Screen projection control method, storage medium and communication apparatus
CN112860212A (zh) 一种音量调节方法及显示设备
US8860773B2 (en) Telepresence for remote collaboration with a gestural interface
WO2022242628A1 (zh) 投屏方法、装置、设备及存储介质
CN111726605A (zh) 一种解像力确定方法、装置、终端设备和存储介质
CN112835506B (zh) 一种显示设备及其控制方法
CN105045510B (zh) 实现视频查看操作的方法及装置
KR102051828B1 (ko) 영상 통화 방법 및 영상 통화 중개 장치
WO2023115316A1 (zh) 投屏方法、装置、存储介质及电子设备
WO2019015089A1 (zh) 一种全局菜单的控制方法、装置、设备和存储介质
JP2011123127A (ja) 画像処理装置、画像表示装置及び画像伝送システム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22815024

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE