CN115278348B

CN115278348B - Screen projection method and device

Info

Publication number: CN115278348B
Application number: CN202210782324.XA
Authority: CN
Inventors: 王志军
Original assignee: Shenzhen Happycast Technology Co Ltd
Current assignee: Shenzhen Happycast Technology Co Ltd
Priority date: 2022-07-05
Filing date: 2022-07-05
Publication date: 2023-11-17
Anticipated expiration: 2042-07-05
Also published as: CN115278348A

Abstract

The embodiment of the application provides a screen projection method and device. The method comprises the following steps: receiving a first audio-video stream sent by a cloud server, wherein the first audio-video stream is determined by the cloud server according to a first file, and the first file is sent to the cloud server by first equipment in a cloud conference; displaying a first audio and video stream in a first area of a display screen; receiving a second audio and video stream sent by the cloud server, wherein the second audio and video stream is determined by the cloud server according to a second file, and the second file is sent to the cloud server by second equipment in the cloud conference; determining a second area in the display screen; and displaying the second audio and video stream in the second area. According to the method, through the interactive communication among the first equipment, the second equipment, the large-screen equipment and the cloud server in the cloud conference, the large-screen equipment can display a plurality of files at the same time, and the high efficiency of the conference is improved.

Description

Screen projection method and device

Technical Field

The present application relates to the field of communications technologies, and in particular, to a screen projection method and apparatus.

Background

With the rapid development of internet technology, online conferences using networks as media can break through the time and region limitations, and the effect of face-to-face communication is directly realized through the internet. When the conference room is used for online conference, a large screen device of the conference room is required to be used for conference file display and explanation, a host throws the conference file to the large screen device through user equipment, the content of the conference file is explained in the full screen state of the large screen device, and when the key function or detail of the content is explained, the host hopes to directly enable the participant to watch the demonstration process of the key function or detail.

At this time, the large screen device is displaying the content of the conference file of the device screen of the presenter in a full screen mode, and the conference file cannot be displayed on the large screen device at the same time in the demonstration process, so that the effect of online conference is affected.

Disclosure of Invention

The embodiment of the application provides a screen projection method and a screen projection device, which can enable large-screen equipment to simultaneously display a plurality of files through interactive communication among first equipment, second equipment, large-screen equipment and a cloud server in a cloud conference, thereby improving the efficiency of the conference.

In a first aspect, an embodiment of the present application provides a screen projection method, including:

receiving a first audio/video stream sent by a cloud server, wherein the first audio/video stream is determined by the cloud server according to a first file, and the first file is sent to the cloud server by first equipment in a cloud conference;

displaying the first audio and video stream in a first area of a display screen;

receiving a second audio and video stream sent by the cloud server, wherein the second audio and video stream is determined by the cloud server according to a second file, and the second file is sent to the cloud server by a second device in the cloud conference;

determining a second area in the display screen;

And displaying the second audio and video stream in the second area.

In the method, the large-screen device receives a first audio/video stream sent by the cloud server and displays the first audio/video stream in a first area of the display screen, and at this time, the large-screen device displays the first audio/video stream in a full-screen state, wherein the first audio/video stream is formed by analyzing a first file sent by the first device by the cloud server; when the large-screen device receives a second audio/video stream sent by the cloud server, an auxiliary screen throwing area is reserved for a first area in a full-screen state of the current display screen immediately, namely, a second area in the display screen is determined, and a second audio/video stream is displayed in the second area, wherein the second audio/video stream is formed by analyzing a second file sent by the second device by the cloud server. According to the method, through the interactive communication among the first equipment, the second equipment, the large-screen equipment and the cloud server in the cloud conference, the large-screen equipment can adaptively adjust the first area and the second area of the display screen, so that audio and video streams corresponding to various files are displayed at the same time, and the high efficiency of the online conference is improved.

In one possible implementation, the first area includes a blank area;

The determining the second region in the display screen includes:

and determining the second area according to the size of the blank area and/or the display requirement of the first audio/video stream.

In the embodiment of the application, the first area comprises a blank area, wherein the blank area refers to an area except for a content display area of the first audio/video stream in the first area, that is, the blank area does not contain the content of the first audio/video stream.

In a possible implementation manner, the determining the second area according to the size of the blank area and/or the display requirement of the first av stream includes:

when the size of the blank area is larger than a preset area threshold value, placing the second area in the blank area of the first area;

and when the size of the blank area is smaller than or equal to the preset area threshold, determining the second area according to the display requirement of the first audio and video stream.

In the embodiment of the application, when the size of the blank area is larger than the preset area threshold, the area of the blank area is larger, the second area is directly placed in the blank area of the first area, the area of the display screen can be maximally utilized to display two audio and video streams, and when the size of the blank area is smaller than or equal to the preset area threshold, the area of the blank area is insufficient, and at the moment, the second area needs to be further determined according to the display requirement of the first audio and video stream. According to the method, the first area and the second area are determined by selecting a proper adjustment mode according to the characteristics of the first audio/video stream, so that the fine processing process is realized.

In one possible implementation, the display requirements include full display and partial display;

the determining the second area according to the display requirement of the first audio/video stream includes:

shrinking the first area;

when the display requirement is the whole display, the second area is placed in other areas of the display screen except the reduced first area, wherein an overlapping area does not exist between the reduced first area and the second area;

And when the display requirement is the partial display, placing the second area in the area of the display screen, wherein an overlapping area exists between the reduced first area and the second area.

In the embodiment of the application, when the size of the blank area is smaller than or equal to the preset area threshold value, the first area is firstly reduced, if the display requirement is that the first audio and video and the second audio and video are displayed completely, the second area is placed in other areas except for the reduced first area in the display screen, no overlapping area exists between the reduced first area and the reduced second area, and two audio and video streams can be displayed completely, so that the display requirement is met; if the display requirement is partial display, it is indicated that the first audio/video stream does not need to be completely displayed, so that the second region is placed in the region of the display screen, and the second region and the reduced first region have an overlapping region, but only the second audio/video stream corresponding to the second region needs to be completely displayed. The method can select the display mode according to the requirements of the user, so that the second area is determined, and the requirements of the user are better met while the audio and video streams corresponding to various files are displayed.

In one possible implementation, the method further includes:

receiving an adjustment instruction input by a user;

and according to the adjustment instruction, adjusting the first area and/or the second area to obtain an adjusted first area and/or an adjusted second area, wherein the adjusted first area is used for displaying the first audio/video stream, and the adjusted second area is used for displaying the second audio/video stream.

In the embodiment of the application, the first area and the second area are determined after the self-adaptive adjustment of the large screen equipment, and a user can input an adjustment instruction to adjust the first area and/or the second area in the conference process according to the requirements and the actual conditions of the conference (for example, repeated display and enlarged display are required), so that the audio and video streams corresponding to various files are displayed on the display screen to be more fit with the actual conditions.

In a second aspect, an embodiment of the present application provides a screen projection method, including:

receiving a first file sent by first equipment in a cloud conference;

recording and opening an interface of the first file to obtain a first audio and video stream;

the first audio and video stream is sent to large-screen equipment in the cloud conference and is used for being displayed in a first area of a display screen;

Receiving a second file sent by a second device in the cloud conference;

recording and opening an interface of the second file to obtain a second audio and video stream;

and sending the second audio-video stream to the large-screen equipment, wherein the second audio-video stream is used for displaying in a second area of the display screen.

In the method, a cloud server receives a first file sent by first equipment in a cloud conference, records an interface for opening the first file, obtains a first audio-video stream, sends the first audio-video stream to large-screen equipment in the cloud conference, and the large-screen equipment displays the first audio-video stream in a first area of a display screen; when the cloud server receives a second file sent by another device (second device) in the cloud conference, recording and opening an interface of the second file to obtain a second audio-video stream, sending the second audio-video stream to the large-screen device, and displaying the second audio-video stream in a second area of the display screen by the large-screen device. According to the method, through the interactive communication among the first equipment, the second equipment, the large-screen equipment and the cloud server in the cloud conference, the large-screen equipment can adaptively adjust the first area and the second area of the display screen, so that audio and video streams corresponding to various files are displayed at the same time, and the high efficiency of the online conference is improved.

In one possible implementation, the method further includes:

receiving a first request sent by the second equipment;

sending the first request to the first device, wherein the first request is used for requesting whether the first device allows the second device to throw a screen on the large-screen device;

receiving acknowledgement information sent by the first equipment;

and sending the confirmation information to the second device, wherein the confirmation information is used for indicating that the second device is allowed to throw a screen on the large-screen device.

In the embodiment of the application, when the second device performs screen projection on the large screen device, a screen projection request (first request) needs to be sent to the first device, and the second device and the first device cannot directly communicate with each other, so that the second device sends the first request to the cloud server, then the cloud server forwards the first request to the first device, after a user determines screen projection through the first device, the first device sends confirmation information to the cloud server, and then the cloud server forwards the confirmation information to the second device, and the second device is allowed to perform screen projection on the large screen device. According to the method, each device in the cloud conference can communicate with each other through the cloud server, and the first device can authorize the screen-throwing request of the second device in the mode, so that a plurality of screen-throwing files are displayed on the large-screen device at the same time.

In one possible implementation, the method further includes:

receiving screen throwing information sent by the large screen equipment, wherein the screen throwing information comprises identification information of the second equipment, and the identification information of the second equipment is selected from the identification information of at least one equipment by a user of the first equipment;

and sending indication information to the second equipment according to the screen throwing information, wherein the indication information is used for indicating to start screen throwing on the large screen equipment.

In the embodiment of the application, when a user uses the large-screen device to explain a file, the large-screen device directly triggers the screen throwing information, the large-screen device sends the screen throwing information to the cloud server, and the cloud server sends the indication information to the second device according to the screen throwing information to indicate the second device to start screen throwing on the large-screen device. The method is another mode for authorizing the second equipment to throw the screen on the large screen equipment, does not need the second equipment to actively send a screen throwing request, and is characterized in that a screen throwing instruction is directly triggered by a user on the large screen equipment end, so that the actual situation of an online meeting is more attached.

In a third aspect, an embodiment of the present application provides a screen projection device, including:

The receiving unit is used for receiving a first audio-video stream sent by a cloud server, wherein the first audio-video stream is determined by the cloud server according to a first file, and the first file is sent to the cloud server by first equipment in a cloud conference;

the display unit is used for displaying the first audio and video stream in a first area of the display screen;

the receiving unit is used for receiving a second audio-video stream sent by the cloud server, wherein the second audio-video stream is determined by the cloud server according to a second file, and the second file is sent to the cloud server by a second device in the cloud conference;

a processing unit, configured to determine a second area in the display screen;

and the display unit is used for displaying the second audio/video stream in the second area.

In one possible implementation, the first area includes a blank area; the processing unit is specifically configured to:

In a possible implementation manner, the processing unit is specifically configured to:

In one possible implementation, the display requirements include full display and partial display; the processing unit is specifically configured to:

shrinking the first area;

In a possible implementation manner, the receiving unit is further configured to:

receiving an adjustment instruction input by a user;

the processing unit is specifically configured to:

The operations and beneficial effects executed by the screen projection device may refer to the method and beneficial effects described in any one of the above first aspect, and the repetition is not repeated.

In a fourth aspect, an embodiment of the present application provides a screen projection device, which is applied to a cloud server, including:

the receiving unit is used for receiving a first file sent by first equipment in the cloud conference;

the processing unit is used for recording and opening the interface of the first file to obtain a first audio/video stream;

the sending unit is used for sending the first audio-video stream to the large-screen equipment in the cloud conference, and the first audio-video stream is used for displaying in a first area of a display screen;

the receiving unit is used for receiving a second file sent by a second device in the cloud conference;

the processing unit is used for recording and opening the interface of the second file to obtain a second audio and video stream;

the sending unit is configured to send the second audio and video stream to the large-screen device, where the second audio and video stream is used for displaying in a second area of the display screen.

receiving a first request sent by the second equipment;

The sending unit is further configured to:

the receiving unit is further configured to:

receiving acknowledgement information sent by the first equipment;

the sending unit is further configured to:

The operations and beneficial effects performed by the screen projection device may refer to the method and beneficial effects described in any one of the second aspect, and the repetition is not repeated.

In a fifth aspect, the present application provides a screen projection device comprising a processor and a memory for storing a computer program; the processor is configured to execute a computer program stored in the memory, to cause the screen projection device to perform the method according to any one of the first aspect or the second aspect.

In a sixth aspect, the present application provides a screen projection device, the screen projection device comprising a processor, a memory and a transceiver, the transceiver being configured to receive a channel or signal, or to transmit a channel or signal; the memory is used for storing a computer program; the processor is configured to invoke the computer program from the memory to perform the method according to any of the first aspect or any of the second aspect.

In a seventh aspect, the present application provides a screen projection device, the screen projection device comprising a processor and interface circuitry for receiving a computer program and transmitting to the processor; the processor runs the computer program to perform the method according to any one of the first aspect or any one of the second aspect.

In an eighth aspect, the present application provides a computer readable storage medium for storing a computer program which, when executed, causes the method according to any one of the first or second aspects to be carried out.

In a ninth aspect, the present application provides a computer program product comprising a computer program which, when executed, causes the method according to any one of the first or second aspects to be carried out.

In a tenth aspect, an embodiment of the present application provides a screen projection system, where the screen projection system includes at least one cloud server, at least one terminal device, where the terminal device is configured to perform the step of any one of the first aspect, and the cloud server is configured to perform the step of any one of the second aspect.

Drawings

The drawings used in the embodiments of the present application are described below.

FIG. 1 is a schematic diagram of a projection system architecture according to an embodiment of the present application;

FIG. 2 is a flowchart of a screen projection method according to an embodiment of the present application;

FIG. 3 is a schematic diagram of a main interface of an application according to an embodiment of the present application;

FIG. 4 is a schematic diagram of an interface for selecting a screen-projection device according to an embodiment of the present application;

FIG. 5 is a schematic view of a coverage area according to an embodiment of the present application;

FIG. 6 is a schematic diagram of a synthetic type region provided in an embodiment of the present application;

FIG. 7 is a schematic diagram of an overlap region according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a screen projection device 800 according to an embodiment of the present application;

fig. 9 is a schematic structural diagram of a screen projection device 900 according to an embodiment of the present application;

fig. 10 is a schematic structural diagram of a screen projection device 1000 according to an embodiment of the present application;

fig. 11 is a schematic structural diagram of a screen projection device 1100 according to an embodiment of the present application.

Detailed Description

The following is a description of some of the terms involved in the present application to facilitate understanding by those skilled in the art.

1. The cloud conference is a conference group created on a cloud server by a terminal device through an application program supporting online conference, and is used for joining participants of the online conference and carrying out the online conference in the cloud conference. The main interface of the cloud conference comprises four display areas, and each display area is the content of a cloud desktop, wherein the content of the cloud desktop refers to file content shared by terminal equipment in the cloud conference, such as documents, tables, videos, audios and screen shots of the terminal equipment. It should be noted that, the four display areas of the main interface of the cloud conference may be slid to the right, for example, the current four display areas of the main interface are the first display area, the second display area, the third display area, and the fourth display area, and at this time, the current four display areas of the main interface are slid to the right, and the fifth display area, the sixth display area, the seventh display area, and the eighth display area are displayed in the main interface, so when the number of files shared in the cloud conference exceeds four, the shared files may be viewed in a right-sliding manner. The cloud conference can realize the transmission of audio and video data, the sharing and management of documents, remote phones or video conferences and the like.

2. The audio and video stream is a file in the received file information, after the analysis is successful, the file which is successfully analyzed is opened, the opened state of the file, the sound information and the like are recorded to form a corresponding audio and video stream, and the audio and video stream is pushed to all terminal devices in the cloud conference for display.

Embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application.

When the conference room is used for online conference, a large screen device of the conference room is required to be used for conference file display and explanation, a host throws the conference file to the large screen device through user equipment, the content of the conference file is explained in the full screen state of the large screen device, and when the key function or detail of the content is explained, the host hopes to directly enable the participant to watch the demonstration process of the key function or detail. At the moment, the large screen device is displaying the content of the conference file of the device screen of the presenter in a full screen mode, the conference file cannot be displayed on the large screen device at the same time in the demonstration process, and the effect of online conference is affected. Referring to fig. 1, fig. 1 is a schematic diagram of a screen projection system architecture provided in an embodiment of the present application, where the system includes a terminal device 101, a cloud server 102, and a large screen device 103, where:

The terminal device 101 is a device having processing capability and data transceiving capability. The terminal device 101 installs and runs an application program supporting online conferences, in which a user account is logged in, and the user can join the cloud conference by clicking a conference link, inputting a verification code, a conference number, and the like using the terminal device 101. The terminal device 101 may include a first device and a second device, where the first device is typically a terminal device of a presenter, and the presenter refers to a participant who obtains control rights of a conference desktop of a cloud conference through the first device, and in the cloud conference, only a single user (a user corresponding to any one terminal device in the cloud conference) controls the conference desktop at the same time. The second device is typically a terminal device of a participant other than the presenter, i.e. other devices than the first device in the cloud conference. The terminal device 101 may be a cloud television, a computer, a notebook, a tablet, a palm top, a desktop, a diagnostic, a cell phone, an Ultra-mobile Personal Computer Ultra Mobile Personal Computer (UMPC), a netbook, a personal digital assistant (Personal Digital Assistant, PDA), or the like.

Alternatively, the terminal device 101 may refer broadly to one of a plurality of terminal devices, and embodiments of the present application are illustrated with respect to terminal device 101 only. The applications installed on the terminal devices may be the same or the applications installed on the two terminal devices may be the same type of application of different operating system platforms, and the number of the terminal devices may be greater or lesser. For example, the number of the terminal devices is tens or hundreds, or more, and the number and the device type of the terminal devices are not limited in the embodiment of the present application.

Cloud server 102 is a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDNs), big data, and artificial intelligence platforms. The cloud server 102 creates and manages the cloud conference, configures the authority of all terminal devices in the cloud conference, can control the switching of a plurality of display areas in the cloud conference, analyzes the received file, opens the successfully analyzed file after the analysis is successful, records the opened state of the file, sound information and the like to form corresponding audio and video streams, and pushes the audio and video streams to all terminal devices with viewing authority in the cloud conference. Cloud server 102 provides computing or application services for terminal device 101 and background services for applications supporting online conferences.

The terminal device 101 and the cloud server 102 may be directly or indirectly connected through wired or wireless communication.

Alternatively, the cloud server 102 may undertake primary computing work, and the terminal device 101 may undertake secondary computing work; alternatively, the cloud server 102 takes on secondary computing work and the terminal device 101 takes on primary computing work; alternatively, the cloud server 102 and the terminal apparatus 101 each separately undertake classification work.

The large screen device 103 refers to a large screen end device in a conference room, and it should be noted that the large screen device 103 may be a first device or not, when the large screen device 103 is the first device, the large screen device 103 is a device with processing capability and data receiving and transmitting capability, a presenter can join a cloud conference through the large screen device 103, upload a file to a cloud server in the cloud conference, and then the cloud server pushes an audio and video stream formed by the file which is successfully parsed to the large screen device 103 for display. When the large-screen device 103 is not the first device, the large-screen device 103 may be a device other than the first device and the second device in the cloud conference, and at this time, the large-screen device 103 is mainly used for displaying various files that the first device and the second device throw a screen to the large-screen device 103. The large screen device 103 includes a display screen, which may be a display screen of the large screen device or a display screen of an external large screen device. The display screen may include a liquid crystal display (Liquid Crystal Display, LCD), a Cathode Ray Tube (CRT) display, an LED display (LED panel), a 3D display, or the like.

The large screen device 103 and the cloud server 102 may be directly or indirectly connected through wired or wireless communication.

Referring to fig. 2, fig. 2 is a flowchart of a screen projection method according to an embodiment of the present application, where the method includes, but is not limited to, the following steps:

s201: the first device sends a first file to the cloud server.

Specifically, the first device is any one terminal device in the cloud conference, and typically the first device is a terminal device of a presenter, and the presenter refers to a participant who obtains control rights of a conference desktop of the cloud conference through the first device, wherein the cloud conference can be created on a cloud server through an application program supporting online conference by the first device, or can be created on the cloud server through an application program supporting online conference by other devices (devices except the first device). The first file may be a document file, a table file, a Photoshop proprietary format (PSD) file, an animation scene creation format (3 Dmax) file, a computer aided design format (DWG) file, a video file, an audio file, a projection screen of the first device, and the like.

The first device starts an application program supporting online conferences, logs in a user account into a main interface of the application program, as shown in fig. 3, fig. 3 is a schematic diagram of the main interface of the application program, where the main interface of the application program includes a user account name and an avatar, scans a sweep, starts discussion, joins the discussion, and mirrors a screen. The first device can scan the two-dimensional code of the cloud discussion group to enter the cloud conference, a new cloud conference can be created by starting the discussion, the joining discussion can be added to the created cloud conference, and the screen image can be used for the first device to throw the screen to the large-screen device.

The first device receives a click operation (click swipe, start discussion or join discussion) input by a user, and responds to the click operation, the first device displays a projection screen content interface, wherein the projection screen content interface comprises a projection file and a projection screen, the projection file is used for the first device to select a file on a local device, and the file can comprise a document file, a table file, a PSD file, a 3Dmax file, a DWG file, a video file, an audio file and the like; the drop screen is used by the first device to select a home screen of the local device. The first device receives a selection operation (selecting a posting or a posting screen) input by a user, selects a first file on the first device in response to the selection operation, and sends the first file to the cloud server.

Optionally, the cloud server and the first device may be directly or indirectly connected through a wired or wireless communication manner.

S202: and the cloud server records an interface for opening the first file to obtain a first audio and video stream.

Specifically, after the cloud server receives the first file, the cloud server analyzes the first file, and when the first file is successfully analyzed, the cloud server opens the first file after the analysis is successfully performed, records a current file interface of the first file, and forms a first audio/video stream.

Optionally, when the first file analysis fails, the cloud server sends prompt information to the first device, where the prompt information is used to instruct a presenter to record the application interface of the first file through the split screen mirroring function, so as to form a first audio/video stream.

S203: and the cloud server sends the first audio and video stream to the large-screen device.

Specifically, the large screen device refers to a large screen end device in a conference room. And after the cloud server forms the first audio and video stream, pushing (sending) the first audio and video stream to the large-screen equipment in the cloud conference.

Optionally, the cloud server and the large-screen device may be directly or indirectly connected through a wired or wireless communication manner, and the cloud server may send the first audio/video stream to the large-screen device through a communication manner such as broadcasting, unicasting or multicasting.

It should be noted that the cloud server may also send the first audio/video stream to all devices (the first device, the second device, the large screen device, and other devices) in the cloud conference.

S204: the large screen device displays the first audio-video stream in a first area of the display screen.

Specifically, the display screen may be a display screen of a large-screen device, or may be a display screen externally connected to a large-screen device. The display screen may include a liquid crystal display (Liquid Crystal Display, LCD), a Cathode Ray Tube (CRT) display, an LED display (LED panel), a 3D display, or the like. The first area is an area on the display screen displaying the first audio-video stream in a full-screen state, i.e. a full-screen area.

And after the large-screen device receives the first audio-video stream, displaying the first audio-video stream in a full-screen state (first area) of the display screen.

S205: and the second device sends the second file to the cloud server.

In particular, the second device is typically a terminal device of a participant other than the presenter, i.e. other devices than the first device in the cloud conference. The second file may be a document file, a table file, a PSD file, a 3Dmax file, a DWG file, a video file, an audio file, a screen shot of the second device, or the like.

In some embodiments, prior to step S205, further comprising:

the cloud server receives a first request sent by second equipment and sends the first request to the first equipment; and the cloud server receives the confirmation information sent by the first equipment and sends the confirmation information to the second equipment.

Specifically, the second device receives a first request input by a user, sends the first request to the cloud server, and forwards the first request to the first device after the cloud server receives the first request, wherein the first request is used for requesting whether the first device allows the second device to throw a screen on the large-screen device. After the first device receives the first request, the first request is displayed in response to the first request, after the first device receives confirmation information input by a presenter, the confirmation information is sent to the cloud server in response to the confirmation information, and after the cloud server receives the confirmation information, the confirmation information is forwarded to the second device, wherein the confirmation information is used for indicating that the second device is allowed to throw a screen on the large-screen device.

Further, after receiving the confirmation information, the second device sends the second file to the cloud server, where a specific embodiment of sending the second file by the second device to the cloud server may refer to the manner in which the first device sends the first file to the cloud server in step S201, which is not described herein.

In this embodiment, the method enables each device in the cloud conference to communicate with each other through the cloud server, and the first device may also authorize the screen-casting request of the second device in this manner, so that the screen-casting files of the two devices are displayed on the large-screen device at the same time.

In some embodiments, prior to step S205, further comprising:

the cloud server receives the screen throwing information sent by the large-screen equipment, and sends prompt information to the second equipment according to the screen throwing information.

Specifically, the screen projection information comprises identification information of a second device, wherein the identification information of the second device is selected from the identification information of at least one device by a user of the first device; the indication information is used for indicating the second equipment to start to throw screen on the large-screen equipment.

The large screen device displays a first audio/video stream in a full screen state (a first area) of a display screen, when a screen throwing instruction input by a host is received, an interface for selecting the screen throwing device is displayed in response to the screen throwing instruction, as shown in fig. 4, fig. 4 is an interface schematic diagram of selecting the screen throwing device, the interface comprises a device list, the device list comprises identification information of at least one other device except the large screen device and the first device in a cloud conference, the host selects identification information of a second device needing to throw a screen from the identification information of the at least one other device, the large screen device receives the screen throwing information (the identification information of the second device) sent by the host, the screen throwing information is sent to a cloud server in response to the screen throwing information, after the cloud server receives the screen throwing information, the second device is searched from all devices in the cloud conference, prompt information is sent to the second device, and after the second device receives the prompt information, the second file is sent to the cloud server.

In the embodiment, the method is another mode of authorizing the second equipment to throw the screen on the large-screen equipment, the second equipment is not required to actively send a screen throwing request, and a screen throwing instruction is directly triggered by a presenter at the large-screen equipment end, so that the actual situation of an online meeting is more attached.

S206: and the cloud server records an interface for opening the second file to obtain a second audio and video stream.

Specifically, after receiving the second file, the cloud server analyzes the second file, and when the second file is successfully analyzed, the cloud server opens the second file after the analysis is successful, records a current file interface of the second file, and forms a second audio/video stream.

It should be noted that, when the second file includes a plurality of files, the cloud server records and opens an interface of each of the plurality of files to obtain a second audio/video stream, where the second audio/video stream includes a plurality of audio/video streams.

Optionally, when the second file analysis fails, the cloud server sends a prompt message to the second device, where the prompt message is used to instruct the presenter to record the application interface of the second file through the split screen mirroring function, so as to form a second audio/video stream.

S207: and the cloud server sends a second audio and video stream to the large-screen device.

Specifically, the specific implementation manner of S207 may refer to the manner in which the cloud server in step S203 sends the first audio/video stream to the large-screen device, which is not described herein.

S208: the large screen device determines a second region in the display screen.

In some embodiments, the first region includes a blank region, and the large screen device determines the second region according to a size of the blank region and/or a display requirement of the first audio-video stream.

Specifically, the first area is an area in which the first audio/video stream is displayed in a full-screen state on the display screen, that is, a full-screen area, where when the first audio/video stream is displayed in the first area, the content of the first audio/video stream cannot be completely distributed in the first area, and in the first area, an area in which the content of the first audio/video stream is not distributed is a blank area, the blank area is related to the content of the first audio/video stream, and different first audio/video streams also have different corresponding blank areas. The display requirements comprise all display and part display, wherein the display requirements refer to whether the content of the first audio-video stream can be blocked, and when the content of the first audio-video stream can be blocked, namely, all the content of the first audio-video stream does not need to be completely displayed on a display screen, the display requirements of the first audio-video stream are all display; when the content of the first audio-video stream cannot be blocked, namely, the whole content of the first audio-video stream needs to be completely displayed on the display screen, the display requirement of the first audio-video stream is that of partial display.

And after the large-screen equipment receives the second audio and video stream sent by the cloud server, determining a second area according to the size of the blank area and/or the display requirement of the first audio and video stream. When the size of the blank area is larger than the preset area threshold, the large screen device directly places the second area in the blank area of the first area, at this time, the first area and the second area are in a coverage type relationship, the original range of the first area is kept unchanged, and the second area is covered on the first area, as shown in fig. 5, and fig. 5 is a schematic diagram of a coverage type area provided by the embodiment of the present application.

When the size of the blank area is smaller than or equal to a preset area threshold value, the large-screen equipment determines a second area according to the display requirement of the first audio and video stream.

Specifically, the large-screen device firstly reduces the first area according to the ratio (the ratio of length to width) of the first area, when the display requirement of the first audio/video stream is that of the whole display, the second area is placed in other areas except for the reduced first area in the display screen, at this time, the reduced first area and the reduced second area are in a composite type relationship, and no overlapping area exists, as shown in fig. 6, fig. 6 is a schematic diagram of a composite type area provided in the embodiment of the present application; when the display requirement of the first audio/video stream is that of partial display, the second area is placed in the area of the display screen, at this time, the reduced first area and the reduced second area are in an overlapping relationship, the reduced first area may be smaller than or equal to the second area or may be larger than the second area, and an overlapping area exists between the reduced first area and the reduced second area, as shown in fig. 7, fig. 7 is a schematic diagram of an overlapping area provided in the embodiment of the present application.

It should be noted that, when the second audio-video stream includes a plurality of audio-video streams, for example, when the second audio-video stream includes two audio-video streams, the large-screen device determines the second area and the third area in the display screen, where the specific implementation method for determining the third area may refer to the specific implementation method for determining the second area, which is not limited in this disclosure.

In the embodiment, the large-screen device can select a proper adjustment mode to determine the first area and the second area according to the size of the blank area and/or the display requirement of the first audio/video stream, so that the processing process is refined, and the user requirement is better met while the audio/video streams corresponding to various files are displayed.

S209: the large screen device displays a second audio-video stream in a second area.

Specifically, after the large-screen device determines the second area and the first area in the display screen, the second audio-video stream is displayed in the second area, and the first audio-video stream is displayed in the first area.

It should be noted that, when the second audio/video stream includes a plurality of audio/video streams, for example, when the second audio/video stream includes two audio/video streams, the large screen device determines the second area and the third area in the display screen, and then displays one audio/video stream in the second area, another audio/video stream in the second audio/video stream in the third area, and displays the first audio/video stream in the first area, which is not limited in the present application.

In some embodiments, the large screen device receives an adjustment instruction input by a user; and adjusting the first area and/or the second area according to the adjustment instruction to obtain an adjusted first area and/or an adjusted second area.

Specifically, the adjustment instruction may include an adjustment area, adjustment information corresponding to the adjustment area, and area placement information, the adjustment area includes at least one of a first area and a second area, the adjustment information corresponding to the adjustment area may include an adjustment ratio, the adjusted area length and width, and the area placement information may include a placement position of the first area and a placement position of the second area. The large screen device displays a first audio and video stream in a first area, and after displaying a second audio and video stream in a second area, receives an adjustment instruction input by a presenter, and responds to the adjustment instruction to adjust the first area and/or the second area to obtain an adjusted first area and/or an adjusted second area.

In this embodiment, the first area and the second area are determined after being adaptively adjusted by the large screen device, so that a user can input an adjustment instruction to adjust the first area and/or the second area in the conference process according to requirements and actual conditions (for example, repeated display and enlarged display are required) of the conference, and the display screen displays audio and video streams corresponding to various files to be more fit with the actual conditions.

In the embodiment of the application, the large-screen equipment receives a first audio-video stream sent by the cloud server and displays the first audio-video stream in a first area of the display screen, and at the moment, the large-screen equipment displays the first audio-video stream in a full-screen state, wherein the first audio-video stream is formed by analyzing a first file sent by the first equipment by the cloud server; when the large-screen device receives a second audio/video stream sent by the cloud server, an auxiliary screen throwing area is reserved for a first area in a full-screen state of the current display screen immediately, namely, a second area in the display screen is determined, and a second audio/video stream is displayed in the second area, wherein the second audio/video stream is formed by analyzing a second file sent by the second device by the cloud server. According to the method, through the interactive communication among the first equipment, the second equipment, the large-screen equipment and the cloud server in the cloud conference, the large-screen equipment can adaptively adjust the first area and the second area of the display screen, so that audio and video streams corresponding to a plurality of files are displayed at the same time, and the high efficiency of the online conference is improved.

The foregoing details of the method according to the embodiments of the present application and the apparatus according to the embodiments of the present application are provided below.

Referring to fig. 8, fig. 8 is a schematic structural diagram of a screen projection device 800 according to an embodiment of the present application, where the device includes a receiving unit 801, a display unit 802, and a processing unit 803, and the detailed descriptions of the respective units are as follows.

A receiving unit 801, configured to receive a first audio/video stream sent by a cloud server, where the first audio/video stream is determined by the cloud server according to a first file, and the first file is sent to the cloud server by a first device in a cloud conference;

a display unit 802, configured to display the first audio/video stream in a first area of a display screen;

a processing unit 803 for determining a second region in the display screen;

the display unit 802 is configured to display the second audio/video stream in the second area.

In one possible implementation, the first area includes a blank area; the processing unit 803 specifically is configured to:

In one possible implementation, the processing unit 803 is specifically configured to:

In one possible implementation, the display requirements include full display and partial display; the processing unit 803 specifically is configured to:

shrinking the first area;

In a possible implementation manner, the receiving unit 801 is further configured to:

receiving an adjustment instruction input by a user;

the processing unit 803 is specifically configured to:

It should be noted that the implementation and beneficial effects of each unit may also correspond to the corresponding description of the method embodiment shown in fig. 2.

Referring to fig. 9, fig. 9 is a schematic structural diagram of a screen projection device 900 according to an embodiment of the present application, where the device is applied to a cloud server, and the device includes a receiving unit 901, a processing unit 902, and a sending unit 903, where the detailed descriptions of the respective units are as follows.

A receiving unit 901, configured to receive a first file sent by a first device in a cloud conference;

the processing unit 902 is configured to record an interface for opening the first file, so as to obtain a first audio/video stream;

a sending unit 903, configured to send the first audio and video stream to a large-screen device in the cloud conference, where the first audio and video stream is used to display the first audio and video stream in a first area of a display screen;

The receiving unit 901 is configured to receive a second file sent by a second device in the cloud conference;

the processing unit 902 is configured to record an interface for opening the second file, to obtain a second audio/video stream;

the sending unit 903 is configured to send the second audio/video stream to the large-screen device, where the second audio/video stream is used for displaying in a second area of the display screen.

In a possible implementation manner, the receiving unit 901 is further configured to:

receiving a first request sent by the second equipment;

the sending unit 903 is further configured to:

the receiving unit 901 is further configured to:

receiving acknowledgement information sent by the first equipment;

the sending unit 903 is further configured to:

Referring to fig. 10, fig. 10 is a schematic structural diagram of a screen projection device 1000 according to an embodiment of the present application, where the device 1000 includes a processor 1001 and a transceiver 1003, and optionally, a memory 1002, and the processor 1001, the memory 1002 and the transceiver 1003 are connected to each other by a bus 1004.

Memory 1002 includes, but is not limited to, random access memory (random access memory, RAM), read-only memory (ROM), erasable programmable read-only memory (erasable programmable read only memory, EPROM), or portable read-only memory (compact disc read-only memory, CD-ROM), with memory 1002 for the associated instructions and data. The transceiver 1003 is used to receive and transmit data.

The processor 1001 may be one or more central processing units (central processing unit, CPU), and in the case where the processor 1001 is one CPU, the CPU may be a single-core CPU or a multi-core CPU.

The processor 1001 in the apparatus 1000 reads the program code stored in the memory 1002 for performing the following operations:

determining a second area in the display screen;

and displaying the second audio and video stream in the second area.

Optionally, the first area includes a blank area; the processor 1001 is further configured to determine the second area according to the size of the blank area and/or the display requirement of the first audio/video stream.

Optionally, the processor 1001 is further configured to place the second area in the blank area of the first area when the size of the blank area is greater than a preset area threshold; and when the size of the blank area is smaller than or equal to the preset area threshold, determining the second area according to the display requirement of the first audio and video stream.

Optionally, the display requirement includes a full display and a partial display; the processor 1001 is further configured to zoom out the first area; when the display requirement is the whole display, the second area is placed in other areas of the display screen except the reduced first area, wherein an overlapping area does not exist between the reduced first area and the second area; and when the display requirement is the partial display, placing the second area in the area of the display screen, wherein an overlapping area exists between the reduced first area and the second area.

Optionally, receiving, by the transceiver 1003, an adjustment instruction input by a user; the processor 1001 is further configured to adjust the first area and/or the second area according to the adjustment instruction, to obtain an adjusted first area and/or an adjusted second area, where the adjusted first area is used for displaying the first audio/video stream, and the adjusted second area is used for displaying the second audio/video stream.

It should be noted that the implementation and beneficial effects of each operation may also correspond to the corresponding description of the method embodiment shown in fig. 2.

Referring to fig. 11, fig. 11 is a schematic structural diagram of a screen projection device 1100 according to an embodiment of the present application, where the device 1100 includes a processor 1101 and a transceiver 1103, and optionally, a memory 1102, and the processor 1101, the memory 1102 and the transceiver 1103 are connected to each other through a bus 1104.

Memory 1102 includes, but is not limited to, random access memory (random access memory, RAM), read-only memory (ROM), erasable programmable read-only memory (erasable programmable read only memory, EPROM), or portable read-only memory (compact disc read-only memory, CD-ROM), with memory 1102 for associated instructions and data. The transceiver 1103 is used to receive and transmit data.

The processor 1101 may be one or more central processing units (central processing unit, CPU), and in the case where the processor 1101 is one CPU, the CPU may be a single-core CPU or a multi-core CPU.

The processor 1101 in the apparatus 1100 reads the program code stored in the memory 1102, for performing the following operations:

receiving a first file sent by first equipment in a cloud conference;

receiving a second file sent by a second device in the cloud conference;

Optionally, receiving, by the transceiver 1103, a first request sent by the second device; sending the first request to the first device, wherein the first request is used for requesting whether the first device allows the second device to throw a screen on the large-screen device; receiving acknowledgement information sent by the first equipment; and sending the confirmation information to the second device, wherein the confirmation information is used for indicating that the second device is allowed to throw a screen on the large-screen device.

Optionally, the transceiver 1103 receives the screen-throwing information sent by the large-screen device, where the screen-throwing information includes identification information of the second device, and the identification information of the second device is selected from the identification information of at least one device by the user of the first device; the processor 1101 is further configured to send, to the second device, indication information according to the screen-casting information, where the indication information is used to indicate that screen-casting is started on the large-screen device.

It should be understood that the above-mentioned screen projection device may be a chip. For example, the screen projection device may be a field programmable gate array (field programmable gate array, FPGA), an application specific integrated chip (application specific integrated circuit, ASIC), a system on chip (SoC), a central processing unit (central processor unit, CPU), a network processor (network processor, NP), a digital signal processing circuit (digital signal processor, DSP), a microcontroller (micro controller unit, MCU), a programmable controller (programmable logic device, PLD) or other integrated chip.

In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or by instructions in the form of software. The steps of a method disclosed in connection with the embodiments of the present application may be embodied directly in a hardware processor for execution, or in a combination of hardware and software modules in the processor for execution. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in a memory, and the processor reads the information in the memory and, in combination with its hardware, performs the steps of the above method. To avoid repetition, a detailed description is not provided herein.

It should be noted that the processor in the embodiments of the present application may be an integrated circuit chip with signal processing capability. In implementation, the steps of the above method embodiments may be implemented by integrated logic circuits of hardware in a processor or instructions in software form. The processor may be a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, or discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be embodied directly in the execution of a hardware decoding processor, or in the execution of a combination of hardware and software modules in a decoding processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in a memory, and the processor reads the information in the memory and, in combination with its hardware, performs the steps of the above method.

According to a method provided by an embodiment of the present application, the present application also provides a computer program product, including: a computer program which, when run on a computer, causes the computer to perform the method of any of the embodiments shown in fig. 2.

According to the method provided by the embodiment of the application, the application further provides a computer readable medium, wherein the computer readable medium stores a computer program, and when the computer program runs on a computer, the computer program causes the computer to execute the method of any one of the embodiments shown in fig. 2.

According to the method provided by the embodiment of the application, the application further provides a screen projection system which comprises the one or more cloud servers and the one or more terminal devices.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the processes or functions described in accordance with embodiments of the present application are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a high-density digital video disc (digital video disc, DVD)), or a semiconductor medium (e.g., a Solid State Disk (SSD)), or the like.

Those of ordinary skill in the art will appreciate that the various illustrative logical blocks (illustrative logical block) and steps (steps) described in connection with the embodiments disclosed herein can be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a positioning server, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a random access memory (random access memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

The foregoing is merely illustrative of the present application, and the present application is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A screen projection method, comprising:

displaying the first audio and video stream in a first area of a display screen, wherein the first area comprises a blank area;

when the size of the blank area is larger than a preset area threshold value, a second area is placed in the blank area of the first area;

When the size of the blank area is smaller than or equal to the preset area threshold, determining the second area according to the display requirement of the first audio and video stream, wherein the display requirement comprises all display and part display;

and displaying the second audio and video stream in the second area.

2. The method of claim 1, wherein determining the second region according to the display requirement of the first audio-video stream comprises:

shrinking the first area;

3. The method according to claim 1 or 2, characterized in that the method further comprises:

receiving an adjustment instruction input by a user;

4. A screen projection method, comprising:

receiving a first file sent by first equipment in a cloud conference;

the first audio and video stream is sent to large-screen equipment in the cloud conference, and the first audio and video stream is used for displaying the large-screen equipment in a first area of a display screen, wherein the first area comprises a blank area;

receiving a second file sent by a second device in the cloud conference;

and sending the second audio and video stream to the large-screen device, wherein the second audio and video stream is used for displaying the large-screen device in a second area of the display screen, when the size of the blank area is larger than a preset area threshold, the second area is an area where the large-screen device is placed in the blank area, and when the size of the blank area is smaller than or equal to the preset area threshold, the second area is determined by the large-screen device according to the display requirement of the first audio and video stream, and the display requirement comprises all display and partial display.

5. The method according to claim 4, wherein the method further comprises:

receiving a first request sent by the second equipment;

receiving acknowledgement information sent by the first equipment;

6. The method according to claim 4, wherein the method further comprises:

7. A screen projection device, comprising:

The display unit is used for displaying the first audio and video stream in a first area of a display screen, wherein the first area comprises a blank area;

a processing unit, configured to place a second area in the blank area of the first area when the size of the blank area is greater than a preset area threshold; when the size of the blank area is smaller than or equal to the preset area threshold, determining the second area according to the display requirement of the first audio and video stream, wherein the display requirement comprises all display and part display;

8. The utility model provides a throw screen device which characterized in that is applied to cloud ware, includes:

The sending unit is used for sending the first audio-video stream to the large-screen equipment in the cloud conference, wherein the first audio-video stream is used for displaying the large-screen equipment in a first area of a display screen, and the first area comprises a blank area;

the sending unit is configured to send the second audio/video stream to the large-screen device, where the second audio/video stream is used for the large-screen device to display in a second area of the display screen, when the size of the blank area is greater than a preset area threshold, the second area is an area where the large-screen device is placed in the blank area, and when the size of the blank area is less than or equal to the preset area threshold, the second area is determined by the large-screen device according to a display requirement of the first audio/video stream, where the display requirement includes all display and part display.

9. A screen projection device, characterized in that the device comprises a processor and a memory, the memory being adapted to store a computer program, the processor being adapted to invoke the computer program to perform the method according to any of claims 1-3 or any of claims 4-6.

10. A computer readable storage medium for storing a computer program which, when run on a computer, causes the computer to perform the method of any one of claims 1-3 or any one of claims 4-6.