CN115278348A

CN115278348A - Screen projection method and device

Info

Publication number: CN115278348A
Application number: CN202210782324.XA
Authority: CN
Inventors: 王志军
Original assignee: Shenzhen Happycast Technology Co Ltd
Current assignee: Shenzhen Happycast Technology Co Ltd
Priority date: 2022-07-05
Filing date: 2022-07-05
Publication date: 2022-11-01
Anticipated expiration: 2042-07-05
Also published as: CN115278348B

Abstract

The embodiment of the application provides a screen projection method and device. The method comprises the following steps: receiving a first audio and video stream sent by a cloud server, wherein the first audio and video stream is determined by the cloud server according to a first file, and the first file is sent to the cloud server by first equipment in a cloud conference; displaying a first audio and video stream in a first area of a display screen; receiving a second audio/video stream sent by the cloud server, wherein the second audio/video stream is determined by the cloud server according to a second file, and the second file is sent to the cloud server by a second device in the cloud conference; determining a second area in the display screen; and displaying the second audio and video stream in the second area. According to the method, the large-screen device can display a plurality of files at the same time through interactive communication among the first device, the second device, the large-screen device and the cloud server in the cloud conference, and the conference efficiency is improved.

Description

Screen projection method and device

Technical Field

The present application relates to the field of communications technologies, and in particular, to a screen projection method and apparatus.

Background

With the rapid development of internet technology, online conferences using a network as a medium can break through the restrictions of time and regions, and realize the effect of face-to-face communication directly through the internet. When an online meeting is carried out in a meeting room, meeting file display and explanation are carried out by using large-screen equipment of the meeting room, a speaker casts the meeting file to the large-screen equipment through user equipment, the content of the meeting file is explained in the full-screen state of the large-screen equipment, and when the key function or the detail of the content is explained, a participant is expected to directly watch the demonstration process of the key function or the detail.

At this time, the large-screen device is displaying the content of the conference file that the device of the main speaker is projecting the screen in full screen, and the presentation process cannot be displayed on the large-screen device together with the conference file, which affects the effect of the online conference.

Disclosure of Invention

The embodiment of the application provides a screen projection method and device, and the large-screen device can display a plurality of files simultaneously through interactive communication among a first device, a second device, the large-screen device and a cloud server in a cloud conference, so that the conference efficiency is improved.

In a first aspect, an embodiment of the present application provides a screen projection method, including:

receiving a first audio and video stream sent by a cloud server, wherein the first audio and video stream is determined by the cloud server according to a first file, and the first file is sent to the cloud server by first equipment in a cloud conference;

displaying the first audio and video stream in a first area of a display screen;

receiving a second audio and video stream sent by the cloud server, wherein the second audio and video stream is determined by the cloud server according to a second file, and the second file is sent to the cloud server by second equipment in the cloud conference;

determining a second area in the display screen;

and displaying the second audio and video stream in the second area.

In the method, a large-screen device receives a first audio and video stream sent by a cloud server and displays the first audio and video stream in a first area of a display screen, and at the moment, the large-screen device displays the first audio and video stream in a full-screen state, wherein the first audio and video stream is formed by analyzing a first file sent by the first device by the cloud server; when the large-screen device receives a second audio/video stream sent by the cloud server, immediately reserving an auxiliary screen projection area for a first area of the current display screen in a full-screen state, namely determining a second area in the display screen, and displaying the second audio/video stream in the second area, wherein the second audio/video stream is formed after the cloud server analyzes a second file sent by the second device. According to the method, the large-screen device can adaptively adjust the first area and the second area of the display screen through interactive communication among the first device, the second device, the large-screen device and the cloud server in the cloud conference, so that audio and video streams corresponding to various files are displayed simultaneously, and the efficiency of the online conference is improved.

In one possible implementation, the first area includes a blank area;

the determining a second region in the display screen comprises:

and determining the second area according to the size of the blank area and/or the display requirement of the first audio and video stream.

In the embodiment of the application, the first area comprises a blank area, wherein the blank area refers to an area except for a content display area of the first audio/video stream in the first area, that is, the blank area does not contain the content of the first audio/video stream.

In a possible implementation manner, the determining the second area according to the size of the blank area and/or the display requirement of the first audio/video stream includes:

when the size of the blank area is larger than a preset area threshold, placing the second area in the blank area of the first area;

and when the size of the blank area is smaller than or equal to the preset area threshold, determining the second area according to the display requirement of the first audio and video stream.

In the embodiment of the application, when the size of the blank area is larger than the preset area threshold, it is indicated that the area of the blank area is large, the second area is directly placed in the blank area of the first area, two audio and video streams can be displayed in the area of the display screen in the maximized manner, when the size of the blank area is smaller than or equal to the preset area threshold, it is indicated that the area of the blank area is not large enough, and at this time, the second area needs to be further determined according to the display requirement of the first audio and video stream. According to the method, the first area and the second area are determined by selecting a proper adjusting mode according to the characteristics of the first audio and video stream, and a fine processing process is realized.

In one possible implementation, the display requirements include full display and partial display;

the determining the second area according to the display requirement of the first audio/video stream includes:

zooming out the first area;

when the display requirement is the full display, placing the second area in other areas except the reduced first area in the display screen, wherein the reduced first area and the second area do not have an overlapping area;

and when the display requirement is the partial display, placing the second area in an area of the display screen, wherein the reduced first area and the reduced second area have an overlapping area.

In the embodiment of the application, when the size of the blank area is smaller than or equal to the preset area threshold, firstly reducing the first area, if the display requirement is that all the audio and video are displayed, the first audio and video and the second audio and video are required to be completely displayed, at this time, placing the second area in other areas except the reduced first area in the display screen, wherein the reduced first area and the reduced second area do not have overlapping areas, so that two audio and video streams can be completely displayed, and the display requirement is met; if the display requirement is partial display, it is indicated that the first audio/video stream does not need to be completely displayed, and therefore, the second area is placed in the area of the display screen, and the second area and the reduced first area have an overlapping area, but only the second audio/video stream corresponding to the second area needs to be completely displayed. The method can select a display mode according to the requirements of the user, so that the second area is determined, and the requirements of the user are better met while the audio and video streams corresponding to various files are displayed.

In one possible implementation, the method further includes:

receiving an adjusting instruction input by a user;

and adjusting the first area and/or the second area according to the adjusting instruction to obtain an adjusted first area and/or an adjusted second area, wherein the adjusted first area is used for displaying the first audio and video stream, and the adjusted second area is used for displaying the second audio and video stream.

In the embodiment of the application, the first area and the second area are determined after self-adaptive adjustment by the large-screen device, and a user can input an adjustment instruction to adjust the first area and/or the second area according to the meeting requirements and actual conditions (for example, repeated display is needed and amplified display is needed) in the meeting process, so that the display screen can display audio and video streams corresponding to various files more fit the actual conditions.

In a second aspect, an embodiment of the present application provides a screen projection method, including:

receiving a first file sent by first equipment in a cloud conference;

recording an interface for opening the first file to obtain a first audio/video stream;

sending the first audio and video stream to large-screen equipment in the cloud conference, wherein the first audio and video stream is used for being displayed in a first area of a display screen;

receiving a second file sent by second equipment in the cloud conference;

recording and opening an interface of the second file to obtain a second audio and video stream;

and sending the second audio and video stream to the large-screen device, wherein the second audio and video stream is used for displaying in a second area of the display screen.

In the method, a cloud server receives a first file sent by first equipment in a cloud conference, records an interface for opening the first file to obtain a first audio and video stream, and sends the first audio and video stream to large-screen equipment in the cloud conference, wherein the large-screen equipment displays the first audio and video stream in a first area of a display screen; when the cloud server receives a second file sent by another device (second device) in the cloud conference, recording and opening an interface of the second file to obtain a second audio and video stream, sending the second audio and video stream to the large-screen device, and displaying the second audio and video stream in a second area of the display screen by the large-screen device. According to the method, the large-screen device can adaptively adjust the first area and the second area of the display screen through interactive communication among the first device, the second device, the large-screen device and the cloud server in the cloud conference, so that audio and video streams corresponding to various files are displayed simultaneously, and the efficiency of the online conference is improved.

In one possible implementation, the method further includes:

receiving a first request sent by the second equipment;

sending the first request to the first device, wherein the first request is used for requesting whether the first device allows the second device to cast a screen on the large-screen device;

receiving confirmation information sent by the first equipment;

and sending the confirmation information to the second equipment, wherein the confirmation information is used for indicating that the second equipment is allowed to cast a screen on the large-screen equipment.

In the embodiment of the application, when the second device casts a screen on the large-screen device, a screen casting request (a first request) needs to be sent to the first device first, and the second device cannot directly communicate with the first device, so that the second device sends the first request to the cloud server first, the cloud server forwards the first request to the first device, and after the user determines that the screen is cast through the first device, the first device sends the confirmation information to the cloud server, and the cloud server forwards the confirmation information to the second device, so that the second device is allowed to cast the screen on the large-screen device. According to the method, each device in the cloud conference can communicate with each other through the cloud server, and the first device can authorize the screen projection request of the second device in such a way, so that a plurality of screen projection files are displayed on the large-screen device at the same time.

In one possible implementation, the method further includes:

receiving screen projection information sent by the large-screen device, wherein the screen projection information comprises identification information of the second device, and the identification information of the second device is selected from identification information of at least one device by a user of the first device;

and sending indication information to the second equipment according to the screen projection information, wherein the indication information is used for indicating that screen projection is started on the large-screen equipment.

In the embodiment of the application, when a user uses the large-screen device to explain a file, the large-screen device directly triggers screen projection information, the large-screen device sends the screen projection information to the cloud server, and the cloud server sends indication information to the second device according to the screen projection information to indicate the second device to start screen projection on the large-screen device. The method is another mode for authorizing the second device to screen on the large-screen device, the second device does not need to actively send a screen-projecting request, but a screen-projecting instruction is directly triggered by the user at the large-screen device end, and the method is more suitable for the actual situation of the online conference.

In a third aspect, an embodiment of the present application provides a screen projection apparatus, including:

the device comprises a receiving unit, a processing unit and a processing unit, wherein the receiving unit is used for receiving a first audio and video stream sent by a cloud server, the first audio and video stream is determined by the cloud server according to a first file, and the first file is sent to the cloud server by first equipment in a cloud conference;

the display unit is used for displaying the first audio and video stream in a first area of a display screen;

the receiving unit is used for receiving a second audio/video stream sent by the cloud server, wherein the second audio/video stream is determined by the cloud server according to a second file, and the second file is sent to the cloud server by a second device in the cloud conference;

a processing unit for determining a second area in the display screen;

and the display unit is used for displaying the second audio and video stream in the second area.

In one possible implementation, the first area includes a blank area; the processing unit is specifically configured to:

In a possible implementation manner, the processing unit is specifically configured to:

when the size of the blank area is larger than a preset area threshold value, placing the second area in the blank area of the first area;

In one possible implementation, the display requirements include full display and partial display; the processing unit is specifically configured to:

zooming out the first area;

In a possible implementation manner, the receiving unit is further configured to:

receiving an adjusting instruction input by a user;

the processing unit is specifically configured to:

The operations and advantageous effects executed by the screen projection device can be referred to the method and advantageous effects of any one of the first aspect, and repeated details are not repeated.

In a fourth aspect, an embodiment of the present application provides a screen projection apparatus, which is applied to a cloud server, and includes:

the receiving unit is used for receiving a first file sent by first equipment in a cloud conference;

the processing unit is used for recording and opening the interface of the first file to obtain a first audio and video stream;

the sending unit is used for sending the first audio and video stream to large-screen equipment in the cloud conference, and the first audio and video stream is used for being displayed in a first area of a display screen;

the receiving unit is used for receiving a second file sent by second equipment in the cloud conference;

the processing unit is used for recording and opening the interface of the second file to obtain a second audio and video stream;

the sending unit is used for sending the second audio and video stream to the large-screen device, and the second audio and video stream is used for being displayed in a second area of the display screen.

receiving a first request sent by the second equipment;

the sending unit is further configured to:

the receiving unit is further configured to:

receiving confirmation information sent by the first equipment;

the sending unit is further configured to:

The operations and advantageous effects executed by the screen projection device can be referred to the method and advantageous effects of any one of the second aspects, and repeated details are not repeated.

In a fifth aspect, the present application provides a screen projection apparatus comprising a processor and a memory for storing a computer program; the processor is configured to execute the computer program stored in the memory to cause the screen projection device to perform the method according to any one of the first aspect or the second aspect.

In a sixth aspect, the present application provides a screen projection apparatus, which includes a processor, a memory, and a transceiver for receiving a channel or a signal or transmitting a channel or a signal; the memory for storing a computer program; the processor is configured to invoke the computer program from the memory to perform the method according to any one of the first aspect or the second aspect.

In a seventh aspect, the present application provides a screen projection apparatus, which includes a processor and an interface circuit, where the interface circuit is configured to receive a computer program and transmit the computer program to the processor; the processor runs the computer program to perform the method according to any one of the first aspect or the second aspect.

In an eighth aspect, the present application provides a computer readable storage medium for storing a computer program which, when executed, causes the method of any one of the first or second aspects to be carried out.

In a ninth aspect, the present application provides a computer program product comprising a computer program which, when executed, causes the method of any one of the first or second aspects to be carried out.

In a tenth aspect, an embodiment of the present application provides a screen projection system, where the screen projection system includes at least one cloud server and at least one terminal device, the terminal device is configured to perform the steps of any one of the first aspect, and the cloud server is configured to perform the steps of any one of the second aspect.

Drawings

The drawings used in the embodiments of the present application are described below.

Fig. 1 is a schematic diagram of a screen projection system provided in an embodiment of the present application;

FIG. 2 is a flowchart of a screen projection method provided by an embodiment of the present application;

FIG. 3 is a schematic diagram of a main interface of an application provided in an embodiment of the present application;

FIG. 4 is a schematic interface diagram of a device for selecting screen projection according to an embodiment of the present application;

FIG. 5 is a schematic illustration of an overlay area provided by an embodiment of the present application;

FIG. 6 is a schematic diagram of a composite molding area provided in an embodiment of the present application;

FIG. 7 is a schematic diagram of an overlapping region provided in an embodiment of the present application;

fig. 8 is a schematic structural diagram of a screen projection apparatus 800 according to an embodiment of the present application;

fig. 9 is a schematic structural diagram of a screen projection device 900 according to an embodiment of the present application;

fig. 10 is a schematic structural diagram of a screen projection device 1000 according to an embodiment of the present application;

fig. 11 is a schematic structural diagram of a screen projection apparatus 1100 according to an embodiment of the present application.

Detailed Description

Some terms referred to in the present application are explained below to facilitate understanding by those skilled in the art.

1. The cloud conference is a conference group created on a cloud server by a terminal device through an application program supporting the online conference, and is used for joining participants of the online conference and carrying out the online conference in the cloud conference. The main interface of the cloud conference comprises four display areas, and each display area is the content of a cloud desktop, wherein the content of the cloud desktop refers to the content of a file shared by terminal devices in the cloud conference, such as a document, a table, a video, an audio, and a screen shot of the terminal devices. It should be noted that the four display areas of the main interface of the cloud conference may be slid to the right, for example, the current four display areas in the main interface are respectively a first display area, a second display area, a third display area, and a fourth display area, at this time, the current four display areas in the main interface are slid to the right, and the fifth display area, the sixth display area, the seventh display area, and the eighth display area are displayed in the main interface, so that when the number of the shared files in the cloud conference exceeds four, the shared files may be viewed in a manner of sliding to the right. The cloud conference can realize transmission of audio and video data, sharing and management of documents, remote telephone or video conference and the like.

2. And the audio and video stream is used for analyzing the file in the received file information by the cloud server, opening the successfully analyzed file after the file is successfully analyzed, recording the opening state of the file, the sound information and the like to form corresponding audio and video streams, and pushing the audio and video streams to all terminal equipment in the cloud conference for display.

The embodiments of the present application will be described below with reference to the drawings.

When an online meeting is carried out in a meeting room, meeting file display and explanation are carried out by using large-screen equipment of the meeting room, a speaker casts the meeting file to the large-screen equipment through user equipment, the content of the meeting file is explained in the full-screen state of the large-screen equipment, and when the key function or the detail of the content is explained, a participant is expected to directly watch the demonstration process of the key function or the detail. At the moment, the large-screen device displays the content of the conference file, which is shot by the device of the speaker, in a full screen mode, the presentation process cannot be displayed on the large-screen device together with the conference file, and the effect of an online conference is affected. Referring to fig. 1, fig. 1 is a schematic diagram of a screen projection system architecture provided in an embodiment of the present application, where the system includes a terminal device 101, a cloud server 102, and a large screen device 103, where:

the terminal apparatus 101 is a device having a processing capability and a data transceiving capability. The terminal device 101 installs and runs an application program supporting the online conference, a user account is logged in the application program, and a user can join the cloud conference by clicking a conference link, inputting a verification code, a conference number and the like by using the terminal device 101. The terminal device 101 may include a first device and a second device, where the first device is usually a terminal device of a speaker, the speaker is a participant who obtains a control authority of a conference desktop of a cloud conference through the first device, and the conference desktop is controlled by only a single user (a user corresponding to any one terminal device in the cloud conference) at the same time in the cloud conference. The second device is typically a terminal device of a participant other than the speaker, i.e. a device other than the first device in the cloud conference. The terminal device 101 may be a cloud tv, a Computer, a notebook Computer, a tablet Computer, a palmtop Computer, a desktop Computer, a diagnostic apparatus, a mobile phone, an Ultra-mobile Personal Computer (UMPC), a netbook, a Personal Digital Assistant (PDA), and the like.

Optionally, the terminal device 101 may refer to one of multiple terminal devices, and this embodiment of the present application is illustrated by the terminal device 101 only. The applications installed on the terminal devices may be the same, or the applications installed on the two terminal devices may be the same type of application on different operating system platforms, and the number of the terminal devices may be greater or smaller. For example, the number of the terminal devices is several tens or several hundreds, or more, and the number of the terminal devices and the device types are not limited in the embodiment of the present application.

The cloud server 102 is a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, web services, cloud communication, middleware services, domain name services, security services, content Delivery Networks (CDNs), and big data and artificial intelligence platforms. The cloud server 102 creates and manages a cloud conference, configures the authority of all terminal devices in the cloud conference, can also control the switching of a plurality of display areas in the cloud conference, analyzes the received file, opens the file which is analyzed successfully after the analysis is successful, records the opening state of the file, sound information and the like to form corresponding audio and video streams, and pushes the audio and video streams to all terminal devices with viewing authority in the cloud conference. The cloud server 102 provides computing or application services for the terminal device 101, and provides background services for applications supporting online conferences.

The terminal device 101 and the cloud server 102 may be directly or indirectly connected by wired or wireless communication.

Optionally, the cloud server 102 may undertake primary computing work, and the terminal device 101 may undertake secondary computing work; or, the cloud server 102 undertakes the secondary computing work, and the terminal device 101 undertakes the primary computing work; alternatively, the cloud server 102 and the terminal device 101 may respectively and individually undertake the classification work.

The large-screen device 103 is a large-screen end device in a conference room, and it should be noted that the large-screen device 103 may be a first device or not, when the large-screen device 103 is a first device, the large-screen device 103 is a device having processing capability and data transceiving capability, a speaker may join a cloud conference through the large-screen device 103, upload a file to a cloud server in the cloud conference, and then push an audio/video stream formed by the successfully parsed file to the large-screen device 103 by the cloud server for display. When the large-screen device 103 is not the first device, the large-screen device 103 may be a device other than the first device and the second device in the cloud conference, and at this time, the large-screen device 103 is mainly used for displaying various files that the first device and the second device cast a screen on the large-screen device 103. The large-screen device 103 includes a display screen, and the display screen may be a display screen of the large-screen device or a display screen of an external large-screen device. The Display screen may include a Liquid Crystal Display (LCD), a Cathode Ray Tube (CRT) Display, an LED Display (LED panel), a 3D Display, and the like.

The large screen device 103 and the cloud server 102 may be directly or indirectly connected through wired or wireless communication.

Referring to fig. 2, fig. 2 is a flowchart of a screen projection method provided in an embodiment of the present application, where the method includes, but is not limited to, the following steps:

s201: the first device sends the first file to the cloud server.

Specifically, the first device is any terminal device in a cloud conference, generally, the first device is a terminal device of a speaker, and the speaker is a participant who obtains a control authority of a conference desktop of the cloud conference through the first device, where the cloud conference may be created on a cloud server by the first device through an application program supporting the online conference, or created on the cloud server by another device (a device other than the first device) through an application program supporting the online conference. The first file may be a document file, a form file, a Photoshop proprietary format (PSD) file, an animation scene creation format (3 Dmax) file, a computer aided design format (DWG) file, a video file, an audio file, a screen projection of the first device, and the like.

The first device starts an application program supporting an online conference, logs in a user account in the application program to enter a main interface of the application program, as shown in fig. 3, fig. 3 is a schematic diagram of the main interface of the application program provided in the embodiment of the present application, and the main interface of the application program includes a user account name and a head portrait, scans a scan, starts a discussion, joins the discussion, and mirrors a screen. The scanning can be used for enabling the first device to scan the two-dimensional code of the cloud discussion group to enter the cloud conference, a new cloud conference can be created when the first device starts to discuss, the discussion can be added into the created cloud conference, and the screen mirror image can be used for enabling the first device to project the screen to the large-screen device.

The first device receives a click operation (click and scan, start discussion or join discussion) input by a user, and in response to the click operation, the first device displays a screen projection content interface which comprises a projection file and a screen projection, wherein the projection file is used for the first device to select a file on the local device, and the file can comprise a document file, a table file, a PSD file, a 3Dmax file, a DWG file, a video file, an audio file and the like; the screen projection is used for the first device to select a main screen of the local device. The first device receives a selection operation (selecting to cast a file or cast a screen) input by a user, selects a first file on the first device in response to the selection operation, and sends the first file to the cloud server.

Optionally, the cloud server and the first device may be directly or indirectly connected through wired or wireless communication.

S202: the cloud server records an interface for opening the first file to obtain a first audio and video stream.

Specifically, after the cloud server receives the first file, the first file is analyzed, and when the first file is successfully analyzed, the cloud server opens the successfully analyzed first file, records a current file interface for opening the first file, and forms a first audio and video stream.

Optionally, when the first file analysis fails, the cloud server sends prompt information to the first device, where the prompt information is used to instruct a speaker to record the application interface of the first file through the split-screen mirroring function and open the application interface of the first file, so as to form a first audio/video stream.

S203: the cloud server sends the first audio and video stream to the large-screen device.

Specifically, the large-screen device refers to a large-screen end device in a conference room. After the first audio and video stream is formed by the cloud server, the first audio and video stream is pushed (sent) to large-screen equipment in the cloud conference.

Optionally, the cloud server and the large-screen device may be directly or indirectly connected in a wired or wireless communication manner, and the cloud server may send the first audio/video stream to the large-screen device in a communication manner such as broadcast, unicast, or multicast.

It should be noted that the cloud server may also send the first audio/video stream to all devices (the first device, the second device, the large-screen device, and other devices) in the cloud conference.

S204: the large-screen device displays a first audio and video stream in a first area of a display screen.

Specifically, the display screen may be a display screen of a large-screen device, or may be a display screen of an external large-screen device. The Display screen may include a Liquid Crystal Display (LCD), a Cathode Ray Tube (CRT) Display, an LED Display (LED panel), a 3D Display, and the like. The first area is an area which displays the first audio and video stream in a full screen state on the display screen, namely, the full screen area.

After the large-screen device receives the first audio and video stream, the first audio and video stream is displayed in a full-screen state (a first area) of the display screen.

S205: and the second equipment sends the second file to the cloud server.

Specifically, the second device is typically a terminal device of a participant other than the speaker, that is, a device other than the first device in the cloud conference. The second file may be a document file, a form file, a PSD file, a 3Dmax file, a DWG file, a video file, an audio file, a screen shot of the second device, and the like.

In some embodiments, before step S205, the method further includes:

the cloud server receives a first request sent by second equipment and sends the first request to first equipment; the cloud server receives the confirmation information sent by the first device and sends the confirmation information to the second device.

Specifically, the second device receives a first request input by a user, and then sends the first request to the cloud server, and after receiving the first request, the cloud server forwards the first request to the first device, where the first request is used to request the first device whether to allow the second device to perform screen projection on the large-screen device. The method comprises the steps that after receiving a first request, first equipment responds to the first request and displays the first request, after receiving confirmation information input by a speaker, the first equipment responds to the confirmation information and sends the confirmation information to a cloud server, and after receiving the confirmation information, the cloud server forwards the confirmation information to second equipment, wherein the confirmation information is used for indicating that the second equipment is allowed to cast a screen on the large-screen equipment.

Further, after receiving the confirmation information, the second device sends the second file to the cloud server, where the specific implementation of sending the second file to the cloud server by the second device may refer to the manner in which the first device sends the first file to the cloud server in step S201, and details are not described here again.

In this embodiment, the method enables each device in the cloud conference to communicate with each other through the cloud server, and the first device may also authorize the screen-casting request of the second device in this way, so that the screen-casting files of the two devices are displayed on the large-screen device at the same time.

In some embodiments, before step S205, the method further includes:

the cloud server receives screen projection information sent by the large-screen device and sends prompt information to the second device according to the screen projection information.

Specifically, the screen projection information includes identification information of the second device, and the identification information of the second device is selected by a user of the first device from identification information of at least one device; the indication information is used for indicating the second device to start to project the screen on the large-screen device.

The method includes the steps that a large-screen device displays a first audio and video stream in a full-screen state (a first area) of a display screen, when a screen projecting instruction input by a main speaker is received, a screen projecting device interface is displayed in response to the screen projecting instruction, as shown in fig. 4, fig. 4 is an interface schematic diagram of a screen projecting device selected according to an embodiment of the application, the interface includes a device list, the device list includes identification information of at least one other device except the large-screen device and the first device in a cloud conference, the main speaker selects identification information of a second device needing screen projecting from the identification information of the at least one other device, the large-screen device receives screen projecting information (identification information of the second device) sent by the main speaker, the screen projecting information is sent to a cloud server in response to the screen projecting information, the cloud server searches the second device from all devices of the cloud after receiving the screen projecting information, and sends prompt information to the second device, and the second device sends a second file to the cloud server after receiving the prompt information.

In the embodiment, the method is another mode for authorizing the second device to screen on the large-screen device, the second device does not need to actively send a screen-projecting request, but a screen-projecting instruction is directly triggered by the speaker at the large-screen device, so that the method is more suitable for the actual situation of the online conference.

S206: and the cloud server records an interface for opening the second file to obtain a second audio and video stream.

Specifically, after the cloud server receives the second file, the second file is analyzed, and when the second file is successfully analyzed, the cloud server opens the successfully analyzed second file, records a current file interface for opening the second file, and forms a second audio and video stream.

It should be noted that, when the second file includes multiple files, the cloud server records an interface for opening each file in the multiple files, respectively, to obtain a second audio/video stream, where the second audio/video stream includes multiple audio/video streams.

Optionally, when the second file fails to be analyzed, the cloud server sends prompt information to the second device, where the prompt information is used to instruct the speaker to record the target software through the split-screen mirroring function and open the application interface of the second file, so as to form a second audio/video stream.

S207: and the cloud server sends a second audio and video stream to the large-screen device.

Specifically, the specific implementation manner of S207 may refer to the manner in which the cloud server in step S203 sends the first audio and video stream to the large-screen device, which is not described herein again.

S208: the large screen device determines a second area in the display screen.

In some embodiments, the first area includes a blank area, and the large-screen device determines the second area according to a size of the blank area and/or a display requirement of the first audio-video stream.

Specifically, the first area is an area where the first audio/video stream is displayed in a full-screen state on the display screen, that is, a full-screen area, where when the first audio/video stream is displayed on the first area, the content of the first audio/video stream may not be completely distributed in the first area, and in the first area, the area where the content of the first audio/video stream is not distributed is a blank area, and the blank area is related to the content of the first audio/video stream, and corresponding blank areas of different first audio/video streams are different from each other. The display requirements comprise full display and partial display, the display requirements refer to whether the content of the first audio/video stream can be shielded, and when the content of the first audio/video stream can be shielded, that is, the full content of the first audio/video stream does not need to be completely displayed on a display screen, the display requirements of the first audio/video stream are full display; when the content of the first audio/video stream cannot be shielded, that is, the whole content of the first audio/video stream needs to be completely displayed on the display screen, the display requirement of the first audio/video stream is partial display.

And after receiving the second audio/video stream sent by the cloud server, the large-screen device determines a second area according to the size of the blank area and/or the display requirement of the first audio/video stream. When the size of the blank area is larger than the preset area threshold, the large-screen device directly places the second area in the blank area of the first area, at this time, the first area and the second area are in an overlay type relationship, the original range of the first area is kept unchanged, and the second area is overlaid on the first area, as shown in fig. 5, fig. 5 is a schematic diagram of an overlay type area provided by an embodiment of the present application.

And when the size of the blank area is smaller than or equal to the preset area threshold, the large-screen device determines a second area according to the display requirement of the first audio and video stream.

Specifically, the large-screen device first reduces the first area according to the proportion (the proportion of the length and the width) of the first area, and when the display requirement of the first audio/video stream is that the first audio/video stream is displayed completely, places the second area in the other area except for the reduced first area in the display screen, at this time, the reduced first area and the reduced second area are in a composite type relationship, and there is no overlapping area, as shown in fig. 6, fig. 6 is a schematic diagram of a composite type area provided in an embodiment of the present application; when the display requirement of the first audio/video stream is partial display, the second area is placed in an area of the display screen, at this time, the reduced first area and the second area are in an overlapping relationship, the reduced first area may be smaller than or equal to the second area, or may be larger than the second area, and there is an overlapping area between the reduced first area and the second area, as shown in fig. 7, fig. 7 is a schematic diagram of an overlapping area provided in an embodiment of the present application.

It should be noted that, when the second audio/video stream includes multiple audio/video streams, for example, when the second audio/video stream includes two audio/video streams, the large-screen device determines the second area and the third area in the display screen, where the specific implementation method for determining the third area may refer to the specific implementation method for determining the second area, and the application is not limited thereto.

In the embodiment, the large-screen device can select a proper adjusting mode to determine the first area and the second area according to the size of the blank area and/or the display requirement of the first audio/video stream, so that a fine processing process is realized, and the user requirements are better met while the audio/video streams corresponding to various files are displayed.

S209: and the large-screen device displays the second audio and video stream in the second area.

Specifically, after the large-screen device determines a second area and a first area in the display screen, a second audio/video stream is displayed in the second area, and a first audio/video stream is displayed in the first area.

It should be noted that, when the second audio/video stream includes multiple audio/video streams, for example, when the second audio/video stream includes two audio/video streams, after the large-screen device determines the second area and the third area in the display screen, one audio/video stream in the second audio/video stream is displayed in the second area, the other audio/video stream in the second audio/video stream is displayed in the third area, and the first audio/video stream is displayed in the first area, which is not limited in this application.

In some embodiments, the large-screen device receives an adjustment instruction input by a user; and adjusting the first area and/or the second area according to the adjusting instruction to obtain the adjusted first area and/or the adjusted second area.

Specifically, the adjustment instruction may include an adjustment area, adjustment information corresponding to the adjustment area, and area placement information, where the adjustment area includes at least one of the first area and the second area, the adjustment information corresponding to the adjustment area may include an adjustment ratio, the adjusted length and width of the area, and the area placement information may include a placement position of the first area and a placement position of the second area. The large-screen device respectively displays the first audio and video stream in the first area, and after the second audio and video stream is displayed in the second area and an adjusting instruction input by a speaker is received, the large-screen device responds to the adjusting instruction to adjust the first area and/or the second area to obtain the adjusted first area and/or the adjusted second area.

In this embodiment, the first area and the second area are determined after self-adaptive adjustment by the large-screen device, and a user can input an adjustment instruction to adjust the first area and/or the second area in the conference process according to the conference requirements and actual conditions (for example, repeated display is required and amplified display is required), so that the display screen displays audio and video streams corresponding to various files more in accordance with the actual conditions.

In the embodiment of the application, a large-screen device receives a first audio and video stream sent by a cloud server and displays the first audio and video stream in a first area of a display screen, and at the moment, the large-screen device displays the first audio and video stream in a full-screen state, wherein the first audio and video stream is formed by analyzing a first file sent by the first device by the cloud server; when the large-screen device receives a second audio/video stream sent by the cloud server, immediately reserving an auxiliary screen projection area for a first area of the current display screen in a full-screen state, namely determining a second area in the display screen, and displaying the second audio/video stream in the second area, wherein the second audio/video stream is formed after the cloud server analyzes a second file sent by the second device. According to the method, the large-screen device can adaptively adjust the first area and the second area of the display screen through interactive communication among the first device, the second device, the large-screen device and the cloud server in the cloud conference, so that audio and video streams corresponding to a plurality of files are displayed simultaneously, and the efficiency of the online conference is improved.

The method of the embodiments of the present application is explained in detail above, and the apparatus of the embodiments of the present application is provided below.

Referring to fig. 8, fig. 8 is a schematic structural diagram of a screen projection apparatus 800 according to an embodiment of the present application, and the apparatus includes a receiving unit 801, a display unit 802, and a processing unit 803, where details of each unit are as follows.

The receiving unit 801 is configured to receive a first audio/video stream sent by a cloud server, where the first audio/video stream is determined by the cloud server according to a first file, and the first file is sent to the cloud server by a first device in a cloud conference;

a display unit 802, configured to display the first audio/video stream in a first area of a display screen;

a processing unit 803 for determining a second area in the display screen;

the display unit 802 is configured to display the second audio/video stream in the second area.

In one possible implementation, the first area includes a blank area; the processing unit 803 is specifically configured to:

In a possible implementation manner, the processing unit 803 is specifically configured to:

In one possible implementation, the display requirements include a full display and a partial display; the processing unit 803 is specifically configured to:

zooming out the first area;

In a possible implementation manner, the receiving unit 801 is further configured to:

receiving an adjusting instruction input by a user;

the processing unit 803 is specifically configured to:

It should be noted that the implementation and beneficial effects of each unit can also correspond to the corresponding description of the method embodiment shown in fig. 2.

Referring to fig. 9, fig. 9 is a schematic structural diagram of a screen projection apparatus 900 provided in an embodiment of the present application, and the apparatus is applied to a cloud server and includes a receiving unit 901, a processing unit 902, and a sending unit 903, where details of each unit are described below.

A receiving unit 901, configured to receive a first file sent by a first device in a cloud conference;

the processing unit 902 is configured to record an interface for opening the first file, and obtain a first audio/video stream;

a sending unit 903, configured to send the first audio/video stream to a large-screen device in the cloud conference, where the first audio/video stream is used for displaying in a first area of a display screen;

the receiving unit 901 is configured to receive a second file sent by a second device in the cloud conference;

the processing unit 902 is configured to record an interface for opening the second file, so as to obtain a second audio/video stream;

the sending unit 903 is configured to send the second audio/video stream to the large-screen device, where the second audio/video stream is used to be displayed in a second area of the display screen.

In a possible implementation manner, the receiving unit 901 is further configured to:

receiving a first request sent by the second equipment;

the sending unit 903 is further configured to:

sending the first request to the first device, wherein the first request is used for requesting whether the first device allows the second device to shoot the screen on the large-screen device;

the receiving unit 901 is further configured to:

receiving confirmation information sent by the first equipment;

the sending unit 903 is further configured to:

It should be noted that the implementation and the beneficial effects of each unit can also correspond to the corresponding description of the method embodiment shown in fig. 2.

Referring to fig. 10, fig. 10 is a schematic structural diagram of a screen projection apparatus 1000 according to an embodiment of the present application, where the apparatus 1000 includes a processor 1001 and a transceiver 1003, and optionally further includes a memory 1002, and the processor 1001, the memory 1002, and the transceiver 1003 are connected to each other through a bus 1004.

The memory 1002 includes, but is not limited to, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or a portable read-only memory (CD-ROM), and the memory 1002 is used for storing instructions and data. The transceiver 1003 is used to receive and transmit data.

The processor 1001 may be one or more Central Processing Units (CPUs), and in the case where the processor 1001 is one CPU, the CPU may be a single-core CPU or a multi-core CPU.

The processor 1001 in the device 1000 reads the program code stored in the memory 1002 for performing the following operations:

receiving a first audio/video stream sent by a cloud server, wherein the first audio/video stream is determined by the cloud server according to a first file, and the first file is sent to the cloud server by first equipment in a cloud conference;

determining a second area in the display screen;

and displaying the second audio and video stream in the second area.

Optionally, the first area includes a blank area; the processor 1001 is further configured to determine the second area according to the size of the blank area and/or the display requirement of the first audio/video stream.

Optionally, the processor 1001 is further configured to place the second area in the blank area of the first area when the size of the blank area is greater than a preset area threshold; and when the size of the blank area is smaller than or equal to the preset area threshold, determining the second area according to the display requirement of the first audio and video stream.

Optionally, the display requirement includes full display and partial display; the processor 1001 is further configured to narrow down the first area; when the display requirement is the full display, placing the second area in other areas except the reduced first area in the display screen, wherein the reduced first area and the second area do not have an overlapping area; and when the display requirement is the partial display, placing the second area in an area of the display screen, wherein the reduced first area and the reduced second area have an overlapping area.

Optionally, an adjustment instruction input by a user is received through the transceiver 1003; the processor 1001 is further configured to adjust the first area and/or the second area according to the adjustment instruction to obtain an adjusted first area and/or an adjusted second area, where the adjusted first area is used to display the first audio/video stream, and the adjusted second area is used to display the second audio/video stream.

It should be noted that the implementation and beneficial effects of the operations may also correspond to the corresponding descriptions of the method embodiment shown in fig. 2.

Referring to fig. 11, fig. 11 is a schematic structural diagram of a screen projection apparatus 1100 according to an embodiment of the present application, where the apparatus 1100 includes a processor 1101, a transceiver 1103, and optionally a memory 1102, and the processor 1101, the memory 1102, and the transceiver 1103 are connected to each other through a bus 1104.

The memory 1102 includes, but is not limited to, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or a portable read-only memory (CD-ROM), and the memory 1102 is used for related instructions and data. The transceiver 1103 is used for receiving and transmitting data.

The processor 1101 may be one or more Central Processing Units (CPUs), and in the case where the processor 1101 is one CPU, the CPU may be a single-core CPU or a multi-core CPU.

The processor 1101 in the apparatus 1100 reads the program code stored in the memory 1102 for performing the following operations:

receiving a first file sent by first equipment in a cloud conference;

recording and opening an interface of the first file to obtain a first audio and video stream;

receiving a second file sent by second equipment in the cloud conference;

Optionally, the transceiver 1103 receives a first request sent by the second device; sending the first request to the first device, wherein the first request is used for requesting whether the first device allows the second device to cast a screen on the large-screen device; receiving confirmation information sent by the first equipment; and sending the confirmation information to the second equipment, wherein the confirmation information is used for indicating that the second equipment is allowed to cast a screen on the large-screen equipment.

Optionally, the transceiver 1103 is configured to receive screen projection information sent by the large-screen device, where the screen projection information includes identification information of the second device, and the identification information of the second device is selected from identification information of at least one device by a user of the first device; the processor 1101 is further configured to send indication information to the second device according to the screen projection information, where the indication information is used to indicate that screen projection is started on the large-screen device.

It should be noted that, the implementation and the beneficial effects of the operations may also correspond to the corresponding description of the method embodiment shown in fig. 2.

It should be understood that the projection device may be a chip. For example, the screen projection device may be a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), a system on chip (SoC), a Central Processing Unit (CPU), a Network Processor (NP), a digital signal processing circuit (DSP), a Microcontroller (MCU), a Programmable Logic Device (PLD), or other integrated chips.

In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in a processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and combines hardware thereof to complete the steps of the method. To avoid repetition, it is not described in detail here.

It should be noted that the processor in the embodiments of the present application may be an integrated circuit chip having signal processing capability. In implementation, the steps of the above method embodiments may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software. The processor described above may be a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor.

According to the method provided by the embodiment of the present application, the present application further provides a computer program product, which includes: computer program, which, when run on a computer, causes the computer to perform the method of any one of the embodiments shown in fig. 2.

According to the method provided by the embodiment of the present application, the present application further provides a computer-readable medium, which stores a computer program, and when the computer program runs on a computer, the computer is caused to execute the method of any one of the embodiments shown in fig. 2.

According to the method provided by the embodiment of the application, the application further provides a screen projection system which comprises the one or more cloud servers and the one or more terminal devices.

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored on a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website, computer, server, or data center to another website, computer, server, or data center via wire (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a Digital Video Disc (DVD)), or a semiconductor medium (e.g., a Solid State Disc (SSD)), among others.

Those of ordinary skill in the art will appreciate that the various illustrative logical blocks and steps (step) described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the technical solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

The functions may be stored in a computer-readable storage medium if they are implemented in the form of software functional units and sold or used as separate products. Based on such understanding, the technical solutions of the present application, or portions thereof, which substantially or partly contribute to the prior art, may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a location server) to perform all or part of the steps of the method described in the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A screen projection method, comprising:

receiving a second audio/video stream sent by the cloud server, wherein the second audio/video stream is determined by the cloud server according to a second file, and the second file is sent to the cloud server by a second device in the cloud conference;

determining a second area in the display screen;

and displaying the second audio and video stream in the second area.

2. The method of claim 1, wherein the first area comprises a blank area;

the determining a second region in the display screen comprises:

3. The method of claim 2, wherein determining the second region according to the size of the blank region and/or the display requirements of the first audio/video stream comprises:

4. The method of claim 3, wherein the display requirements include full display and partial display;

narrowing the first region;

when the display requirement is the whole display, placing the second area in other areas except the reduced first area in the display screen, wherein the reduced first area and the second area have no overlapping area;

5. The method according to any one of claims 1-4, further comprising:

receiving an adjusting instruction input by a user;

6. A screen projection method, comprising:

receiving a first file sent by first equipment in a cloud conference;

receiving a second file sent by second equipment in the cloud conference;

7. The method of claim 6, further comprising:

receiving a first request sent by the second equipment;

receiving confirmation information sent by the first equipment;

sending the confirmation information to the second device, wherein the confirmation information is used for indicating that the second device is allowed to cast a screen on the large-screen device.

8. The method of claim 6, further comprising:

receiving screen projection information sent by the large-screen device, wherein the screen projection information comprises identification information of the second device, and the identification information of the second device is selected from the identification information of at least one device by a user of the first device;

9. A screen projection apparatus, comprising:

a processing unit for determining a second region in the display screen;

10. A screen projection device is applied to a cloud server and comprises:

the receiving unit is used for receiving a first file sent by first equipment in the cloud conference;

the sending unit is used for sending the first audio and video stream to large-screen equipment in the cloud conference, wherein the first audio and video stream is used for being displayed in a first area of a display screen;

11. A screen projection apparatus, characterized in that the apparatus comprises a processor and a memory for storing a computer program, the processor being adapted to invoke the computer program to perform the method of any of claims 1-5 or any of claims 6-8.

12. A computer-readable storage medium for storing a computer program which, when run on a computer, causes the computer to perform the method of any one of claims 1-5 or any one of claims 6-8.