CN114449300A

CN114449300A - Real-time video stream playing method and server

Info

Publication number: CN114449300A
Application number: CN202210027250.9A
Authority: CN
Inventors: 刘利明; 刘石勇
Original assignee: Hisense Group Holding Co Ltd
Current assignee: Hisense Group Holding Co Ltd
Priority date: 2022-01-11
Filing date: 2022-01-11
Publication date: 2022-05-06
Anticipated expiration: 2042-01-11
Also published as: CN114449300B

Abstract

The application discloses a real-time video stream playing method and a server, and relates to the technical field of video playing. The server can determine a first target display device from a plurality of first display devices to display a second real-time video stream, the first real-time video stream collected by the first target display device comprises images of a first user, and the collection ranges of the cameras of the first display devices are different. Therefore, the server provided by the application can flexibly select the first target display device from the plurality of first display devices to play the second real-time video stream based on the position of the first user, and the second real-time video stream is not continuously pushed to a certain fixed first display device, so that the flexible playing of the real-time video stream is realized.

Description

Real-time video stream playing method and server

Technical Field

The present application relates to the field of video playing technologies, and in particular, to a playing method and a server for real-time video streams.

Background

During the video call, the display device (e.g., a mobile phone) may continuously send the acquired real-time video stream to the server, so that the server sends the real-time video stream to the display device of the opposite terminal for display.

Disclosure of Invention

The application provides a playing method and a server of real-time video stream, and the technical scheme is as follows:

on one hand, a playing method of a real-time video stream is provided, which is applied to a server, wherein the server is respectively connected with a plurality of first display devices of a first user and a second display device of a second user, wherein the plurality of first display devices are located at different positions in a first room, and the second display device is located in a second room; the method comprises the following steps:

receiving a first real-time video stream acquired by the plurality of first display devices through the cameras and a second real-time video stream acquired by the second display devices through the cameras, wherein the acquisition ranges of the cameras of the plurality of first display devices are different;

determining a first target display device from the plurality of first display devices based on first real-time video streams acquired by the plurality of first display devices, wherein the first real-time video streams acquired by the first target display device comprise images of the first user;

and sending the second real-time video stream to the first target display device, wherein the second real-time video stream is used for being played by the first target display device.

In another aspect, a server is provided, the server being connected to a plurality of first display devices of a first user and a second display device of a second user, respectively, wherein the plurality of first display devices are located at different positions in a first room and the second display device is located in a second room; the server includes: a processor; the processor is configured to:

Optionally, the processor is configured to:

determining a plurality of alternative display devices from the plurality of first display devices based on first real-time video streams acquired by the plurality of first display devices, wherein the first real-time video streams acquired by each alternative display device comprise images of the first user;

and determining the candidate display device which is closest to the first user in the plurality of candidate display devices as a first target display device.

Optionally, the processor is configured to:

receiving a first real-time video stream acquired by one first display device in the plurality of first display devices through a camera;

if the first real-time video stream acquired by the first display device does not include the image of the first user, controlling each other first display device except the first display device in the plurality of first display devices to acquire the first real-time video stream through a camera;

and receiving the first real-time video stream collected by each other first display device.

Optionally, the processor is further configured to:

if the first real-time video stream collected by the first target display equipment does not comprise the image of the first user, determining second target display equipment from the plurality of first display equipment based on the first real-time video stream collected by the plurality of first display equipment, wherein the first real-time video stream collected by the second target display equipment comprises the image of the first user;

and sending the second real-time video stream to the second target display device, wherein the second real-time video stream is used for being played by the second target display device.

Optionally, the processor is configured to:

sending a switching reminding message to the second target display device, wherein the switching reminding message is used for being displayed by the second target display device and prompting the first user whether to confirm that the second real-time video stream is played in the second target display device;

and if a confirmation instruction aiming at the switching reminding message is received, sending the second real-time video stream to the second target display equipment.

Optionally, the processor is further configured to:

and sending a first real-time video stream acquired by the first target display device to the second display device, wherein the first real-time video stream is used for being played by the second display device.

Optionally, the display screen of the first target display device has a display area and a first whiteboard demonstration area, where the display area is used to play the second real-time video stream; the processor is further configured to:

receiving a first whiteboard video stream sent by the first target display device, wherein the first whiteboard video stream is obtained by recording a screen of the first whiteboard demonstration area;

and sending the first whiteboard video stream to the second display device, wherein the first whiteboard video stream is used for being played by the second display device.

Optionally, the display screen of the second display device has a second whiteboard demonstration area, and the second whiteboard demonstration area is used for playing the first whiteboard video stream; the processor is further configured to:

receiving a second whiteboard video stream sent by the second display device, wherein the second whiteboard video stream is obtained by recording a screen of the second whiteboard demonstration area after the second display device receives a touch operation for the second whiteboard demonstration area, and an operation track of the touch operation is also displayed in the second whiteboard demonstration area;

and sending the second whiteboard video stream to the first target display device, wherein the second whiteboard video stream is used for being played by the first target display device in the first whiteboard demonstration area.

Optionally, a first real-time video stream acquired by the first target display device is a real-time learning video stream of the first user;

and the second real-time video stream collected by the second display device is the real-time learning video stream of the second user.

In yet another aspect, a server is provided, the server including: a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method for playing a real-time video stream as described in the above aspect when executing the computer program.

In yet another aspect, a computer-readable storage medium is provided, in which a computer program is stored, the computer program being loaded and executed by a processor to implement the method for playing back a real-time video stream as described in the above aspect.

In yet another aspect, a computer program product comprising instructions is provided, which when run on a server, causes the server to perform the method for playing a real-time video stream according to the above aspect.

The beneficial effect that technical scheme that this application provided brought includes at least:

the application provides a real-time video stream playing method and a server, wherein the server can determine a first target display device from a plurality of first display devices to display a second real-time video stream, and the first real-time video stream acquired by the first target display device comprises an image of a first user. Therefore, the server provided by the application can flexibly select the first target display device from the plurality of first display devices to play the second real-time video stream based on the position of the first user, and the second real-time video stream is not continuously pushed to a certain fixed first display device, so that the flexible playing of the real-time video stream is realized.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic structural diagram of a playing system for real-time video streams according to an embodiment of the present application;

fig. 2 is a flowchart of a method for playing a real-time video stream according to an embodiment of the present application;

fig. 3 is a flowchart of another method for playing a real-time video stream according to an embodiment of the present application;

fig. 4 is a schematic diagram of switching a first display device for displaying a second real-time video stream according to an embodiment of the present application;

fig. 5 is a schematic view of display interfaces of a first target display device and a second display device when the first target display device performs a whiteboard demonstration according to an embodiment of the present application;

fig. 6 is a schematic structural diagram of another real-time video stream playing system according to an embodiment of the present application.

Detailed Description

To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

Fig. 1 is a schematic structural diagram of a playing system of a real-time video stream according to an embodiment of the present application. As shown in fig. 1, the system may include: a server 01, a plurality of first display devices 02 (e.g., 3 first display devices 02 are shown in fig. 1) of a first user 110, and a second display device 03 of a second user 120. The server 01 may be connected to each of the first display device 02 and the second display device 03, respectively.

The first user and the second user are both parties for video discussion. The second display device 03 and each of the first display devices 02 have a camera, and may capture a real-time video stream via the camera. And, the second display device 03 and each first display device 02 also have a display screen, and can play the real-time video stream pushed by the server in the display screen. The plurality of first display devices 02 may be located at different positions in a first room, and the second display device is located in a second room, which is different from the first room.

Optionally, the server 01 may be one server, or may be a server cluster composed of several servers, or may be one cloud computing service center. Each of the first display device 02 and the second display device 03 may be a mobile phone, a tablet, a laptop, a desktop computer, or a smart television. The first room may be a first user's accommodation room (e.g., a home space), or an office room (e.g., a conference room), or other space. The second room may be a second user's accommodation room, or an office room (e.g., a conference room), or another space. The implementation manners of the first room and the second room are not limited, and only the first room and the second room are different.

The embodiment of the application provides a method for playing a real-time video stream, which can be applied to a server, such as the server shown in fig. 1. The server is respectively connected with a plurality of first display devices of a first user and a second display device of a second user, wherein the plurality of first display devices are located in different positions in a first room, and the second display device is located in a second room. Referring to fig. 2, the method includes:

step 101, receiving a first real-time video stream acquired by a plurality of first display devices through a camera, and a second real-time video stream acquired by a second display device through the camera.

In the process of video call of a first user and a second user, each first display device in the plurality of first display devices can acquire a first real-time video stream through the camera and can send the acquired first real-time video stream to the first display device. The second display device can also collect the second real-time video stream through the camera, and can send the collected second real-time video stream to the second display device. Accordingly, the server may receive a first real-time video stream transmitted by a plurality of first display devices and may receive a second real-time video stream transmitted by a second display device. The cameras of the first display devices are different in acquisition range.

Step 102, determining a first target display device from a plurality of first display devices based on a first real-time video stream collected by the plurality of first display devices.

The first real-time video stream collected by the first target display device comprises images of the first user. The image of the first user may include: at least one of a face image of the first user and a limb image of the first user.

If the position of the first user is within the capture range of a certain first display device, the first real-time video stream captured by the first display device may include an image of the first user. Therefore, the server provided by the embodiment of the application can determine the first target display device from the plurality of first display devices based on the position of the first user.

Step 103, sending the second real-time video stream to the first target display device.

After determining the first target display device, the server can send the second real-time video stream to the first target display device. And after the first target display equipment receives the second real-time video stream, the second real-time video stream can be played. In addition, the server can also send the first real-time video stream collected by the first target display device to the second display device for playing by the second display device, so that the first user and the second user can carry out video call.

To sum up, an embodiment of the present application provides a method for playing a real-time video stream, where a server may determine a first target display device from a plurality of first display devices to display a second real-time video stream, where the first real-time video stream acquired by the first target display device includes an image of a first user, and acquisition ranges of cameras of the plurality of first display devices are different. Therefore, the server provided by the embodiment of the application can flexibly select the first target display device from the plurality of first display devices to play the second real-time video stream based on the position of the first user, and does not continuously push the second real-time video stream to a certain fixed first display device, so that the flexible playing of the real-time video stream is realized.

Fig. 3 is another method for playing a real-time video stream according to an embodiment of the present application, where the method can be applied to a server, such as the server shown in fig. 1. Referring to fig. 3, the method may include:

step 201, receiving a first real-time video stream acquired by a plurality of first display devices through a camera, and a second real-time video stream acquired by a second display device through the camera.

In this embodiment of the application, each of the plurality of first display devices may acquire the first real-time video stream through the camera, and may send the acquired first real-time video stream to the first display device. The second display device can also collect the second real-time video stream through the camera, and can send the collected second real-time video stream to the second display device. Accordingly, the server may receive a first real-time video stream transmitted by a plurality of first display devices and may receive a second real-time video stream transmitted by a second display device. The cameras of the first display devices are different in acquisition range.

Optionally, each of the first display device and the second display device may collect a real-time video stream through the camera in a process of the first user and the second user performing a study discussion. Accordingly, the second real-time video stream captured by the second display device may be a real-time learning video stream for the second user. The first real-time video stream captured by one of the plurality of first display devices may be a real-time learning video stream of the first user.

Optionally, the multiple first display devices may acquire the first real-time video stream through their own cameras at the same time. Accordingly, the server may receive the first real-time video streams captured by the plurality of first display devices simultaneously.

Alternatively, a first display device of the plurality of first display devices may first capture a first real-time video stream via the camera. After the first real-time video stream acquired by the first display device does not include the image of the first user, the server may control each of the other first display devices except the first display device to acquire the first real-time video stream through its own camera. Accordingly, the server may receive the first real-time video stream collected by the first display device first, and then receive the first real-time video streams collected by each of the other first display devices.

Optionally, the image of the first user may include: at least one of a face image and a limb image of the first user. For example, the image of the first user may be a facial image of the first user. Each first display device may also capture a first real-time audio stream via a microphone and may transmit the captured first real-time audio stream to a server. The second display device may capture a second real-time audio stream via the microphone and send the captured second real-time audio stream to the server. Accordingly, the server may receive a first real-time audio stream sent by the first display device and a second real-time audio stream sent by the second display device. The first user and the second user may both be students.

Optionally, for each of the plurality of first display devices of the first user, the first display device may be connected to each of the other first display devices of the plurality of first display devices except the first display device. If the server determines that the first real-time video stream collected by one first display device does not include the image of the first user, a notification message can be sent to the one first display device. After receiving the notification message, the first display device may send a first video stream capture instruction to each of the other first display devices connected thereto. Each other first display device may capture a first real-time video stream through its camera in response to the first video stream capture instruction.

Or, the server stores in advance a target address set, where the target address set includes an Internet Protocol (IP) address of each first display device in the plurality of first display devices of the first user. After determining that the real-time video stream acquired by one first display device does not include the image of the first user, the server may determine a target address set to which the IP address of the one first display device belongs, and determine the IP addresses of each of the other first display devices except the one first display device from the target address set. Thereafter, for each other first display device, the server may send a first video stream capture instruction to the other first display device based on the IP address of the other first display device. After receiving the first video stream acquisition instruction, the other first display devices can respond to the first video stream acquisition instruction and acquire a first real-time video stream through the cameras thereof.

In this embodiment of the application, if a first user and a second user need to perform a video call, the first user may enable one first display device of the first user to enter a pre-established virtual chat room (for example, a virtual discussion room or a virtual conference room), and the second user may also enable a second display device of the second user to enter the virtual chat room. After the first display device and the second display device enter the virtual chat room, the first display device and the second display device can respectively acquire real-time video streams through the camera.

For a plurality of first display devices of a first user simultaneously acquiring scenes of a first real-time video stream through a camera, when one first display device joins a chat room, a third video stream acquisition instruction can be sent to each other first display device connected with the first display device. Each other first display device may capture a first real-time video stream via the camera in response to the third video stream capture instruction.

In the embodiment of the present application, the virtual chat room may be created by a first display device of the first user in response to a creation operation of the first user, or may be created by a second display device in response to a creation operation of the second user. Taking the creator of the virtual chat room as a first display device of the first user, and the virtual chat room is a virtual discussion room as an example, the creation process of the virtual chat room is exemplarily described.

The first user may touch an application identification of a discussion application installed in one of the first display devices. The background server of the discussion application may be the server described above. The one first display device may display a creation control of the virtual discussion room in response to a touch operation identified by the user for the application. Then, the one first display device can respond to the touch control of the first user aiming at the creation control, and a creation interface is displayed, wherein the creation interface can comprise a people number setting control, an open property setting control, a discussion theme setting control and a completion control. The first display device can respond to the touch operation of the first user aiming at the number setting control, and the total number of the display devices which can be added into the virtual discussion room is set; the method comprises the steps that the open property of a virtual discussion room can be set in response to touch operation of a first user on an open property setting control; the discussion theme of the virtual discussion room may be set in response to a touch operation of the discussion theme setting control. Thereafter, the one first display device may create a virtual discussion room in response to a touch operation of the first user with respect to the completion control.

Wherein the open nature of the virtual discussion room may be public or private. If the open attribute of the virtual discussion room is private, the third display device of the third user can join in the virtual discussion room by means of invitation of the first user when needing to join in the virtual discussion room. If the open attribute of the virtual discussion room is public, the third display device of the third user can join in the virtual discussion room by the first user invitation mode or the self-service joining application mode when needing to join in the virtual discussion room.

Optionally, if the first to third users are students, the discussion topic may include: learning at least one of experience sharing and problem communication.

In the embodiment of the application, the creator of the virtual chat room can respond to the role setting operation of the user and set the role of the user to which the display equipment added into the virtual chat room belongs. If the virtual chat room is applied to a class learning scenario, the role may be one of the following roles: learning committees and general discussions. The learning board can give the ordinary reviewer writing privileges and can be responsible for the discipline issues of the virtual discussion room. For example, a learning board may have the authority to kick a virtual discussion room out of ordinary discussions that do not comply with discipline. Wherein the default role of the creator of the virtual chat room is the committee of learning. The creator may also dismiss the virtual discussion room after the discussion is over. For example, the creator may touch a dismissal control of the virtual discussion room. The creator's display device may dismiss the virtual discussion room in response to a touch operation directed to the dismissal control.

It is to be appreciated that a virtual chat room can allow a display device of a user to join. That is, a plurality of display devices of one user cannot all join the virtual chat room at the same time.

In the embodiment of the application, the display device which is added into the virtual chat room and interacts with the server to display the online state of other display devices in the virtual chat room except the display device, wherein the online state is online or offline. Wherein, online may refer to: the display device joining the virtual chat room is currently interacting with the server, for example, may send the collected real-time video stream to the server, and may receive the real-time video stream sent by the server. Offline may mean: the display devices that join the virtual chat room do not currently interact with the server.

In the embodiment of the application, the display device can send an online notification to the server when entering the virtual chat room. The server can determine that the online state of the display device is online after receiving the online notification sent by the display device, and can send the online states of other display devices except the display device, which are located in the virtual chat room, to the display device so that the display device can display the online states.

It will be appreciated that the display device may also send device information to the server upon entering the virtual chat room. The device information may include: device type and/or device number. For example, the device information may include: device type and device identification. The device type may be one of the following types: cell-phone, panel computer, notebook computer, desktop computer and smart television. The device identifier may be a device number written by a worker into the display device before the device leaves the factory.

Step 202, determining a first target display device from the plurality of first display devices based on the first real-time video streams collected by the plurality of first display devices.

The first real-time video stream acquired by the first target display device comprises an image of the first user, namely the position of the first user is located in the acquisition range of the camera of the first target display device. The server provided by the embodiment of the application can flexibly select the first target display device from the plurality of first display devices to play the second real-time video stream based on the position of the first user, and the second real-time video stream is not continuously pushed to the fixed first display device to be displayed by the first display device, so that the flexible playing of the real-time video stream is realized.

Optionally, for each first display device, in the process of conducting a learning discussion between the first user and the second user, a scene of the real-time video stream is acquired by the camera, and the first real-time video stream acquired by the first target display device is the real-time learning video stream of the first user.

For the first real-time video stream acquired by each first display device, the server may detect whether the first real-time video stream includes an image of the first user, that is, may detect whether the position of the first user is within an acquisition range of a camera in the first display device. The image of the first user may be pre-stored by the server.

If the server detects that the first real-time video stream acquired by one first display device includes the image of the first user, the one first display device can be directly determined as the first target display device. If the server detects that the first real-time video streams collected by a plurality of candidate display devices in the plurality of first display devices all include images of the first user, the candidate display device closest to the first user in the plurality of candidate display devices may be determined as the first target display device.

The closer the distance from the user to the display device is, the more clearly the content displayed by the display device can be known, and the first target display device is the display device closest to the first user among the multiple candidate display devices, so that the first user can be ensured to clearly know the second real-time video stream played by the first target display device, and the user experience is improved.

In this embodiment of the application, after detecting that each of the first real-time video streams acquired by the multiple candidate display devices includes an image of a first user, for each of the first real-time video streams acquired by the candidate display devices, the server may use an image processing algorithm to process a video frame included in the first real-time video stream acquired by the candidate display device, so as to obtain a distance from the first user to the candidate display device, and thus obtain a distance from the first user to each of the multiple candidate display devices. The server may then determine a first target display device from the plurality of candidate display devices based on the plurality of distances.

For example, for a first real-time video stream captured by each alternative display device, the server may match the image of the first user from the video frames included in the first real-time video stream by using an image processing algorithm based on the pre-stored image of the first user. Thereafter, the server may determine a distance from the first user to the alternative display device based on a size of an image of the first user in the video frame, and pre-stored internal and external parameters of a camera of the alternative display device.

Step 203, sending the second real-time video stream to the first target display device.

And after receiving the second real-time video stream sent by the second display device and determining the first target display device, the server can send the second real-time video stream to the first target display device. Accordingly, the first target display device may play the second real-time video stream.

It can be understood that, for a scenario where the server first receives the first real-time video stream sent by one first display device and does not include the first user, and then controls each of the other first display devices to collect the first real-time video stream, the server may first push the received second real-time video stream to the one first display device for playing by the one first display device. And then, after determining the first target display device, the server can push the second real-time video stream to the first target display device so as to be played by the first target display device.

Optionally, the first display device may further display the first real-time video stream collected by itself. For example, the display screen of the first target display device includes a first sub-display area and a second sub-display area. The first display device can play the collected first real-time video stream in the first sub-display area, and can play the second real-time video stream sent by the server in the second sub-display area.

For each scene in which the first display device and the second display device also collect real-time audio streams, the server may also send the second real-time audio streams collected by the second display device to the first target display device for playing by the first target display device.

In the embodiment of the present application, the server may periodically acquire the connection state between each display device connected to the server. The connection state may be a normal connection or an abnormal connection (i.e., disconnection) disconnection. After the server receives a second real-time video stream sent by a second display device and determines a first target display device, if the connection state between the server and the first target display device is determined to be normal, the server can push the second real-time video stream to the first target display device.

And step 204, sending the first real-time video stream acquired by the first target display device to the second display device.

After the server determines the first target display device, the server may also send a first real-time video stream acquired by the first target display device to the second display device. Accordingly, the second display device may play the first real-time video stream.

Optionally, the second display device may also display the second real-time video stream captured by itself. For example, the display screen of the second target display device includes a third sub display area and a fourth sub display area. The second display device may play the captured second real-time video stream in the third sub-display area, and may play the first real-time video stream sent by the server in the fourth sub-display area.

Optionally, after determining the first target display device, if it is determined that the connection state between the server and the second display device is normal, the server may send the first real-time video stream acquired by the first target display device to the second display device.

Step 205, if the first real-time video stream collected by the first target display device does not include the image of the first user, determining a second target display device from the plurality of first display devices based on the first real-time video streams collected by the plurality of first display devices.

In an embodiment of the application, the first user may be ambulatory. If the position of the first user is not within the acquisition range of the camera of the first target display device, that is, the first real-time video stream acquired by the first target display device and detected by the server does not include the image of the first user, the server may determine, based on the first real-time video streams acquired by the plurality of first display devices, a second target display device from the plurality of first display devices.

The first real-time video stream collected by the second target display device comprises images of the first user. The server may refer to the related implementation process of step 202 from the process of determining the second target display device from the plurality of first display devices from the first real-time video streams acquired based on the plurality of first display devices, which is not described herein again in this embodiment of the application.

Step 206, sending the second real-time video stream to a second target display device.

In this embodiment of the application, after determining the second target display device from the plurality of first display devices, the server may directly send the second real-time video stream to the second target display device. Accordingly, the second target display device may play the second real-time video stream.

Or, after determining the second target display device, the server may send a switching reminding message to the second target display device. The second target display device may display the handover reminder message after receiving the handover reminder message. The switch alert message may be used to prompt the first user whether to confirm display of the second real-time video stream in the second target display device. Then, if the server receives a confirmation instruction for the switching reminding message, the server may send a second real-time video stream to a second target display device.

As can be seen from the above description of step 205 and step 206, in this embodiment of the application, when the position of the first user is within the capture range of the camera of the first target display device, the server will push the second real-time video stream captured by the second display device to the first target display device. After the position of the first user moves from the acquisition range of the camera of the first target display device to the acquisition range of the camera of the second target display device, the server can push the second real-time video stream to the second target display device. Therefore, the method provided by the embodiment of the application can flexibly switch the first display device for displaying the real-time video stream based on the change of the position of the first user, and does not adopt a certain fixed first display device to continuously display the second real-time video stream, so that the flexible display of the real-time video stream is realized. That is, the method provided in the embodiment of the present application may not be limited to one first display device in the process of displaying the second real-time video stream. Accordingly, the first user may not have to stay at one location in the room.

Optionally, the second target display device may also display the first real-time video stream acquired by the second target display device, and reference may be made to the related implementation process of the first target display device in step 203 for the process of displaying the first real-time video stream and the second real-time video stream by the second target display device, which is not described herein again in this embodiment of the present application.

For example, assume that the first user is a student a, the second user is a student B, and the student a has two first display devices, one first display device is a tablet computer 02a shown in fig. 4, and the other first display device is an intelligent television 02B shown in fig. 4.

As shown in fig. 4, firstly, the tablet computer 02a collects the first real-time video stream including the face image of the student a, so that the server can push the second real-time video stream to the tablet computer 02a for the tablet computer 02a to display. As can be seen from fig. 4, the second real-time video stream comprises face images of student B.

Afterwards, the first real-time video stream collected by the tablet computer 02a does not include the face image of the student a, and the first real-time video stream collected by the smart television 02b includes the face image of the student a, so that the server can push the second real-time video stream to the smart television 02b for the smart television 02b to display.

In the embodiment of the application, the second display device and each first display device have a whiteboard demonstration function, so that at least one of the first user and the second user can perform whiteboard demonstration through the whiteboard demonstration function, and therefore the effect of explaining for the other side is achieved. The whiteboard sharing function may refer to: one display device can share a display interface with another display device in a screen recording mode. The display screen of the first target display device may have a display area and a first whiteboard presentation area, the display area being different from the first whiteboard presentation area and the display area being capable of being used to play the second real-time video stream. Based on this, the server provided in the embodiment of the present application may further perform the following steps:

step 207, receiving a first whiteboard video stream sent by the first target display device.

In this embodiment of the application, if the first user needs to perform a presentation for the second user through the presentation whiteboard, the whiteboard presentation function of the first target display device may be enabled. After the whiteboard demonstration function of the first target display device is started, the first target display device can record a screen of the first whiteboard demonstration area to obtain a first whiteboard video stream, and the first whiteboard video stream is sent to the server. Correspondingly, the server can receive the first whiteboard video stream sent by the first target display device. At this time, the first target display device may be referred to as a whiteboard share terminal.

It will be appreciated that when the whiteboard demonstration function of the first target display device is enabled, the demonstration whiteboard may be displayed in the first whiteboard demonstration area of the display screen. In addition, in the process of recording the screen of the first whiteboard demonstration area, the first user may touch (for example, write in) the demonstration whiteboard. The first target display device may display a touch trajectory of the first user in response to a touch operation of the first user. Accordingly, the first whiteboard video stream may include the touch trajectory.

Optionally, the first whiteboard video stream may be obtained by encoding a plurality of acquired initial video frames of the first whiteboard presentation area by using a Moving Picture Experts Group (MPEG) algorithm. The first whiteboard video stream includes a plurality of target video frames in one-to-one correspondence with the plurality of initial video frames. Each target video frame of the plurality of target videos other than the first target video frame may record: the initial video frame corresponding to the target video frame is a partial image changed compared with the previous initial video frame of the initial video frame.

According to the above description, the method provided by the embodiment of the application can simultaneously send the first real-time video stream and the first whiteboard video stream to the second display device, so that the synchronization of writing and discussion is realized, and then the online discussion mode is truly restored, so that the user experience is better.

Step 208, sending the first whiteboard video stream to the second display device.

After receiving the first whiteboard video stream, the server may send the first whiteboard video stream to a second display device (i.e., a whiteboard receiving end). After receiving the first whiteboard video stream, the second display device may play the first whiteboard video stream.

Optionally, the display screen of the second display device has a second whiteboard presentation area, and the second display device may play the first whiteboard video stream in the second whiteboard presentation area.

Further, for the second display device to display the scene of the first real-time video stream captured by the first target display device, the display screen of the second display device also has a display area, which is different from the second whiteboard demonstration area, where the second display device can display the first video stream.

For example, referring to fig. 5, the first whiteboard presentation area of the first target display device 02 displays a character "3-8 × 5", and after the server transmits the first whiteboard video stream to the second display device 03, the second display device 03 may display a character "3-8 × 5" in the second whiteboard presentation area thereof.

Also, as can be seen from fig. 5, the first whiteboard rendering area of the first target display device 02 may be a left area of the display screen of the first target display device, and the display area may be a right area of the display screen. The second whiteboard presentation area of the second display device 03 may be a left area of the display screen of the second display device and the display area may be a right area of the display screen of the second display device.

And step 209, receiving a second whiteboard video stream sent by the second display device.

In this embodiment of the application, in the process of playing the first whiteboard video stream by the second display device, the second user may touch the second whiteboard presentation area in the second display device. For example, a second user may write within the second whiteboard presentation area. Correspondingly, the second display device can respond to the touch operation of the second user on the second whiteboard demonstration area, record a screen of the second whiteboard demonstration area to obtain a second whiteboard video stream, and send the acquired second whiteboard video stream to the server. Accordingly, the server may receive a second whiteboard video stream.

The second display device can also respond to the touch operation of the second user on the second whiteboard demonstration area, and display the operation track of the touch operation in the second whiteboard demonstration area. Accordingly, the second whiteboard video stream may further include the operation trajectory.

Optionally, the second display device responds to a touch operation of a second user for the second whiteboard demonstration area, and if it is determined that the second display device has a screen recording permission for recording a screen of the second whiteboard demonstration area, the screen of the second whiteboard demonstration area may be recorded.

In this embodiment of the application, if a second user needs a second display device to record a screen of a second whiteboard demonstration area in response to a touch operation for the second whiteboard demonstration area in a process of displaying a first whiteboard video stream, a screen recording permission may be applied to a creator of a virtual discussion room through the second display device. After the creator of the virtual discussion room agrees to grant the screen recording authority to the second display device, the second display device can record the screen of the second whiteboard demonstration area.

Step 210, sending the second whiteboard video stream to the first target display device.

And after receiving the second whiteboard video stream, the server can send the second whiteboard video stream to the first display device. After the first display device receives the second whiteboard video stream, the second whiteboard video stream can be played in the first whiteboard display area.

In this embodiment, as shown in fig. 6, the real-time video stream playing system provided in this embodiment may further include a fourth display device 04 of a fourth user 130, where the fourth display device 04 may be connected to the server 01. The fourth user 130 may be a guardian of the first user or the second user. For example, if the first user and the second user are both students, the fourth user may be a parent.

Optionally, the third display device 04 may be a mobile phone, a tablet, a notebook computer, or a desktop computer.

In this embodiment of the present application, the server may further store the real-time video stream collected by each display device. And if the fourth user needs to check the chat situation of the person under guardianship of the fourth user through the fourth display device, a check request can be sent to the server. The server may query an online status of a display device of an associated user associated with the fourth user after receiving the viewing request. If the server determines that the online state is online, the server may push the currently received real-time video stream sent by the display device of the associated user to the fourth display device, so as to be displayed by the fourth display device. If the server determines that the online status is offline, the server may send the stored real-time video stream collected by the display device of the associated user to the fourth display device, so as to provide the fourth display device with the stored real-time video stream.

Optionally, for each display device, the server may store a portion of the real-time video stream collected by that display device that includes the user's image. In this way, storage resources of the server can be saved.

Optionally, if the server determines that the online status is offline, a prompt message indicating that the display device of the associated user is offline may be sent to the fourth display device. And after receiving the determination instruction for the prompt message, the server may send the stored real-time video stream acquired by the display device to the fourth display device, so as to provide the fourth display device.

Therefore, the fourth user can view the chat situation of the person under guardianship of the fourth user in real time through the fourth display device, and can review the chat situation of the person under guardianship after the person under guardianship finishes the chat.

The above embodiment is an exemplary description that the method provided by the embodiment of the present application is applied to a scene where two users perform video chat. It can be understood that the method provided in the embodiment of the present application may also be applied to a scene where three or more users perform video chat, and the embodiment of the present application does not limit this.

It should be noted that, the order of the steps of the method for displaying a real-time video stream provided in the embodiment of the present application may be appropriately adjusted, and the steps may also be increased or decreased according to the situation. For example, step 205 and step 206 may be deleted as appropriate, or step 209 and step 210 may be deleted as appropriate, or step 207 to step 210 may be deleted as appropriate. Any method that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application is covered by the protection scope of the present application, and thus the detailed description thereof is omitted.

In summary, an embodiment of the present application provides a method for playing a real-time video stream, where a server may determine a first target display device from a plurality of first display devices to display a second real-time video stream, where the first real-time video stream collected by the first target display device includes an image of a first user. Therefore, the server provided by the embodiment of the application can flexibly select the first target display device from the plurality of first display devices to play the second real-time video stream based on the position of the first user, and does not continuously push the second real-time video stream to a certain fixed first display device, so that the flexible playing of the real-time video stream is realized.

The embodiment of the application provides a server, and the server is respectively connected with a plurality of first display devices of a first user and a second display device of a second user. The server includes: a processor to:

receiving a first real-time video stream acquired by a plurality of first display devices through a camera and a second real-time video stream acquired by a second display device through the camera, wherein the acquisition ranges of the cameras of the first display devices are different;

determining a first target display device from the plurality of first display devices based on a first real-time video stream acquired by the plurality of first display devices, wherein the first real-time video stream acquired by the first target display device comprises an image of a first user;

and sending a second real-time video stream to the first target display device, wherein the second real-time video stream is used for being played by the first target display device.

Optionally, the processor may be configured to:

determining a plurality of alternative display devices from the plurality of first display devices based on first real-time video streams acquired by the plurality of first display devices, wherein the first real-time video streams acquired by each alternative display device comprise images of a first user;

and determining the candidate display device closest to the first user in the plurality of candidate display devices as the first target display device.

Optionally, the processor may be configured to:

receiving a first real-time video stream acquired by one first display device in a plurality of first display devices through a camera;

if the first real-time video stream acquired by one first display device does not comprise the image of the first user, controlling each other first display device except one first display device in the plurality of first display devices to acquire the first real-time video stream through a camera;

Optionally, the processor may be configured to:

if the first real-time video stream acquired by the first target display equipment does not comprise the image of the first user, determining second target display equipment from the plurality of first display equipment based on the first real-time video streams acquired by the plurality of first display equipment, wherein the first real-time video stream acquired by the second target display equipment comprises the image of the first user;

and sending a second real-time video stream to a second target display device, wherein the second real-time video stream is used for being played by the second target display device.

Optionally, the processor may be configured to:

sending a switching reminding message to the second target display equipment, wherein the switching reminding message is used for the second target display equipment to display;

and if a confirmation instruction aiming at the switching reminding message is received, sending a second real-time video stream to a second target display device.

Optionally, the processor may be further configured to:

and sending a first real-time video stream acquired by the first target display device to a second display device, wherein the first real-time video stream is used for being played by the second display device.

Optionally, the display screen of the first target display device has a display area and a first whiteboard demonstration area, and the display area is used for playing the second real-time video stream. The processor may be further configured to:

receiving a first whiteboard video stream sent by first target display equipment, wherein the first whiteboard video stream is obtained by recording a first whiteboard demonstration area;

and sending a first whiteboard video stream to the second display device, wherein the first whiteboard video stream is used for being played by the second display device.

Optionally, the display screen of the second display device has a second whiteboard demonstration area, and the second whiteboard demonstration area is used for playing the first whiteboard video stream. The processor may be further configured to:

receiving a second whiteboard video stream sent by second display equipment, wherein the second whiteboard video stream is obtained by recording a screen of a second whiteboard demonstration area after the second display equipment receives touch operation aiming at the second whiteboard demonstration area, and an operation track of the touch operation is also displayed in the second whiteboard demonstration area;

and sending a second whiteboard video stream to the first target display device, wherein the second whiteboard video stream is used for playing in the first whiteboard demonstration area by the first target display device.

Optionally, the first real-time video stream acquired by the first target display device is a real-time learning video stream of the first user;

In summary, the present application provides a server, which is capable of determining a first target display device from a plurality of first display devices to display a second real-time video stream, where the first real-time video stream captured by the first target display device includes an image of a first user. Therefore, the server provided by the embodiment of the application can flexibly select the first target display device from the plurality of first display devices to play the second real-time video stream based on the position of the first user, and does not continuously push the second real-time video stream to a certain fixed first display device, so that the flexible playing of the real-time video stream is realized.

Embodiments of the present application provide a server, which may include a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the method for playing a real-time video stream provided in the above embodiments, such as the method shown in fig. 2 or fig. 3.

The embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored, and the computer program is loaded by a processor and executes the playing method of the real-time video stream provided in the above embodiment, for example, the method shown in fig. 2 or fig. 3.

Embodiments of the present application further provide a computer program product including instructions, which, when the computer program product runs on a server, causes the server to execute a playing method of a real-time video stream provided by the foregoing method embodiments, for example, the method shown in fig. 2 or fig. 3.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

It should be understood that reference herein to "and/or" means that there may be three relationships, for example, a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. Also, the term "at least one" in the present application means one or more, and the term "a plurality" in the present application means two or more.

The terms "first," "second," and the like in this application are used for distinguishing between similar items and items that have substantially the same function or similar functionality, and it should be understood that "first," "second," and "nth" do not have any logical or temporal dependency or limitation on the number or order of execution. For example, a first display device may be referred to as a second display device, and similarly, a second display device may be referred to as a first display device, without departing from the scope of the various described examples.

The above description is only exemplary of the present application and should not be taken as limiting the present application, and any modifications, equivalents, improvements and the like that are made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. The method for playing the real-time video stream is applied to a server, wherein the server is respectively connected with a plurality of first display devices of a first user and a second display device of a second user, the plurality of first display devices are positioned at different positions in a first room, and the second display device is positioned in a second room; the method comprises the following steps:

2. The method of claim 1, wherein determining a first target display device from the plurality of first display devices based on the first real-time video streams captured by the plurality of first display devices comprises:

3. The method of claim 1, wherein receiving the first real-time video stream captured by the plurality of first display devices via a camera comprises:

4. The method of claim 1, wherein after transmitting the second real-time video stream to the first target display device, the method further comprises:

5. The method of claim 4, wherein sending the second real-time video stream to the second target display device comprises:

6. The method of any of claims 1 to 5, further comprising:

7. The method of any of claims 1 to 5, wherein the display screen of the first target display device has a display area and a first whiteboard presentation area, the display area being used to play the second live video stream; the method further comprises the following steps:

8. The method of claim 7, wherein the display screen of the second display device has a second whiteboard presentation area for playing the first whiteboard video stream; the method further comprises the following steps:

9. The method of any of claims 1 to 5, wherein the first real-time video stream captured by the first target display device is a real-time learning video stream of the first user;

10. A server, wherein the server is connected to a plurality of first display devices of a first user and a second display device of a second user, wherein the plurality of first display devices are located at different positions in a first room and the second display device is located in a second room; the server includes: a processor; the processor is configured to: