CN110650341A - Video interaction method and related equipment - Google Patents

Video interaction method and related equipment Download PDF

Info

Publication number
CN110650341A
CN110650341A CN201910947188.3A CN201910947188A CN110650341A CN 110650341 A CN110650341 A CN 110650341A CN 201910947188 A CN201910947188 A CN 201910947188A CN 110650341 A CN110650341 A CN 110650341A
Authority
CN
China
Prior art keywords
video stream
terminal
encoded
display area
rate
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910947188.3A
Other languages
Chinese (zh)
Inventor
聂鹏程
汪碧海
杨泽渠
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Device Co Ltd
Original Assignee
Huawei Device Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Device Co Ltd filed Critical Huawei Device Co Ltd
Priority to CN201910947188.3A priority Critical patent/CN110650341A/en
Publication of CN110650341A publication Critical patent/CN110650341A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/21805—Source of audio or video content, e.g. local disk arrays enabling multiple viewpoints, e.g. using a plurality of cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/262—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists
    • H04N21/26208—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints
    • H04N21/26216—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints involving the channel capacity, e.g. network bandwidth
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
    • H04N21/2662—Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

The embodiment of the application provides a video interaction method and related equipment, wherein the method comprises the following steps: the first terminal encodes the first video stream and the second video stream respectively by using a first encoding rate and a second encoding rate and then sends the encoded first video stream and the second video stream to the second terminal; after receiving the encoded first video stream and the encoded second video stream, the second terminal decodes the encoded first video stream and the encoded second video stream by using the first decoding rate and the second decoding rate respectively, and displays the first video stream and the second video stream on a first display area and a second display area of a display screen respectively, so that the second terminal can display pictures of the two paths of video streams. Further, if the first terminal sends the first video stream and the second video stream to the second terminal through the server, the server may adaptively adjust the encoding rates of the two video streams by using an Artificial Intelligence (AI) technique. The embodiment of the application can improve the display quality of two paths of video pictures and provides an intelligent video interaction scheme.

Description

Video interaction method and related equipment
Technical Field
The present application relates to the field of communications technologies, and in particular, to a video interaction method and related devices.
Background
With the development of communication technology and the popularization of intelligent equipment, people use equipment with a camera shooting function to shoot videos and share videos to family and friends, and people become a part of daily life.
When the user encounters beauty of the scenery on the journey, the user can share the front scenery with the distant relatives and friends by turning on the mobile phone camera. However, when a user shares a landscape with one camera and wants to share the own state with another camera, the transmission effect of the video is often unsatisfactory. For example, in a tour of a certain scenic spot, a family B wants to share a landscape before the eyes and a pleasant expression when the family B sees a beautiful scene, so that the front camera and the rear camera of the mobile phone are turned on to transmit the landscape of the front camera and the rear camera at the same time. And B, turning on a large-screen television at home, and displaying two pictures shot by A on a screen by the large-screen television to enable B to be personally on the scene. Because the current video transmission method does not support sharing two videos, even if the two videos are supported to be shared, the quality of the two shared picture display pictures is low, the sharing effect is greatly reduced, and the user experience is influenced.
How to improve the display quality of two paths of videos is a hot problem for current technicians to research.
Disclosure of Invention
The embodiment of the application discloses a video interaction method and related equipment, and the display quality of two paths of videos is improved by switching encoding and decoding of the two paths of videos, so that the sharing effect of video pictures shot by two cameras is improved, and the experience of user video interaction is improved.
In a first aspect, an embodiment of the present application discloses a video interaction method, including:
the first terminal encodes a first video stream acquired by a first camera module through a first encoding module to obtain a first encoded video stream;
the first terminal encodes a second video stream acquired by a second camera module through a second encoding module to obtain a second encoded video stream; the coding rate of the first coding module is a first coding rate, the coding rate of the second coding module is a second coding rate, and the first coding rate is greater than the second coding rate;
the first terminal sends a first coded video stream and a second coded video stream to a second terminal; wherein the video stream encoded at the first encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the second encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
It can be seen that the first terminal collects the first video stream and the second video stream, and transmits the first video stream and the second video stream to the second terminal after being encoded by the first encoding rate and the second encoding rate respectively. The second terminal receives the two paths of video streams, displays the first coded video stream with a larger coding rate in a first display area of the display screen, and displays the second coded video stream in a second display area of the display screen, so that a user of the second terminal can watch the two paths of video pictures sent by the first terminal; correspondingly, because the second display area is smaller, the video stream of the second display area can properly reduce the encoding rate, so that the user experience is not influenced, and the network resources are saved; therefore, when a user shares videos, the video interaction method provided by the embodiment of the application can improve the sharing effect of the video pictures shot by the two cameras, and improve the experience of video interaction of the user.
In an optional aspect of the first aspect, after the first terminal sends the first encoded video stream and the second encoded video stream to the second terminal, the method further includes:
a first terminal receives a first adjustment request;
the first terminal adjusts the coding rate of the first coding module to the second coding rate and adjusts the coding rate of the second coding module to the first coding rate according to the first adjustment request;
the first terminal encodes the first video stream acquired by the first camera module through the first encoding module to obtain a third encoded video stream; the first terminal encodes the second video stream acquired by the second camera module through the second encoding module to obtain a fourth encoded video stream;
the first terminal sends the third encoded video stream and the fourth encoded video stream to the second terminal.
It can be seen that a user can switch the display area of the second terminal, that is, the second encoded video stream is displayed in the first display area, the first encoded video stream is displayed in the second display area, the second terminal switches the display area and then sends a first adjustment request to the first terminal, and after receiving the adjustment request, the first terminal encodes the second video stream by using a first encoding rate with higher encoding quality, and reduces the encoding rate of the first video stream, so that the video stream displayed in the first display area adopts a higher encoding rate to obtain a better display effect; therefore, by adopting the method provided by the embodiment of the application, the user can independently select the video picture to be displayed in the first display area, the flexibility and diversity of video sharing are enriched, the requirement of the user on interactivity is met, and the experience of video interaction of the user is improved.
In another optional aspect of the first aspect, after the first terminal sends the first encoded video stream and the second encoded video stream to the second terminal, the method further includes:
the first terminal receives a second adjustment request sent by the server;
the first terminal adjusts the coding rate of the first coding module to a third coding rate and adjusts the coding rate of the second coding module to a fourth coding rate according to the second adjustment request, wherein the third coding rate is greater than the fourth coding rate;
the first terminal encodes the first video stream acquired by the first camera module through the first encoding module to obtain a fifth encoded video stream; the first terminal encodes the second video stream acquired by the second camera module through the second encoding module to obtain a sixth encoded video stream; the fifth video stream and the sixth encoded video stream satisfy the network channel bandwidth indicated by the second adjustment request;
the first terminal sends a fifth coded video stream and a sixth coded video stream to the second terminal; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal.
It can be seen that the first terminal is also regulated by the server when the first video stream and the second video stream are transmitted by the server. The server adaptively controls the bandwidth of the uploading link to generate a second adjustment request by analyzing the network condition, and feeds the second adjustment request back to the first terminal, and the first terminal enables the coding code rates of the first video stream and the second video stream to meet the network channel bandwidth indicated by the first adjustment request after receiving the second adjustment request, thereby improving the stability of video transmission.
In another optional aspect of the first aspect, after the first terminal sends the third encoded video stream and the fourth encoded video stream to the second terminal, the method further includes:
the first terminal receives a second adjustment request sent by the server;
the first terminal adjusts the coding rate of the first coding module to a fourth coding rate according to the second adjustment request; adjusting the coding rate of the second coding module to a third coding rate, wherein the third coding rate is greater than a fourth coding rate;
the first terminal encodes the first video stream acquired by the first camera module through the first encoding module to obtain a seventh encoded video stream; the first terminal encodes the second video stream acquired by the second camera module through the second encoding module to obtain an eighth encoded video stream, wherein the third encoding rate is greater than the fourth encoding rate; the seventh encoded video stream and the eighth encoded video stream satisfy the network channel bandwidth indicated by the second adjustment request;
the first terminal sends the seventh coded video stream and the eighth coded video stream to the second terminal; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal.
In another optional aspect of the first aspect, after the first terminal sends the fifth encoded video stream and the sixth encoded video stream to the second terminal, the method further includes:
a first terminal receives a first adjustment request;
the first terminal adjusts the coding rate of the first coding module to a fourth coding rate and adjusts the coding rate of the second coding module to a third coding rate according to the first adjustment request, wherein the third coding rate is greater than the fourth coding rate;
the first terminal encodes the first video stream acquired by the first camera module through the first encoding module to obtain a ninth encoded video stream; the first terminal encodes the second video stream acquired by the second camera module through the second encoding module to obtain a tenth encoded video stream;
the first terminal sends a ninth coded video stream and a tenth coded video stream to the second terminal; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal.
In another optional aspect of the first aspect, after the first terminal sends the first encoded video stream and the second encoded video stream to the second terminal, the method further includes:
a first terminal receives a first audio stream sent by a second terminal;
and the first terminal plays audio according to the first audio stream.
It can be seen that, after sharing the video picture to the second terminal, the first terminal also receives the audio stream fed back by the second terminal. After the first terminal receives the audio of the second terminal, the voice sharing of the user at the second terminal can be known, the interaction of video sharing between the first terminal user and the second terminal user is realized, and the flexibility and diversity of video interaction are enriched.
In another optional aspect of the first aspect, after the first terminal encodes the first video stream captured by the first camera module through the first encoding module to obtain a first encoded video stream, the method further includes:
the first terminal displays a first video stream acquired by the first camera module in a third display area of the first display screen;
after the first terminal encodes the second video stream collected by the second camera module through the second encoding module to obtain a second encoded video stream, the method further comprises:
the first terminal displays a second video stream acquired by a second camera module in a fourth display area of the first display screen, wherein the third display area is larger than the fourth display area;
after the first terminal transmits the first encoded video stream and the second encoded video stream to the second terminal, the method further comprises:
the first terminal receives a feedback coding video stream sent by the second terminal, wherein the feedback coding video stream is a video stream obtained by coding the feedback video stream acquired by the second terminal through the third camera module;
the first terminal decodes the feedback coding video stream to obtain a feedback decoding video stream;
the first terminal displays the feedback decoding video stream in a third display area of the first display screen, and adjusts the first video stream acquired by the first camera module to be displayed in a fifth display area of the first display screen, wherein the fifth display area of the first display screen is smaller than the third display area.
After the first terminal shares the video picture with the second terminal, the second terminal can also send the video picture of the local terminal to the first terminal, so that a user of the first terminal can also view the video stream shared by the second terminal, the two-way interaction between the first terminal and the second terminal is effectively realized, the requirement of the user on the interactivity is met, the flexibility and diversity of video sharing are enriched, and the experience of video interaction of the user is improved.
In a second aspect, an embodiment of the present application provides a video interaction method, including:
the method comprises the steps that a first coded video stream and a second coded video stream sent by a first terminal are received by a second terminal, the first coded video stream is a video stream obtained by coding the first video stream collected by the first terminal at a first coding rate, the second coded video stream is a video stream obtained by coding the second video stream collected by the first terminal at a second coding rate, and the first coding rate is greater than the second coding rate;
the second terminal decodes the first coded video stream through the first decoding module to obtain a first decoded video stream, and decodes the second coded video stream through the second decoding module to obtain a second decoded video stream; the decoding rate of the first decoding module is a first decoding rate, and the decoding rate of the second decoding module is a second decoding rate;
the second terminal displays the first decoded video stream in a first display area of a second display screen, the second terminal displays the second decoded video stream of the second decoded video stream in a second display area of the second display screen, and the first display area is larger than the second display area.
In an alternative arrangement of the second aspect, after the second terminal displays the first decoded video stream encoded at the first encoding rate in the first display area of the second display screen, and the second terminal displays the second decoded video stream encoded at the second encoding rate in the second display area of the second display screen, the method further includes:
the second terminal receives a switching command of the display area, generates a first adjustment request according to the switching command and sends the first adjustment request to the first terminal;
the second terminal receives a third coded video stream and a fourth coded video stream sent by the first terminal, wherein the third coded video stream is a video stream coded by the first terminal at a second coding rate after receiving the first adjustment request, and the fourth coded video stream is a video stream coded by the first terminal at a first coding rate after receiving the first adjustment request;
the second terminal decodes the third coded video stream through the second decoding module to obtain a third decoded video stream, and decodes the fourth coded video stream through the first decoding module to obtain a fourth decoded video stream;
and the second terminal displays the fourth decoded video stream in a first display area of a second display screen, and displays the third decoded video stream in a second display area of the second display screen, wherein the first display area is larger than the second display area.
In another optional aspect of the second aspect, after the second terminal displays the first decoded video stream encoded at the first encoding rate in the first display area of the second display screen, and the second terminal displays the second decoded video stream encoded at the second encoding rate in the second display area of the second display screen, the method further includes:
the second terminal receives a fifth coded video stream and a sixth coded video stream, wherein the fifth coded video stream is a video stream which is coded and sent by the first terminal at a third coding rate after receiving the second adjustment request, and the sixth coded video stream is a video stream which is coded by the first terminal at a fourth coding rate after receiving the second adjustment request; the second adjustment request is sent to the first terminal by the server, and the fifth coded video stream and the sixth coded video stream meet the network channel bandwidth indicated by the second adjustment request;
the second terminal decodes the fifth coded video stream through the first decoding module to obtain a fifth decoded video stream, and decodes the sixth coded video stream through the second decoding module to obtain a sixth decoded video stream; the decoding rate of the first decoding module is a third decoding rate, and the decoding rate of the second decoding module is a fourth decoding rate;
and the second terminal displays the fifth decoded video stream in a first display area of a second display screen, and displays the sixth decoded video stream in a second display area of the second display screen, wherein the first display area is larger than the second display area.
In another optional scenario of the second aspect, after the second terminal receives the third encoded video stream and the fourth encoded video stream transmitted by the first terminal, the method further includes:
the second terminal receives a seventh encoded video stream and an eighth encoded video stream, the seventh encoded video stream being a video stream that the first terminal transmits by encoding at a fourth encoding rate after receiving the second adjustment request, the eighth encoded video stream being a video stream that the first terminal transmits by encoding at a third encoding rate after receiving the second adjustment request; the second adjustment request is sent to the second terminal by the server, and the seventh coded video stream and the eighth coded video stream meet the network channel bandwidth indicated by the second adjustment request;
the second terminal decodes the seventh coded video stream through the second decoding module to obtain a seventh decoded video stream, and decodes the eighth coded video stream through the first decoding module to obtain an eighth decoded video stream; the decoding rate of the first decoding module is a third decoding rate, and the decoding rate of the second decoding module is a fourth decoding rate;
and the second terminal displays the eighth decoded video stream in a first display area of a second display screen, and displays the seventh decoded video stream in a second display area of the second display screen, wherein the first display area is larger than the second display area.
In another optional scenario of the second aspect, after the second terminal receives the fifth encoded video stream and the sixth encoded video stream transmitted by the first terminal, the method further includes:
the second terminal receives a ninth encoded video stream and a tenth encoded video stream, the ninth encoded video stream being a video stream which is encoded and transmitted by the first terminal at a fourth encoding rate after receiving the first adjustment request, the tenth encoded video stream being a video stream which is encoded by the first terminal at a third encoding rate after receiving the second adjustment request;
the second terminal decodes the ninth encoded video stream through the second decoding module to obtain a ninth decoded video stream, and decodes the tenth encoded video stream through the first decoding module to obtain a tenth decoded video stream; the decoding rate of the first decoding module is a fourth decoding rate, and the decoding rate of the second decoding module is a third decoding rate;
and the second terminal displays the tenth decoded video stream in a first display area of a second display screen and displays the ninth decoded video stream in a second display area of the second display screen, wherein the first display area is larger than the second display area.
In another optional scenario of the second aspect, after the second terminal displays the fourth decoded video stream in the first display area of the second display screen, and the second terminal displays the third decoded video stream in the second display area of the second display screen, the method further includes:
the second terminal collects a first audio stream;
and the second terminal sends a first audio stream to the first terminal, and the first audio stream is used for the first terminal to play audio.
Therefore, after the second terminal receives the video stream shared by the first terminal, the second terminal can acquire the audio of the local terminal and send the audio to the first terminal. Correspondingly, after the first terminal receives the audio, the user at the second terminal can know the voice sharing of the user at the second terminal, the video sharing interaction between the first terminal and the second terminal is realized, and the flexibility and the diversity of the video interaction are enriched.
In another optional aspect of the second aspect, the first video stream is further used for the first terminal to display in a third display area of the first display screen, and the second video stream is further used for the first terminal to display in a fourth display area of the first display screen; after the second terminal displays the first decoded video stream in the first display area of the second display screen and displays the second decoded video stream in the second display area of the second display screen, the method further comprises the following steps:
the second terminal collects feedback video stream through a third camera module;
the second terminal displays the feedback video stream in a sixth display area of the second display screen, wherein the sixth display area is smaller than the first display area;
the second terminal encodes the feedback video stream to obtain a feedback encoded video stream;
and the second terminal sends a feedback coding video stream to the first terminal, wherein the feedback coding video stream is used for the first terminal to display in a third display area of the first display screen, the first video stream is also used for displaying in a fifth display area of the first display screen of the first terminal, and the fifth display area is smaller than the third display area.
In a third aspect, an embodiment of the present application provides a video interaction method, including: the method comprises the steps that a server receives a first coded video stream and a second coded video stream sent by a first terminal, the first coded video stream is a video stream obtained by coding the first video stream collected by the first terminal at a first coding rate, the second coded video stream is a video stream obtained by coding the second video stream collected by the first terminal at a second coding rate, and the first coding rate is greater than the second coding rate;
the server forwards the first coded video stream and the second coded video stream to the second terminal; wherein the video stream encoded at the first encoding rate is for display in a first display area of a second display screen of the second terminal, and the video stream encoded at the second encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
In an optional aspect of the third aspect, after the server receives the first encoded video stream and the second encoded video stream sent by the first terminal, the method further includes:
the server receives a first adjustment request sent by a second terminal;
the server forwards a first adjustment request to the first terminal, wherein the first adjustment request is used for the first terminal to adjust the coding rate;
the server receives a third coded video stream and a fourth coded video stream sent by the first terminal, wherein the third coded video stream is a video stream coded by the first terminal at a second coding rate after receiving the first adjustment request, and the fourth coded video stream is a video stream coded by the first terminal at a first coding rate after receiving the first adjustment request;
transmitting the third encoded video stream and the fourth encoded video stream to the second terminal; wherein the video stream encoded at the first encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the second encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
In another optional scenario of the third aspect, after the server receives the first encoded video stream and the second encoded video stream sent by the first terminal, the method further includes:
the server analyzes the bandwidth of the network channel to obtain a second adjustment request;
the server sends a second adjustment request to the first terminal, wherein the second adjustment request is used for the first terminal to adjust the coding rate;
the server receives a fifth coded video stream and a sixth coded video stream, wherein the fifth coded video stream is a video stream which is sent by the first terminal after receiving the second adjustment request through coding at a third coding rate, and the sixth coded video stream is a video stream which is sent by the first terminal after receiving the second adjustment request through coding at a fourth coding rate; wherein the third encoding rate is greater than the fourth encoding rate, and the fifth encoded video stream and the sixth encoded video stream satisfy the network channel bandwidth indicated by the second adjustment request;
the server forwards the fifth coded video stream and the sixth coded video stream to the second terminal; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
In another optional scenario of the third aspect, after the server receives the first encoded video stream and the second encoded video stream sent by the first terminal, the method further includes:
the server analyzes the bandwidth of the network channel to obtain a second adjustment request;
the server sends a second adjustment request to the first terminal, wherein the second adjustment request is used for the first terminal to adjust the coding rate;
the server receives a seventh coded video stream and an eighth coded video stream, wherein the seventh coded video stream is a video stream which is sent by the first terminal through coding at a fourth coding rate after receiving the second adjustment request, and the eighth coded video stream is a video stream which is sent by the first terminal through coding at a third coding rate after receiving the second adjustment request; wherein the third encoding rate is greater than the fourth encoding rate, and the seventh encoded video stream and the eighth encoded video stream satisfy the network channel bandwidth indicated by the second adjustment request;
the server forwards the seventh coded video stream and the eighth coded video stream to the second terminal; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
In another optional aspect of the third aspect, after the server receives the fifth encoded video stream and the sixth encoded video stream, the method further includes:
the server analyzes the bandwidth of the network channel to obtain a second adjustment request;
the server sends a second adjustment request to the first terminal, wherein the second adjustment request is used for the first terminal to adjust the coding rate;
the server receives a ninth encoded video stream and a tenth encoded video stream, wherein the ninth encoded video stream is a video stream which is sent by the first terminal after receiving the second adjustment request through encoding at a fourth encoding rate, and the tenth encoded video stream is a video stream which is sent by the first terminal after receiving the second adjustment request through encoding at a third encoding rate; wherein the third encoding rate is greater than the fourth encoding rate, and the fifth encoded video stream and the sixth encoded video stream satisfy the network channel bandwidth indicated by the second adjustment request;
the server forwards the ninth coded video stream and the tenth coded video stream to the second terminal; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
In another optional scenario of the third aspect, after the server receives the first encoded video stream and the second encoded video stream sent by the first terminal, the method further includes:
the server receives a first audio stream sent by a second terminal;
and the server forwards the first audio stream to the first terminal, wherein the first audio stream is used for the first terminal to play audio.
In another optional aspect of the third aspect, the first video stream is further used for the first terminal to display in a third display area of the first display screen, and the second video stream is further used for the first terminal to display in a fourth display area of the first display screen; after the server forwards the first encoded video stream and the second encoded video stream to the second terminal, the method further includes:
receiving a feedback coding video stream sent by the second terminal, wherein the feedback coding video stream is a video stream obtained by coding the feedback video stream collected by the second terminal;
and forwarding the feedback coding video stream to the first terminal, wherein the feedback video stream is used for the first terminal to display in a third display area of the first display screen, the first video stream is also used for the first terminal to display in a fifth display area of the first display screen, and the fifth display area is smaller than the third display area.
In a fourth aspect, an embodiment of the present application provides a terminal, including a memory, a processor, a transceiver, a first camera module, a second camera module, a first encoding module, and a second encoding module, where the memory stores a program, and the processor calls the program to perform the following operations:
coding a first video stream acquired by a first camera module through a first coding module to obtain a first coded video stream; coding a second video stream acquired by a second camera module through a second coding module to obtain a second coded video stream; the coding rate of the first coding module is a first coding rate, the coding rate of the second coding module is a second coding rate, and the first coding rate is greater than the second coding rate;
transmitting, by the transceiver, the first encoded video stream and the second encoded video stream to the second terminal; wherein the video stream encoded at the first encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the second encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
In an optional aspect of the fourth aspect, the processor is further configured to:
receiving, by the transceiver, a first adjustment request after transmitting the first encoded video stream and the second encoded video stream to the second terminal through the transceiver;
adjusting the coding rate of the first coding module to a second coding rate and adjusting the coding rate of the second coding module to a first coding rate according to the first adjustment request;
coding a first video stream acquired by a first camera module through a first coding module to obtain a third coded video stream; the processor is further configured to encode a second video stream acquired by the second camera module through the second encoding module to obtain a fourth encoded video stream;
the third encoded video stream and the fourth encoded video stream are transmitted to the second terminal through the transceiver.
In another optional aspect of the fourth aspect, the processor is further configured to:
after the first encoded video stream and the second encoded video stream are transmitted to the second terminal through the transceiver, receiving, through the transceiver, a second adjustment request transmitted by the server;
adjusting the coding rate of the first coding module to a third coding rate and the coding rate of the second coding module to a fourth coding rate according to the second adjustment request, wherein the third coding rate is greater than the fourth coding rate;
coding a first video stream acquired by a first camera module through a first coding module to obtain a fifth coded video stream; the processor is further configured to encode a second video stream acquired by the second camera module through the second encoding module to obtain a sixth encoded video stream; the fifth video stream and the sixth encoded video stream satisfy the network channel bandwidth indicated by the second adjustment request;
transmitting the fifth encoded video stream and the sixth encoded video stream to a second terminal through a transceiver; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal.
In another optional aspect of the fourth aspect, the processor is further configured to:
receiving, by the transceiver, a second adjustment request transmitted by the server after transmitting the first encoded video stream and the second encoded video stream to the second terminal through the transceiver;
adjusting the coding rate of the first coding module to a fourth coding rate according to the second adjustment request; adjusting the coding rate of the second coding module to a third coding rate, wherein the third coding rate is greater than a fourth coding rate;
coding a first video stream acquired by a first camera module through a first coding module to obtain a seventh coded video stream; the processor is further configured to encode a second video stream acquired by the second camera module through the second encoding module to obtain an eighth encoded video stream, where the third encoding rate is greater than the fourth encoding rate; the seventh encoded video stream and the eighth encoded video stream satisfy the network channel bandwidth indicated by the second adjustment request;
transmitting, by the transceiver, the seventh encoded video stream and the eighth encoded video stream to the second terminal; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal.
In an optional aspect of the fourth aspect, the processor is further configured to:
receiving, by the transceiver, a first adjustment request after transmitting the fifth encoded video stream and the sixth encoded video stream to the second terminal through the transceiver;
adjusting the coding rate of the first coding module to a fourth coding rate and the coding rate of the second coding module to a third coding rate according to the first adjustment request;
coding a first video stream acquired by a first camera module through a first coding module to obtain a ninth coded video stream; the processor is further configured to encode a second video stream acquired by the second camera module through the second encoding module to obtain a tenth encoded video stream; the decoding rate of the first decoding module is a third decoding rate, and the decoding rate of the second decoding module is a fourth decoding rate;
transmitting, by the transceiver, the ninth encoded video stream and the tenth encoded video stream to the second terminal; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal.
In another optional aspect of the fourth aspect, the terminal further comprises a player; the processor is further configured to:
after transmitting the first encoded video stream and the second encoded video stream to the second terminal through the transceiver, receiving a first audio stream transmitted by the second terminal through the transceiver;
playing audio according to the first audio stream by the player.
In another optional aspect of the fourth aspect, the terminal further includes a first display screen and a decoder; the processor is further configured to:
after a first video stream acquired by a first camera module is coded by a first coding module to obtain a first coded video stream, the first video stream acquired by the first camera module is displayed in a third display area of a first display screen;
after a second video stream acquired by a second camera module is coded by a second coding module to obtain a second coded video stream, the second video stream acquired by the second camera module is displayed in a fourth display area of the first display screen, and the third display area is larger than the fourth display area;
after the first video stream and the second video stream are sent to the second terminal, a feedback coding video stream sent by the second terminal is received through the transceiver, and the feedback coding video stream is a video stream obtained after the second terminal codes the feedback video stream collected by the third camera module;
decoding the feedback coding video stream through a decoder to obtain a feedback decoding video stream;
and displaying the feedback decoded video stream in a third display area of the first display screen, and adjusting the first video stream acquired by the first camera module to be displayed in a fifth display area of the first display screen, wherein the fifth display area of the first display screen is smaller than the third display area.
In a fifth aspect, an embodiment of the present application provides a terminal, including a memory, a processor, a transceiver, a first decoding module, a second decoding module, and a second display screen, where the memory stores a program, and the processor calls the program to perform the following operations:
receiving a first coded video stream and a second coded video stream sent by a first terminal through a transceiver, wherein the first coded video stream is a video stream obtained by coding the first video stream acquired by the first terminal at a first coding rate, the second coded video stream is a video stream obtained by coding the second video stream acquired by the first terminal at a second coding rate, and the first coding rate is greater than the second coding rate;
the first coding video stream is decoded by a first decoding module to obtain a first decoding video stream, and the second coding video stream is decoded by a second decoding module to obtain a second decoding video stream; the decoding rate of the first decoding module is a first decoding rate, and the decoding rate of the second decoding module is a second decoding rate;
the video stream encoded at the first encoding rate is displayed in a first display area of a second display screen, and the video stream encoded at the second encoding rate is displayed in a second display area of the second display screen, the first display area being larger than the second display area.
In an optional aspect of the fifth aspect, the processor is further configured to:
after a first decoded video stream encoded by a first encoding rate is displayed in a first display area of a second display screen, a second decoded video stream encoded by a second encoding rate is displayed in a second display area of the second display screen, receiving a switching command of the display areas through a transceiver, generating a first adjustment request according to the switching command, and transmitting the first adjustment request to a first terminal through the transceiver;
receiving a third coded video stream and a fourth coded video stream sent by the first terminal, wherein the third coded video stream is a video stream sent by the first terminal through coding at a second coding rate after receiving the first adjustment request, and the fourth coded video stream is a video stream sent by the first terminal through coding at a first coding rate after receiving the first adjustment request;
the third coded video stream is decoded by a second decoding module to obtain a third decoded video stream, and the fourth coded video stream is decoded by a first decoding module to obtain a fourth decoded video stream;
the video stream encoded at the first encoding rate is displayed in a first display area of a second display screen, and the video stream encoded at the second encoding rate is displayed in a second display area of the second display screen, the first display area being larger than the second display area.
In another optional aspect of the fifth aspect, the processor is further configured to:
receiving, by the transceiver, a fifth encoded video stream and a sixth encoded video stream after displaying the first decoded video stream in the first display area of the second display screen and displaying the second decoded video stream in the second display area of the second display screen, the fifth encoded video stream being a video stream transmitted by the first terminal through encoding at the third encoding rate after receiving the second adjustment request, and the sixth encoded video stream being a video stream transmitted by the first terminal through encoding at the fourth encoding rate after receiving the second adjustment request; the second adjustment request is sent to the first terminal by the server, and the fifth coded video stream and the sixth coded video stream meet the network channel bandwidth indicated by the second adjustment request;
the fifth coded video stream is decoded by the first decoding module to obtain a fifth decoded video stream, and the sixth coded video stream is decoded by the second decoding module to obtain a sixth decoded video stream; the decoding rate of the first decoding module is a third decoding rate, and the decoding rate of the second decoding module is a fourth decoding rate;
and displaying the video stream encoded by the third encoding rate in a first display area of the second display screen, and displaying the video stream encoded by the third encoding rate in a second display area of the second display screen, wherein the first display area is larger than the second display area.
In another optional aspect of the fifth aspect, the processor is further configured to:
after receiving the third encoded video stream and the fourth encoded video stream transmitted by the first terminal, receiving a seventh encoded video stream and an eighth encoded video stream through the transceiver, wherein the seventh encoded video stream is a video stream transmitted by the first terminal after receiving the second adjustment request through encoding at a fourth encoding rate, and the eighth encoded video stream is a video stream transmitted by the first terminal after receiving the second adjustment request through encoding at the fourth encoding rate; the second adjustment request is sent to the second terminal by the server, and the seventh coded video stream and the eighth coded video stream meet the network channel bandwidth indicated by the second adjustment request;
the seventh coded video stream is decoded by the second decoding module to obtain a seventh decoded video stream, and the eighth coded video stream is decoded by the first decoding module to obtain an eighth decoded video stream; the decoding rate of the first decoding module is a third decoding rate, and the decoding rate of the second decoding module is a fourth decoding rate;
the video stream encoded at the third encoding rate is displayed in a first display area of a second display screen, and the video stream encoded at the fourth encoding rate is displayed in a second display area of the second display screen, the first display area being larger than the second display area.
In an optional aspect of the fifth aspect, the processor is further configured to:
after receiving the fifth encoded video stream and the sixth encoded video stream through the transceiver, receiving a switching command of the display area through the transceiver, generating a first adjustment request according to the switching command, and sending the first adjustment request to the first terminal through the transceiver;
receiving a ninth encoded video stream and a tenth encoded video stream sent by the first terminal, wherein the ninth encoded video stream is a video stream sent by the first terminal through encoding at a fourth encoding rate after receiving the first adjustment request, and the tenth encoded video stream is a video stream sent by the first terminal through encoding at a third encoding rate after receiving the first adjustment request;
a ninth coded video stream is decoded by a second decoding module to obtain a third decoded video stream, and a tenth coded video stream is decoded by a first decoding module to obtain a fourth decoded video stream; the decoding rate of the first decoding module is a third decoding rate, and the decoding rate of the second decoding module is a fourth decoding rate;
the video stream encoded at the third encoding rate is displayed in a first display area of a second display screen, and the video stream encoded at the fourth encoding rate is displayed in a second display area of the second display screen, the first display area being larger than the second display area.
In another optional aspect of the fifth aspect, the terminal further includes an audio acquisition module; the processor is further configured to:
after a video stream encoded by a first encoding rate is displayed in a first display area of a second display screen, a second decoded video stream of the video stream encoded by a second encoding rate is displayed in a second display area of the second display screen, a first audio stream is captured by an audio capture module;
and sending a first audio stream to the first terminal through the transceiver, wherein the first audio stream is used for playing audio by the first terminal.
It can be seen that, after sharing the video picture to the second terminal, the first terminal also receives the audio stream fed back by the second terminal. After the first terminal receives the audio of the second terminal, the voice sharing of the user at the second terminal can be known, the interaction of video sharing between the first terminal user and the second terminal user is realized, and the flexibility and diversity of video interaction are enriched.
In another optional solution of the fifth aspect, the terminal further includes a third camera module and an encoder; the first video stream is also used for the first terminal to display in a third display area of the first display screen, and the second video stream is also used for the first terminal to display in a fourth display area of the first display screen; the processor is further configured to:
after the video stream coded by the first coding rate is displayed in a first display area of a second display screen, a second decoded video stream of the video stream coded by the second coding rate is displayed in a second display area of the second display screen, and a feedback video stream is collected by a third camera module;
displaying the feedback video stream in a sixth display area of a second display screen, wherein the sixth display area is smaller than the first display area;
coding the feedback video stream through an encoder to obtain a feedback coded video stream;
and sending a feedback coded video stream to the first terminal through the transceiver, wherein the feedback coded video stream is used for the first terminal to display in a third display area of the first display screen, the first video stream is also used for displaying in a fifth display area of the first display screen of the first terminal, and the fifth display area is smaller than the third display area.
In a sixth aspect, an embodiment of the present application provides a server, including a memory, a processor, and a transceiver, where the memory stores a program, and the processor calls the program to perform the following operations:
receiving a first coded video stream and a second coded video stream sent by a first terminal through a transceiver, wherein the first coded video stream is a video stream obtained by coding the first video stream collected by the first terminal at a first coding rate, the second coded video stream is a video stream obtained by coding the second video stream collected by the first terminal at a second coding rate, and the first coding rate is greater than the second coding rate;
forwarding, by the transceiver, the first encoded video stream and the second encoded video stream to a second terminal; wherein the video stream encoded at the first encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the second encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
In an alternative of the sixth aspect, the processor is further configured to:
after receiving a first encoded video stream and a second encoded video stream transmitted by a first terminal through a transceiver, receiving a first adjustment request transmitted by a second terminal through the transceiver;
forwarding, by the transceiver, a first adjustment request to the first terminal, the first adjustment request being for the first terminal to adjust the coding rate;
receiving, by the transceiver, a third encoded video stream and a fourth encoded video stream transmitted by the first terminal, where the third encoded video stream is a video stream encoded by the first terminal at the second encoding rate after receiving the first adjustment request, and the fourth encoded video stream is a video stream encoded by the first terminal at the first encoding rate after receiving the first adjustment request;
a processor further configured to transmit, via the transceiver, the third encoded video stream and the fourth encoded video stream to the second terminal; wherein the video stream encoded at the first encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the second encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
In another optional aspect of the sixth aspect, the processor is further configured to:
after receiving a first coded video stream and a second coded video stream sent by a first terminal through a transceiver, analyzing a network channel bandwidth to obtain a second adjustment request;
transmitting, by the transceiver, a second adjustment request to the first terminal, the second adjustment request being for the first terminal to adjust the coding rate;
receiving, by the transceiver, a fifth encoded video stream and a sixth encoded video stream, the fifth encoded video stream being a video stream that is transmitted by the first terminal after receiving the second adjustment request by encoding at the third encoding rate, the sixth encoded video stream being a video stream that is transmitted by the first terminal after receiving the second adjustment request by encoding at the fourth encoding rate; wherein the third encoding rate is greater than the fourth encoding rate, and the fifth encoded video stream and the sixth encoded video stream satisfy the network channel bandwidth indicated by the second adjustment request;
forwarding, by the transceiver, the fifth encoded video stream and the sixth encoded video stream to the second terminal; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
In another optional aspect of the sixth aspect, the processor is further configured to:
after the third coded video stream and the fourth coded video stream sent by the first terminal are received through the transceiver, analyzing network bandwidth information to obtain a second adjustment request;
transmitting, by the transceiver, a second adjustment request to the first terminal, the second adjustment request being for the first terminal to adjust the coding rate;
receiving, by the transceiver, a seventh encoded video stream and an eighth encoded video stream, the seventh encoded video stream being a video stream that the first terminal transmitted by encoding at the fourth encoding rate after receiving the second adjustment request, the eighth encoded video stream being a video stream that the first terminal transmitted by encoding at the third encoding rate after receiving the second adjustment request; wherein the third encoding rate is greater than the fourth encoding rate, and the seventh encoded video stream and the eighth encoded video stream satisfy the network channel bandwidth indicated by the second adjustment request;
forwarding, by the transceiver, the seventh encoded video stream and the eighth encoded video stream to the second terminal; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
In an alternative of the sixth aspect, the processor is further configured to:
after forwarding the fifth encoded video stream and the sixth encoded video stream to the second terminal through the transceiver, receiving, through the transceiver, a first adjustment request sent by the second terminal;
forwarding, by the transceiver, a first adjustment request to the first terminal, the first adjustment request being for the first terminal to adjust the coding rate;
receiving, by the transceiver, a ninth encoded video stream and a tenth encoded video stream transmitted by the first terminal, the ninth encoded video stream being a video stream encoded by the first terminal at a fourth encoding rate after receiving the first adjustment request, and the tenth encoded video stream being a video stream encoded by the first terminal at a third encoding rate after receiving the first adjustment request;
a processor further configured to transmit the ninth encoded video stream and the tenth encoded video stream to the second terminal through the transceiver; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
In another optional aspect of the sixth aspect, the processor is further configured to:
receiving, by the transceiver, a first audio stream transmitted by a second terminal after said forwarding the first encoded video stream and the second encoded video stream to the second terminal by the transceiver;
and forwarding the first audio stream to the first terminal through the transceiver, wherein the first audio stream is used for playing audio by the first terminal.
In another optional scenario of the sixth aspect, the first video stream is further used for the first terminal to display in a third display area of the first display screen, and the second video stream is further used for the first terminal to display in a fourth display area of the first display screen; the processor is further configured to:
after the first coded video stream and the second coded video stream are forwarded to the second terminal through the transceiver, receiving a feedback coded video stream sent by the second terminal through the transceiver, wherein the feedback coded video stream is a video stream obtained by coding the feedback video stream collected by the second terminal;
forwarding a feedback encoded video stream to the first terminal through the transceiver, wherein the feedback video stream is used for the first terminal to display in a third display area of the first display screen, the first video stream is also used for the first terminal to display in a fifth display area of the first display screen, and the fifth display area is smaller than the third display area
In a seventh aspect, an embodiment of the present application provides a terminal, including a first camera unit, a second camera unit, a first encoding unit, a second encoding unit, and a communication unit;
the first encoding unit is configured to encode a first video stream acquired by the first camera unit to obtain a first encoded video stream; the second encoding unit is configured to encode a second video stream acquired by the second camera unit to obtain a second encoded video stream; wherein, the coding rate of the first coding unit is a first coding rate, the coding rate of the second coding unit is a second coding rate, and the first coding rate is larger than the second coding rate;
the communication unit is configured to send the first encoded video stream and the second encoded video stream to the second terminal; wherein the video stream encoded at the first encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the second encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
In an optional aspect of the seventh aspect, the terminal further includes a decision unit; the communication unit is further configured to receive a first adjustment request after transmitting the first encoded video stream and the second encoded video stream to the second terminal;
the decision unit is further configured to adjust the coding rate of the first coding unit to a second coding rate and adjust the coding rate of the second coding unit to a first coding rate according to the first adjustment request;
the first encoding unit is further configured to encode the first video stream acquired by the first camera unit to obtain a third encoded video stream; the second encoding unit is further configured to encode a second video stream acquired by the second camera unit to obtain a fourth encoded video stream;
the communication unit is further configured to transmit the third encoded video stream and the fourth encoded video stream to the second terminal.
In another optional aspect of the seventh aspect, the terminal further includes a decision unit; the communication unit is further configured to receive a second adjustment request sent by the server after sending the first encoded video stream and the second encoded video stream to the second terminal;
the decision unit is further configured to adjust the coding rate of the first coding unit to a third coding rate and adjust the coding rate of the second coding unit to a fourth coding rate according to the second adjustment request, where the third coding rate is greater than the fourth coding rate;
the first encoding unit is further configured to encode the first video stream acquired by the first camera unit through the first encoding unit to obtain a fifth encoded video stream; the second encoding unit is further configured to encode a second video stream acquired by the second camera unit through the second encoding unit to obtain a sixth encoded video stream; the fifth video stream and the sixth encoded video stream satisfy the network channel bandwidth indicated by the second adjustment request;
the communication unit is further configured to send the fifth encoded video stream and the sixth encoded video stream to the second terminal; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal.
In another optional aspect of the seventh aspect, the terminal further includes a decision unit; the communication unit is further configured to receive a second adjustment request sent by the server after sending the first encoded video stream and the second encoded video stream to the second terminal;
the decision unit is further configured to adjust the coding rate of the first coding unit to a fourth coding rate according to the second adjustment request; adjusting the coding rate of the second coding unit to a third coding rate, the third coding rate being greater than a fourth coding rate;
the first encoding unit is further configured to encode the first video stream acquired by the first camera unit to obtain a seventh encoded video stream; the second encoding unit is further configured to encode a second video stream acquired by the second camera unit to obtain an eighth encoded video stream, where the third encoding rate is greater than the fourth encoding rate; the seventh encoded video stream and the eighth encoded video stream satisfy the network channel bandwidth indicated by the second adjustment request;
the communication unit is further configured to send a seventh encoded video stream and an eighth encoded video stream to the second terminal; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal.
In an optional aspect of the seventh aspect, the terminal further includes a decision unit; the communication unit is further configured to receive a first adjustment request after transmitting the first encoded video stream and the second encoded video stream to the second terminal;
the decision unit is further configured to adjust the coding rate of the first coding unit to a fourth coding rate and adjust the coding rate of the second coding unit to a third coding rate according to the first adjustment request;
the first encoding unit is further configured to encode the first video stream acquired by the first camera unit to obtain a ninth encoded video stream; the second encoding unit is further configured to encode a second video stream acquired by the second camera unit through the second encoding unit to obtain a tenth encoded video stream; the decoding rate of the first decoding unit is a third decoding rate, and the decoding rate of the second decoding unit is a fourth decoding rate;
the communication unit is further configured to send a ninth encoded video stream and a tenth encoded video stream to the second terminal; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal.
In another optional aspect of the seventh aspect, the terminal further includes an output unit; the communication unit is further configured to receive a first audio stream transmitted by the second terminal after the first encoded video stream and the second encoded video stream are transmitted to the second terminal through the communication unit;
the output unit is used for playing the audio according to the first audio stream.
In another optional solution of the seventh aspect, the terminal further includes a decision unit, a first display screen, and a decoding unit; the decision unit is further configured to display the first video stream acquired by the first camera unit in a third display area of the first display screen after the first video stream acquired by the first camera unit is encoded by the first encoding unit to obtain a first encoded video stream;
the decision unit is further configured to display the second video stream acquired by the second camera unit in a fourth display area of the first display screen after the second video stream acquired by the second camera unit is encoded by the second encoding unit to obtain a second encoded video stream, where the third display area is larger than the fourth display area;
the communication unit is further configured to receive a feedback encoded video stream sent by the second terminal after the first video stream and the second video stream are sent to the second terminal, where the feedback encoded video stream is a video stream obtained by the second terminal after the feedback video stream acquired by the third camera unit is encoded;
the decoding unit is further configured to decode the feedback encoded video stream by the decoding unit to obtain a feedback decoded video stream;
the decision unit is further configured to display the feedback decoded video stream in a third display area of the first display screen, and the processing unit is further configured to adjust the first video stream acquired by the first camera unit to be displayed in a fifth display area of the first display screen, where the fifth display area of the first display screen is smaller than the third display area.
In an eighth aspect, an embodiment of the present application provides a terminal, including a communication unit, a first decoding unit, a second decoding unit, and a second display screen;
the communication unit is configured to receive a first encoded video stream and a second encoded video stream sent by a first terminal, where the first encoded video stream is a video stream obtained by encoding the first video stream collected by the first terminal at a first encoding rate, the second encoded video stream is a video stream obtained by encoding the second video stream collected by the first terminal at a second encoding rate, and the first encoding rate is greater than the second encoding rate;
the first decoding unit is configured to decode the first encoded video stream to obtain a first decoded video stream;
the second decoding unit is configured to decode a second encoded video stream to obtain a second decoded video stream; the decoding rate of the first decoding unit is a first decoding rate, and the decoding rate of the second decoding unit is a second decoding rate;
the decision unit is configured to display the first decoded video stream in a first display area of a second display screen, and display the second decoded video stream in a second display area of the second display screen, where the first display area is larger than the second display area.
In an optional scenario of the eighth aspect, the communication unit is further configured to receive a switching command of a display area after the decision unit displays the first decoded video stream in a first display area of a second display screen and displays the second decoded video stream in a second display area of the second display screen;
the decision unit is further configured to generate a first adjustment request according to the handover command, and send the first adjustment request to the first terminal through the communication unit;
the communication unit is further configured to receive a third encoded video stream and a fourth encoded video stream sent by the first terminal, where the third encoded video stream is a video stream sent by the first terminal after receiving the first adjustment request and encoded at the second encoding rate, and the fourth encoded video stream is a video stream sent by the first terminal after receiving the first adjustment request and encoded at the first encoding rate;
the second decoding unit is further configured to obtain a third decoded video stream by decoding the third encoded video stream; the fourth decoding unit is further configured to decode the fourth encoded video stream by the first decoding unit to obtain a fourth decoded video stream;
the decision unit is further configured to display the fourth decoded video stream in a first display area of a second display screen, and display the third decoded video stream in a second display area of the second display screen, where the first display area is larger than the second display area.
In another optional aspect of the eighth aspect, the communication unit is further configured to receive a fifth encoded video stream and a sixth encoded video stream after the decision unit displays the first decoded video stream in the first display area of the second display screen and displays the second decoded video stream in the second display area of the second display screen, where the fifth encoded video stream is a video stream that is sent by the first terminal through encoding at the third encoding rate after receiving the second adjustment request, and the sixth encoded video stream is a video stream that is encoded by the first terminal through the fourth encoding rate after receiving the second adjustment request; the second adjustment request is sent to the first terminal by the server, and the fifth coded video stream and the sixth coded video stream meet the network channel bandwidth indicated by the second adjustment request;
the first decoding unit is further configured to decode a fifth encoded video stream to obtain a fifth decoded video stream; the second decoding unit is further configured to decode the sixth encoded video stream by the second decoding unit to obtain a sixth decoded video stream; the decoding rate of the first decoding unit is a third decoding rate, and the decoding rate of the second decoding unit is a fourth decoding rate;
the decision unit is further configured to display the fifth decoded video stream in a first display area of a second display screen, and display the sixth decoded video stream in a second display area of the second display screen, where the first display area is larger than the second display area.
In another optional aspect of the eighth aspect, the communication unit is further configured to receive a seventh encoded video stream and an eighth encoded video stream after receiving the third encoded video stream and the fourth encoded video stream transmitted by the first terminal, where the seventh encoded video stream is a video stream transmitted by the first terminal after receiving the second adjustment request through encoding at a fourth encoding rate, and the eighth encoded video stream is a video stream transmitted by the first terminal after receiving the second adjustment request through encoding at the fourth encoding rate; the second adjustment request is sent to the second terminal by the server, and the seventh coded video stream and the eighth coded video stream meet the network channel bandwidth indicated by the second adjustment request;
the second decoding unit is further configured to decode the seventh encoded video stream to obtain a seventh decoded video stream; the first decoding unit is further configured to decode the eighth encoded video stream by the first decoding unit to obtain an eighth decoded video stream; the decoding rate of the first decoding unit is a third decoding rate, and the decoding rate of the second decoding unit is a fourth decoding rate;
the decision unit is further configured to display the eighth decoded video stream in a first display area of a second display screen, and display the seventh decoded video stream in a second display area of the second display screen, where the first display area is larger than the second display area.
In an optional aspect of the eighth aspect, the communication unit is further configured to receive a switching command of the display area after receiving the fifth encoded video stream and the sixth encoded video stream through the communication unit;
the decision unit is further configured to generate a first adjustment request according to the handover command, and send the first adjustment request to the first terminal through the communication unit;
the above communication unit is further configured to receive a ninth encoded video stream and a tenth encoded video stream sent by the first terminal, where the ninth encoded video stream is a video stream sent by the first terminal after receiving the first adjustment request and encoded at a fourth encoding rate, and the tenth encoded video stream is a video stream sent by the first terminal after receiving the first adjustment request and encoded at a third encoding rate;
the second decoding unit is further configured to obtain a third decoded video stream by decoding the ninth encoded video stream through the second decoding unit; the first decoding unit is further configured to decode the tenth encoded video stream by the first decoding unit to obtain a fourth decoded video stream; the decoding rate of the first decoding unit is a third decoding rate, and the decoding rate of the second decoding unit is a fourth decoding rate;
the decision unit is further configured to display the tenth decoded video stream in a first display area of the second display screen, and display the ninth decoded video stream in a second display area of the second display screen, where the first display area is larger than the second display area.
In another optional aspect of the eighth aspect, the terminal further includes an audio acquisition unit; the audio acquiring unit is further configured to acquire the first audio stream after the decision unit displays the video stream encoded at the first encoding rate in the first display area of the second display screen and displays the second decoded video stream of the video stream encoded at the second encoding rate in the second display area of the second display screen;
the communication unit is further configured to send a first audio stream to the first terminal, where the first audio stream is used for the first terminal to play audio.
In another optional solution of the eighth aspect, the terminal further includes a third camera unit and an encoding unit; the first video stream is also used for the first terminal to display in a third display area of the first display screen, and the second video stream is also used for the first terminal to display in a fourth display area of the first display screen; the third camera unit is further configured to capture a feedback video stream after displaying the video stream encoded at the first encoding rate in the first display area of the second display screen and displaying the second decoded video stream of the video stream encoded at the second encoding rate in the second display area of the second display screen;
the decision unit is further configured to display the feedback video stream in a sixth display area of the second display screen, where the sixth display area is smaller than the first display area;
the encoding unit is further configured to encode the feedback video stream to obtain a feedback encoded video stream;
the communication unit is further configured to send a feedback encoded video stream to the first terminal, where the feedback encoded video stream is used for the first terminal to display in a third display area of the first display screen, and the first video stream is also used for displaying in a fifth display area of the first display screen of the first terminal, where the fifth display area is smaller than the third display area.
In a ninth aspect, an embodiment of the present application provides a server, including a communication unit,
the communication unit is configured to receive a first encoded video stream and a second encoded video stream sent by a first terminal, where the first encoded video stream is a video stream obtained by encoding the first video stream collected by the first terminal at a first encoding rate, the second encoded video stream is a video stream obtained by encoding the second video stream collected by the first terminal at a second encoding rate, and the first encoding rate is greater than the second encoding rate;
the communication unit is further configured to forward the first encoded video stream and the second encoded video stream to the second terminal; wherein the video stream encoded at the first encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the second encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
In an optional scenario of the ninth aspect, the communication unit is further configured to receive a first adjustment request sent by the second terminal after receiving the first encoded video stream and the second encoded video stream sent by the first terminal through the communication unit;
the communication unit is further configured to forward a first adjustment request to the first terminal, where the first adjustment request is used for the first terminal to adjust the coding rate;
the communication unit is further configured to receive a third encoded video stream and a fourth encoded video stream sent by the first terminal, where the third encoded video stream is a video stream encoded by the first terminal at the second encoding rate after receiving the first adjustment request, and the fourth encoded video stream is a video stream encoded by the first terminal at the first encoding rate after receiving the first adjustment request;
the communication unit is further configured to send a third encoded video stream and a fourth encoded video stream to the second terminal; wherein the video stream encoded at the first encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the second encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
In another optional aspect of the ninth aspect, the server further includes a decision unit; the decision unit is configured to analyze a network channel bandwidth to obtain a second adjustment request after the communication unit receives the first encoded video stream and the second encoded video stream sent by the first terminal;
the communication unit is further configured to send a second adjustment request to the first terminal, where the second adjustment request is used for the first terminal to adjust the coding rate;
the communication unit is further configured to receive a fifth encoded video stream and a sixth encoded video stream, where the fifth encoded video stream is a video stream that is encoded and transmitted by the first terminal at the third encoding rate after receiving the second adjustment request, and the sixth encoded video stream is a video stream that is encoded by the first terminal at the fourth encoding rate after receiving the second adjustment request; wherein the third encoding rate is greater than the fourth encoding rate, and the fifth encoded video stream and the sixth encoded video stream satisfy the network channel bandwidth indicated by the second adjustment request;
the communication unit is further configured to forward the fifth encoded video stream and the sixth encoded video stream to the second terminal; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
In another optional aspect of the ninth aspect, the server further includes a decision unit; the decision unit is configured to analyze network bandwidth information to obtain a second adjustment request after the communication unit receives a third encoded video stream and a fourth encoded video stream sent by the first terminal;
the communication unit is further configured to send a second adjustment request to the first terminal, where the second adjustment request is used for the first terminal to adjust the coding rate;
the communication unit is further configured to receive a seventh encoded video stream and an eighth encoded video stream, where the seventh encoded video stream is a video stream that is encoded and transmitted by the first terminal at the fourth encoding rate after receiving the second adjustment request, and the eighth encoded video stream is a video stream that is encoded by the first terminal at the third encoding rate after receiving the second adjustment request; wherein the third encoding rate is greater than the fourth encoding rate, and the seventh encoded video stream and the eighth encoded video stream satisfy the network channel bandwidth indicated by the second adjustment request;
the communication unit is further configured to forward the seventh encoded video stream and the eighth encoded video stream to the second terminal; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
In an optional scenario of the ninth aspect, the communication unit is further configured to receive a first adjustment request sent by the second terminal after forwarding the fifth encoded video stream and the sixth encoded video stream to the second terminal through the communication unit;
the communication unit is further configured to forward a first adjustment request to the first terminal, where the first adjustment request is used for the first terminal to adjust the coding rate;
the above-mentioned communication unit is further configured to receive a ninth encoded video stream and a tenth encoded video stream sent by the first terminal, where the ninth encoded video stream is a video stream encoded by the first terminal at a fourth encoding rate after receiving the first adjustment request, and the tenth encoded video stream is a video stream encoded by the first terminal at a third encoding rate after receiving the first adjustment request;
the communication unit is further configured to send a ninth encoded video stream and a tenth encoded video stream to the second terminal; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
In another optional scenario of the ninth aspect, the communication unit is further configured to receive a first audio stream sent by a second terminal after forwarding the first encoded video stream and the second encoded video stream to the second terminal;
the communication unit is further configured to forward a first audio stream to the first terminal, where the first audio stream is used for the first terminal to play audio.
In another optional scenario of the ninth aspect, the first video stream is further used for the first terminal to display in a third display area of the first display screen, and the second video stream is further used for the first terminal to display in a fourth display area of the first display screen; the communication unit is further configured to receive a feedback encoded video stream sent by the second terminal after forwarding the first encoded video stream and the second encoded video stream to the second terminal through the communication unit, where the feedback encoded video stream is a video stream obtained by encoding a feedback video stream acquired by the second terminal;
the communication unit is further configured to forward a feedback encoded video stream to the first terminal, where the feedback video stream is used for the first terminal to display in a third display area of the first display screen, the first video stream is also used for the first terminal to display in a fifth display area of the first display screen, and the fifth display area is smaller than the third display area
In a tenth aspect, an embodiment of the present application provides a video interaction system, where the system includes a first terminal, a second terminal, and a server, where: the first terminal is the terminal described in the first aspect or any one of possible implementation manners of the first aspect; the second terminal is the terminal described in the second aspect, or any possible implementation manner of the second aspect; the server is the server described in the third aspect or any one of possible implementation manners of the third aspect.
In an eleventh aspect, an embodiment of the present application provides a video interaction system, where the system includes a first terminal, a second terminal, and a server, where: the first terminal is the seventh aspect, or the terminal described in any possible implementation manner of the seventh aspect; the second terminal is the terminal described in the eighth aspect or any possible implementation manner of the eighth aspect; the server is the server described in the ninth aspect or any one of the possible implementation manners of the ninth aspect.
In a twelfth aspect, an embodiment of the present application provides a video interaction system, where the system includes a first terminal and a second terminal, where: the first terminal is the terminal described in the first aspect or any one of possible implementation manners of the first aspect; the second terminal is the terminal described in the second aspect, or any possible implementation manner of the second aspect.
In a thirteenth aspect, an embodiment of the present application provides a video interaction system, where the system includes a first terminal and a second terminal, where: the first terminal is the seventh aspect, or the terminal described in any possible implementation manner of the seventh aspect; the second terminal is the terminal described in the eighth aspect or any possible implementation manner of the eighth aspect.
In a fourteenth aspect, an embodiment of the present application provides a computer-readable storage medium, where a computer program is stored, and when the computer program runs on a processor, the method described in any one of the first aspect, the second aspect, and the third aspect, or an alternative of any one of the first aspect, the second aspect, and the third aspect is implemented.
In a fifteenth aspect, embodiments of the present application provide a computer program product for implementing the method described in any one of the first aspect, the second aspect, and the third aspect, or in an alternative aspect of any one of the first aspect, the second aspect, and the third aspect, when the computer program product runs on a processor.
It can be understood that the technical solutions of the second aspect to the ninth aspect are the same as the technical solution of the first aspect, and the beneficial effects may refer to the beneficial effects of the technical solution of the first aspect, and are not described herein again.
Drawings
The drawings used in the embodiments of the present application are described below.
Fig. 1 is a schematic architecture diagram of a video interaction system according to an embodiment of the present application;
fig. 2 is a schematic view of an operation scenario of a video interaction system according to an embodiment of the present application;
fig. 3 is a schematic flowchart of a video interaction method according to an embodiment of the present application;
fig. 4 is a schematic architecture diagram of a video interaction method according to an embodiment of the present application;
fig. 5 is a scene schematic diagram of a video interaction method provided in an embodiment of the present application;
fig. 6 is a schematic flowchart of another video interaction method provided in an embodiment of the present application;
fig. 7 is a schematic architecture diagram of another video interaction method provided in an embodiment of the present application;
fig. 8 is a schematic flowchart of another video interaction method provided in an embodiment of the present application;
fig. 9 is a scene schematic diagram of another video interaction method provided in an embodiment of the present application;
fig. 10 is a schematic architecture diagram of another video interaction method provided in an embodiment of the present application;
fig. 11 is a schematic architecture diagram of another video interaction method provided in an embodiment of the present application;
fig. 12 is a schematic structural diagram of a terminal according to an embodiment of the present application;
fig. 13 is a schematic structural diagram of another terminal provided in the embodiment of the present application;
fig. 14 is a schematic structural diagram of a server provided in an embodiment of the present application;
fig. 15 is a schematic structural diagram of another terminal provided in the embodiment of the present application;
fig. 16 is a schematic structural diagram of another terminal provided in the embodiment of the present application;
fig. 17 is a schematic structural diagram of another server provided in the embodiment of the present application.
Detailed Description
The embodiments of the present application will be described below with reference to the drawings.
Referring to fig. 1, fig. 1 is a schematic structural diagram of a video interaction system according to an embodiment of the present application, where the video interaction system includes a first terminal 11, a server 12, and a second terminal 13.
The first terminal 11 may include a first camera module 111, a second camera 112, an encoder 113, and a transceiver 114. Wherein the first camera module 111 and the second camera module 112 are used for capturing a first video stream MV1 and a second video stream MV 2. The encoder 113 encodes the first video stream MV1 into a first encoded video stream CMV1 and the second video stream MV2 into a second encoded video stream CMV 2. The first transmission module transmits the first encoded video stream CMV1 and the second encoded video stream CMV2 to the server 12.
The server 12 comprises a transceiver 121 for receiving the first CMV1 and the second CMV2 encoded video stream transmitted by the first terminal 11 and forwarding CMV1 and CMV2 to the second terminal 103.
The second terminal 13 includes a transceiver 131, a decoder 132, and a second display 133. After the second terminal 13 receives the first encoded video stream CMV1 and the second encoded video stream CMV2 through the transceiver 131, the decoder 132 decodes the CMV1 and the CMV2 to obtain the first decoded video stream DMV1 and the second decoded video stream DMV2, respectively, displays the first decoded video stream DMV1 in the first display area of the second display screen 133, and displays the second decoded video stream DMV2 in the second display area of the second display screen.
Optionally, the first terminal may be a mobile phone or other image capturing device, and the second terminal may be a large screen display or other display device.
The first camera module 111 may be a camera, or a camera module formed by a plurality of cameras, or a camera module coupled in the first terminal, or a camera device connected to the first terminal, which is not limited in this application. Correspondingly, the second camera module can also be a camera, or a camera module consisting of a plurality of cameras, or a camera module coupled in the first terminal, or a camera device connected with the first terminal, which is not limited in this application. The second camera module can be a front camera of the mobile phone. For example, taking the first terminal as a mobile phone as an example, the first camera may be a rear camera module of the mobile phone, and the second camera module may be a front camera module of the mobile phone.
The encoder 113 may be an encoding apparatus integrating a first encoding module and a second encoding library, and encode two video streams respectively using the first encoding module and the second encoding module, or encode different encoding units in the same encoder, or encode using different time slots, and only the manner in which the encoder integrates the first encoding module and the first encoding module to encode will be described below.
Referring to fig. 2, fig. 2 is a schematic view of a scene during operation of a video interaction system, a mobile phone terminal 201 captures a landscape picture through a rear camera, another picture is captured through a front camera, and is transmitted to a server 202, and then is forwarded to a large screen terminal by the server 202, after receiving two video pictures, the large screen terminal 203 displays the picture of the rear camera as a large screen, displays the picture of the front camera as a picture-in-picture mode, and a user of a second terminal can view two videos shared by a first terminal.
Optionally, when the first terminal shares the two paths of videos with the second terminal through the server, a video sharing request needs to be sent to the second terminal first. After the second terminal receives the video sharing request of the first terminal, the server establishes a video sharing channel between the first terminal and the second terminal. Optionally, the first terminal may be bound to the large screen end through an account via the server, for example, the server provides an account binding service, the first terminal may be bound to the second terminal after inputting an identifier of the bound second terminal, and then the first terminal sends a video sharing request to the second terminal bound thereto through the server.
Referring to fig. 3, fig. 3 is a video interaction method provided by an embodiment of the present application, which may be implemented based on the video interaction system shown in fig. 2, and the method includes, but is not limited to, the following steps:
step S301: the first terminal acquires a first video stream through the first camera module, and acquires a second video stream through the second camera module.
Specifically, the first camera module and the second camera module are functional modules capable of taking images, the first camera module can be used for acquiring the first video stream MV1, and the second camera module can be used for acquiring the second video stream MV 2. The first camera module and the second camera module collect video streams and transmit the first video stream MV1 and the second video stream MV2 to the encoder. Optionally, the first terminal may further include a microphone or other audio collecting module, and the audio collecting module collects the audio stream HA1 and transmits the audio stream HA1 to the encoder.
Optionally, the first terminal may receive a selection operation, where the selection operation indicates any one of the camera modules as the first camera module, and may default any one of the two or more camera modules as the first camera module in addition to any one of the first camera modules as the second camera module. For example, taking the first terminal as a mobile phone, the mobile phone includes a front camera and a rear camera module, the rear camera module of the mobile phone can be defined as a first camera module, and the front camera of the mobile phone is a second camera module. Step S302: and the second video stream is encoded by the second encoding module by adopting a second encoding rate to obtain a second encoded video stream.
Specifically, the first terminal may encode different video streams using different encoding rates, such as 1080P, 720P, and so on. Wherein the first encoding rate is greater than the second encoding rate. In encoding, the first video stream MV1 is encoded using a higher encoding rate to obtain a first encoded video stream CMV1, and the second encoding rate MV2 is encoded using a lower encoding rate to obtain a second encoded video stream CMV2, such that the encoding rate of the first video stream is greater than the encoding rate of the second video stream. For example, the first terminal encodes the first video stream using a 1080P coding rate through the first encoding module, and encodes the second video stream MV2 using a 720P coding rate through the second encoding module. The first encoding module and the first encoding module may be two different encoding modules, or may be different modules integrated in the same encoding apparatus, or may be the same module integrated in the same encoding apparatus.
Optionally, if the first terminal collects the audio stream HA1, the HA1 is encoded by the encoder to obtain CHA1, and the CHA1 is transmitted to the transceiver. Step S303: the first terminal transmits the first encoded video stream and the second encoded video stream.
Specifically, the first terminal receives the first encoded video stream CMV1 and the second encoded video stream CMV2 transmitted by the encoding module, transmits the first encoded video stream CMV1 to the server through the network channel satisfying the first encoding rate, and transmits the second encoded video stream CMV2 to the server through the network channel satisfying the second encoding rate. For example, referring to fig. 4, fig. 4 is a scene schematic diagram of a video interaction method provided in an embodiment of the present application, and includes a first terminal 401, a server 402, and a second terminal 403. Wherein the bandwidth of network channel 1 satisfies video transmission at a coding rate of 1080P and the bandwidth of network channel 2 satisfies video transmission at a coding rate of 720P, the first terminal transmits the first encoded video stream CMV1 to the server through network channel 1 and the second encoded video stream CMV2 to the server through network channel 2.
Optionally, the transceiver sends the encoded audio stream CHA1 to the server over a network channel that satisfies the audio transmission.
Step S304: the server forwards the first encoded video stream and the second encoded video stream.
Specifically, after receiving a first coded video stream CMV1 and a second coded video stream CMV2 sent by a first terminal, the server forwards CMV1 and CMV2 to a second terminal which performs video interaction with the first terminal. For example, referring to fig. 4, the server receives a first encoded video stream CMV1 from network channel 1, receives a second encoded video stream CMV2 from network channel 2, and transmits the first encoded video stream CMV1 to the second terminal through a network channel satisfying the encoding rate of 1080p and transmits the second encoded video stream CMV2 to the second terminal through a network channel satisfying the encoding rate of 720 p.
Optionally, the server receives the audio stream CHA1 sent by the first terminal, and forwards CHA1 to the second terminal.
Step S305: a first encoded video stream and a second encoded video stream are received.
Specifically, the second terminal receives the first video stream CMV1 and CMV2 transmitted by the first terminal and forwarded by the server, and transmits the first encoded video stream CMV1 and the second encoded video stream CMV2 to the decoding module for decoding. Optionally, the second terminal further receives the audio stream CHA1 forwarded by the server and sent by the first terminal.
Step S306: and decoding the first coded video stream by adopting a first decoding rate through a first decoding module to obtain a first decoded video stream, and decoding the second coded video stream by adopting a second decoding rate through a second decoding module to obtain a second decoded video stream.
Specifically, the second terminal may include a first decoding module and a second decoding module, which are used to parse a video stream that needs to be displayed on the display screen. After receiving the first encoded video stream CMV1 and the second encoded video stream CMV2, the decoding module of the second terminal decodes the first encoded video stream CMV1 in the first decoding module to obtain a first decoded video stream DMV1, wherein the decoding rate of the first decoding module is a first decoding rate via which the video stream encoded using the first encoding rate can be decoded. The second terminal decodes the second encoded video stream CMV2 in a second decoding module having a decoding rate that is a second decoding rate via which the video stream encoded using the second encoding rate may be decoded to obtain a second decoded video stream DMV 2. For example, a first decoding rate may decode a video stream encoded using 1080P, a second decoding rate may decode a video stream encoded using 720P, the first encoded video stream is encoded using a 1080P coding rate, the first encoded video stream is decoded using the first decoding rate, and similarly, the first encoded video stream is decoded using the second decoding rate if the first encoded video stream is encoded using a 720P coding rate. The decoding module transmits the decoded video stream to a display.
Optionally, the second terminal decodes the encoded audio stream CHA 1.
Optionally, the first decoding module and the first decoding module may be two different decoding modules, or may be different modules integrated in the same decoding apparatus, or may be the same module integrated in the same encoding apparatus.
Step S307: the first decoded video stream is displayed in a first display area, and the second decoded video stream is displayed in a second display area.
Specifically, the second terminal displays the first decoded video stream DMV1 in the first display area of the second display screen, and displays the second decoded video stream DMV2 in the second display area of the second display screen. For example, referring to fig. 4, the second terminal displays the first decoded video stream as a large screen, and displays the second decoded video stream in a picture-in-picture form at the upper left corner or the upper right corner of the video, so that the user of the second terminal can view the scene of the first video stream shot by the first terminal on the large screen and can view the scene of the second video stream at the same time. Correspondingly, because the second display area is smaller, the video stream of the second display area can properly reduce the encoding rate, so that the user experience is not influenced, and the network resources are saved. Optionally, the second terminal plays the decoded audio stream DHA1 as audio through a speaker or other audio devices, so that the user can also hear the sound shared by the first terminal when watching a scene.
Optionally, when the first terminal collects two paths of video streams, the collection mode mainly includes the following conditions:
in a first situation, the first terminal may simultaneously start to use the first camera module and the second camera module to respectively capture the first video stream and the second video stream, for example, simultaneously start the first camera module and the second camera module to capture the first video stream and the second video stream, encode the first video stream and the second video stream, and send the encoded first video stream and second video stream to the second terminal.
In the second situation, the first terminal may also acquire the first video stream through the first camera module, and then start the second camera module to acquire the first video stream when the first camera module is started. The video acquisition process is carried out in real time, the first camera module acquires a first video stream at a first moment, and then the first video stream is coded and sent to the second terminal, and the second camera module starts to acquire a second video stream at a second moment when the first camera continuously acquires the video stream and then sends the second video stream to the second terminal after the second moment, namely, the first terminal sends two paths of video streams which are acquired and coded in real time to the second terminal after the second moment. Optionally, the first terminal may also start the second camera to acquire the second video stream first, which is referred to above for detailed description and is not described here again.
In the method described in fig. 3, a first terminal collects a first video stream and a second video stream, and the first video stream and the second video stream are encoded by using a first encoding rate and a second encoding rate, respectively, and then sent to a server, and forwarded to a second terminal by the server. Correspondingly, the second terminal receives the two paths of video streams, displays the first coded video stream with higher coding rate in the first display area of the display screen, and displays the second coded video stream in the second display area of the display screen, so that a user of the second terminal can watch a plurality of video pictures sent by the first terminal; correspondingly, because the second display area is smaller, the video stream of the second display area can properly reduce the encoding rate, so that the user experience is not influenced, and the network resources are saved; therefore, when a user shares videos, the video interaction method provided by the embodiment of the application can improve the sharing effect of the video pictures shot by the two cameras, and improve the experience of video interaction of the user.
Alternatively, referring to fig. 5, the second terminal may switch the display area of the second terminal after displaying the first decoded video stream in the first display area and the second decoded video stream in the second display area. Referring to fig. 5, fig. 5 is a schematic diagram of switching a video scene according to an embodiment of the present application, and includes a second terminal 501 and a remote control terminal 502. The second terminal 501 receives a signal for switching the display area from the remote control terminal 502, displays the second encoded video stream DMV2 in the first display area of the second display screen, and correspondingly displays the first encoded video stream DMV1 in the second display area of the second display screen. Optionally, the second terminal may also receive a signal for switching the display area through a touch screen or other selection operations. After the second terminal switches the display area, because the second encoded video stream has low encoding quality and insufficient definition when displayed in a large area, a first adjustment request needs to be sent to the first terminal to adjust the encoding rate of the video stream.
Therefore, after the video interaction method shown in fig. 3, the video encoding rate of the first terminal may also be adjusted by the monitoring of the second terminal. Referring to fig. 6, fig. 6 is a video interaction method provided by an embodiment of the present application, which may be implemented based on the video interaction system shown in fig. 2, and the method includes, but is not limited to, the following steps:
step S601: and the second terminal switches the display area, displays the second decoded video stream in the first display area, displays the first decoded video stream in the second display area, generates a first adjustment request and sends the first adjustment request to the first terminal.
Specifically, the second terminal switches the display area, generates a first adjustment request according to the switched display area, and sends the first adjustment request to the first terminal. Namely, the second terminal switches the second decoded video stream to the first display area for display and switches the first decoded video stream to the second display area for display. Alternatively, the first adjustment request indicates that the encoding rate of the video stream required to be displayed in the first display region is increased while the encoding rate of the video stream required to be displayed in the second display region is decreased.
Step S602: and the server receives a first adjustment request sent by the second terminal and forwards the first adjustment request to the first terminal.
Specifically, after receiving a first adjustment request sent by the second terminal, the server forwards the first adjustment request to the first terminal performing video interaction with the second terminal.
Step S603: the first terminal receives the first adjustment request, adjusts the coding rate of the first coding module to a second coding rate, and adjusts the coding rate of the second coding module to a first coding rate.
Specifically, the first encoding module is configured to encode a first video stream captured by the first camera, and correspondingly, the second encoding module is configured to encode a second video stream captured by the second camera. After receiving the first adjustment request, the first terminal adjusts the coding rate of the first coding module to a second coding rate, and adjusts the coding rate of the second coding module to a first coding rate. For example, the default coding rate of the first coding module is 1080p coding rate, the default coding rate of the second coding module is 720p coding rate, and after receiving the first adjustment request, the coding rate of the first coding module is adjusted to 720p coding rate, and the coding rate of the second coding module is adjusted to 1080p coding rate.
Step S604: and coding the second video stream by a fourth coding module to obtain a fourth coded video stream.
Specifically, as the first terminal and the second terminal are performing video interaction, the first camera and the second camera are acquiring shot pictures in real time, after receiving the first adjustment request, the first terminal encodes the first video stream MV1 acquired by the first camera through the first encoding module to obtain a third encoded video stream CMV3, the encoding rate of the first encoding module has been adjusted to the second encoding rate, and correspondingly, the first terminal encodes the second video stream MV2 acquired by the second camera in real time through the second encoding module to obtain a fourth encoded video stream CMV4, wherein the encoding rate of the second encoding module has been adjusted to the first encoding rate. For example, the first encoding module encodes the first video stream MV1 at a coding rate of 720p to obtain a third encoded video stream CMV3, and correspondingly, the second encoding module encodes the first video stream MV2 at a coding rate of 1080p to obtain a fourth encoded video stream CMV 4.
That is, the third encoded video stream CMV3 is a video stream of the first video stream captured by the first camera encoded using the second encoding rate via the first encoding module, and accordingly, the fourth encoded video stream CMV4 is a video stream of the first video stream captured by the first camera encoded using the second encoding rate via the first encoding module. Because the first encoding rate is higher than the second encoding rate, after the second terminal switches the display area, the second terminal sends an adjustment request to the first terminal, the first terminal increases the encoding rate of the second video stream which needs to be displayed in a larger display area, and correspondingly, the encoding rate of the first video stream which needs to be displayed in a smaller display area is reduced.
Step S605: and the first terminal transmits the third coded video stream and the fourth coded video stream.
Specifically, the first terminal receives the third encoded video stream CMV3 and the fourth encoded video stream CMV4 transmitted by the encoding module, transmits the third encoded video stream CMV3 to the server through the network channel satisfying the second encoding rate, and transmits the fourth encoded video stream CMV4 to the server through the network channel satisfying the first encoding rate. For example, if the bandwidth of network channel 1 satisfies a 1080P coding rate video transmission and the bandwidth of network channel 2 satisfies a 720P coding rate video transmission, the first terminal transmits the third encoded video stream CMV3 to the server through network channel 1 and the fourth encoded video stream CMV4 to the server through network channel 2. For specific implementation, please refer to fig. 4, which is not described herein again.
Step S606: the server forwards the third encoded video stream and the fourth encoded video stream to the second terminal.
Specifically, after receiving the third coded video stream CMV3 and the fourth coded video stream CMV4 sent by the first terminal, the server forwards the CMV3 and the CMV4 to the second terminal performing video interaction with the first terminal. For example, the server receives the third encoded video stream CMV3 from network channel 1, receives the fourth encoded video stream CMV4 from network channel 2, and transmits to the second terminal through a network channel that satisfies the encoding rate of the first video stream, i.e., the third encoded video stream CMV3 is transmitted to the second terminal through a network channel that satisfies the encoding rate of 720p, and the fourth encoded video stream CMV4 is transmitted to the second terminal through a network channel that satisfies the encoding rate of 1080 p.
Step S607: the second terminal receives the third encoded video stream and the fourth encoded video stream.
Specifically, the second terminal receives the third video stream CMV3 and the fourth encoded video stream CMV4 transmitted by the first terminal and forwarded by the server, and transmits the third encoded video stream CMV3 and the fourth encoded video stream CMV4 to the decoding module for decoding.
Step S608: and the second terminal decodes the third coded video stream by the second decoding module at the second decoding rate to obtain a third decoded video stream, and decodes the fourth coded video stream by the first decoding module at the first decoding rate to obtain a fourth decoded video stream.
Specifically, the second terminal may use the first decoding module for encoding the video stream that needs to be displayed in the first display area, and correspondingly, the second encoding module for encoding the video stream that needs to be displayed in the second display area. After receiving the third encoded video stream CMV3 and the fourth encoded video stream CMV4, the decoding module decodes the third encoded video stream CMV3 in the second decoding module to obtain a third decoded video stream DMV3, wherein the second decoding module has a second decoding rate at which the video stream encoded using the second encoding rate can be decoded. The second terminal decodes the fourth encoded video stream CMV4 in a first decoding module having a decoding rate which is the first decoding rate via which the video stream encoded using the first encoding rate may be decoded to obtain a fourth decoded video stream DMV 4. For example, a first decoding rate may decode a video stream encoded using 1080P, a second decoding rate may decode a video stream encoded using 720P, a third encoded video stream encoded using a 720P coding rate, the third encoded video stream decoded using the second decoding rate, and similarly, a fourth encoded video stream decoded using the first decoding rate if the fourth encoded video stream encoded using a 1080P coding rate. The decoding module transmits the decoded video stream to a display.
Optionally, the first decoding module and the first decoding module may be two different decoding modules, or may be different modules integrated in the same decoding apparatus, or may be the same module integrated in the same encoding apparatus.
Step S609: and displaying the fourth decoded video stream in the first display area, and displaying the second decoded video stream in the second display area.
Specifically, the second terminal displays the fourth decoded video stream DMV4 in the first display area of the second display screen, and displays the third decoded video stream DMV3 in the second display area of the second display screen. The area of the first display area is larger than that of the second display area, so that a user of the second terminal can view a scene of the first video stream shot by the first terminal on a large screen and can view a scene of the second video stream at the same time. The specific display scene is shown in fig. 2, and is not described herein again.
In the method shown in fig. 6, the second terminal switches the display area and transmits a first adjustment request to the first terminal, and the first terminal transmits the encoded third encoded video stream and the encoded fourth encoded video stream to the second terminal, wherein the fourth decoded video stream DMV4 is the video stream encoded by the second video stream MV2 at the first encoding rate, and the encoding quality is improved compared to the previous first encoding rate, so that the display quality is also improved accordingly after switching to the first display area. Accordingly, the third decoded video stream DMV3 is a video stream encoded by the first video stream MV1 at the second encoding rate, and since the previous encoding quality is higher, after switching to the second display area, since the second display area is smaller, the display effect is not obviously improved and network resources are wasted by using the first encoding rate with high quality, and after adjusting the second encoding rate, network resources are saved, the forwarding pressure of the server is reduced, the computing pressure of the second terminal is reduced, and user requirements are also satisfied.
Optionally, referring to fig. 7, fig. 7 is a schematic structural diagram of another video interaction method provided in the embodiment of the present application, and includes a first terminal 701, a server 702, and a second terminal 703. When the network channel bandwidth does not meet the video stream uploading requirement or other reasons requiring adjustment of the video encoding rate, the server 702 generates a second adjustment request according to the current network channel bandwidth and sends the second adjustment request to the first terminal to adjust the video stream encoding rate of the first terminal. For example, a plurality of terminals share a video through a server, and the server can only receive 1000 video streams with an encoding rate of 1080P and 1000 video streams with an encoding rate of 720P, and when the server receives the video streams again, because the limitation of processing resources needs to reduce the bandwidth of a network channel of the video, the uplink network bandwidth which can be currently met is sent to the first terminal through the second adjustment request.
Therefore, after the video interaction method shown in fig. 3, the video encoding rate of the first terminal may also be intelligently adjusted through a link control function provided by the server. Referring to fig. 8, fig. 8 is a video interaction method provided by an embodiment of the present application, which may be implemented based on the video interaction system shown in fig. 2, and the method includes, but is not limited to, the following steps:
step S801: and the server analyzes the network channel bandwidth to obtain a second adjustment request and sends the second adjustment request to the first terminal.
Specifically, the server receives video streams uploaded by a plurality of terminals and monitors transmission links of the video streams. And after the sent first adjustment request, forwarding the first adjustment request to the first terminal which is carrying out video interaction with the second terminal. When the network channel bandwidth does not meet the video stream uploading requirement or other reasons for adjusting the video encoding rate, a second adjustment request is generated according to the current network channel bandwidth and sent to the first terminal, and the video stream encoding rate of the first terminal is adjusted.
And after receiving the second adjustment request, the first terminal adjusts the coding rate of the coding module. Optionally, when the server generates the adjustment request, the server preferentially ensures the transmission quality of the video stream displayed in the first display area. Optionally, the server may continuously optimize the effect of link adaptive control according to a plurality of calculation methods or Artificial Intelligence (AI) techniques (e.g., computer vision correlation techniques), so that the maximum utilization of network channel resources can be achieved while ensuring the quality of the video stream displayed in the first display region.
Optionally, the server may send the coding rate indicated by the second adjustment request to the second terminal, and the second terminal may correspondingly adjust the decoding rate of the decoding module after receiving the second adjustment request.
Step S802: the first terminal receives the second adjustment request, adjusts the coding rate of the first coding module to a third coding rate, and adjusts the coding rate of the second coding module to a fourth coding rate.
Specifically, after receiving the second adjustment request, the first terminal adjusts the coding rate of the first coding module to a third coding rate, and adjusts the coding rate of the second coding module to a fourth coding rate, where the third coding rate is higher than the fourth coding rate. For example, the default coding rate of the first coding module is 1080p coding rate, the default coding rate of the second coding module is 720p coding rate, and after receiving the second adjustment request, the coding rate of the first coding module is adjusted to 720p coding rate, and the coding rate of the second coding module is adjusted to 480p coding rate.
Step S803: and the first terminal obtains a fifth coded video stream through the coding of the first coding module, and obtains the fifth coded video stream through the coding of the second video stream by the second coding module.
Specifically, as the first terminal and the second terminal are performing video interaction, the first camera and the second camera are acquiring shot pictures in real time, after receiving the first adjustment request, the first terminal encodes the first video stream MV1 acquired by the first camera through the first encoding module to obtain a fifth encoded video stream CMV5, the encoding rate of the first encoding module has been adjusted to the third encoding rate, and correspondingly, the first terminal encodes the second video stream MV2 acquired by the second camera in real time through the second encoding module to obtain a sixth encoded video stream CMV6, wherein the encoding rate of the second encoding module has been adjusted to the fourth encoding rate. For example, the first encoding module encodes the first video stream MV1 at a coding rate of 720p to obtain a fifth encoded video stream CMV5, and the second encoding module encodes the second video stream MV2 at a coding rate of 480p to obtain a sixth encoded video stream CMV 6.
That is, the fifth encoded video stream CMV5 is a video stream of the first video stream captured by the first camera encoded using the third encoding rate via the first encoding module, and accordingly, the sixth encoded video stream CMV6 is a video stream of the first video stream captured by the first camera encoded using the fourth encoding rate via the first encoding module. The server monitors the change of the network channel, adaptively adjusts the encoding rate of the video stream, and can preferentially ensure the encoding quality and the transmission quality of the video displayed in the first display area when the first video stream and the second video stream are transmitted, and simultaneously reduce the calculation pressure of the server.
Alternatively, the server may monitor the channel changes of the uplink and the downlink simultaneously, and generate the second adjustment request according to the network bandwidth information.
Optionally, the server may monitor a network change condition, and when there are other network channels in the network, the analyzing and generating the second adjustment request may instruct the first terminal to switch the other network channels.
Step S804: the first terminal transmits the fifth encoded video stream and the sixth encoded video stream.
Specifically, the first terminal receives the fifth encoded video stream CMV5 and the sixth encoded video stream CMV6 transmitted by the encoding module, transmits the fifth encoded video stream CMV5 to the server through the network channel satisfying the third encoding rate, and transmits the sixth encoded video stream CMV6 to the server through the network channel satisfying the third encoding rate. For example, if the bandwidth of network channel 1 satisfies the video transmission at the 720P coding rate and the bandwidth of network channel 2 satisfies the video transmission at the 480P coding rate, the first terminal transmits the fifth encoded video stream CMV5 to the server through network channel 1 and the sixth encoded video stream CMV6 to the server through network channel 2. For a specific implementation, please refer to fig. 4, which is not described herein again.
Step S805: the server forwards the fifth encoded video stream and the sixth encoded video stream to the second terminal.
Specifically, after receiving the fifth coded video stream CMV5 and the sixth coded video stream CMV6 sent by the first terminal, the server forwards the CMV5 and the CMV6 to the second terminal performing video interaction with the first terminal. For example, the server receives the fifth encoded video stream CMV5 from network channel 1, receives the sixth encoded video stream CMV6 from network channel 2, and transmits to the second terminal through a network channel that satisfies the encoding rate of the first video stream, i.e., the fifth encoded video stream CMV5 is transmitted to the second terminal through a network channel that satisfies the encoding rate of 720p, and the sixth encoded video stream CMV6 is transmitted to the second terminal through a network channel that satisfies the encoding rate of 480 p.
Step S806: the second terminal receives the fifth encoded video stream and the sixth encoded video stream.
Specifically, the second terminal receives the fifth video stream CMV5 and the sixth encoded video stream CMV6 transmitted by the first terminal and forwarded by the server, and transmits the fifth encoded video stream CMV5 and the sixth encoded video stream CMV6 to the decoding module for decoding.
Step S607: and the second terminal decodes the fifth coded video stream by the first decoding module at the third decoding rate to obtain a fifth decoded video stream, and decodes the sixth coded video stream by the second decoding module at the fourth decoding rate to obtain a sixth decoded video stream.
Specifically, the second terminal may use the first decoding module for encoding the video stream that needs to be displayed in the first display area, and correspondingly, the second encoding module for encoding the video stream that needs to be displayed in the second display area. After receiving the fifth encoded video stream CMV5 and the sixth encoded video stream CMV6, the decoding module decodes the fifth encoded video stream CMV5 in the first decoding module to obtain a third decoded video stream DMV5, wherein the decoding rate of the first decoding module is a third decoding rate via which the video stream encoded using the third encoding rate can be decoded. The second terminal decodes the sixth encoded video stream CMV6 in a second decoding module having a decoding rate which is a fourth decoding rate via which the video stream encoded using the fourth encoding rate may be decoded to obtain a sixth decoded video stream DMV 6. For example, the third decoding rate may decode a video stream encoded using 720P, the fourth decoding rate may decode a video stream encoded using 480P, the fifth encoded video stream is encoded using a 720P coding rate, the fifth encoded video stream is decoded using the third decoding rate, and similarly, if the fourth encoded video stream is encoded using a 480P coding rate, the fourth encoded video stream is decoded using the fourth coding rate. The decoding module transmits the decoded video stream to a display.
Optionally, the first decoding module and the first decoding module may be two different decoding modules, or may be different modules integrated in the same decoding apparatus, or may be the same module integrated in the same encoding apparatus.
Step S608: and the second terminal displays the fifth decoded video stream in the first display area and displays the sixth decoded video stream in the second display area.
Specifically, the second terminal displays the fifth decoded video stream DMV5 in the first display area of the second display screen, and displays the third decoded video stream DMV6 in the second display area of the second display screen. The area of the first display area is larger than that of the second display area, so that a user of the second terminal can view a scene of the first video stream shot by the first terminal on a large screen and can view a scene of the second video stream at the same time. The specific display scene is shown in fig. 2, and is not described herein again.
In the method shown in fig. 8, the server monitors the network channel variation to generate a second adjustment request, and sends the second adjustment request to the first terminal, and the first terminal sends the encoded third encoded video stream and the encoded fourth encoded video stream with the adjusted encoding rates to the second terminal, where the fifth decoded video stream DMV5 is the video stream encoded by the first video stream MV1 at the third encoding rate, and correspondingly, the sixth decoded video stream DMV6 is the video stream encoded by the second video stream MV2 at the fourth encoding rate. The first terminal saves network resources, reduces the forwarding pressure of the server and reduces the calculation pressure of the second terminal by adjusting the encoding rates of the two paths of video streams.
In some possible implementation processes, after the user switches the display areas of the two video streams, the server still monitors the network channel change in real time. For example, when the server monitors that the bandwidth of the network channel becomes smaller, the second terminal displays the fifth decoded video stream in the first display area of the second display screen and displays the sixth video stream in the second display area of the second display screen, the server sends a second adjustment instruction to the first terminal. After the first terminal receives the second adjustment request sent by the server through the transceiver, the first video stream MV1 acquired by the first acquisition device after receiving the second adjustment request is adjusted to be coded by using a fourth coding rate through the coder to obtain a seventh coded video stream CMV7, and correspondingly, the second video stream MV2 acquired by the second acquisition device after receiving the second adjustment request is adjusted to be coded by using a third coding rate through the coder to obtain an eighth coded video stream CMV 8. And the first terminal sends the seventh coded video stream and the eighth coded video stream to the server and forwards the seventh coded video stream and the eighth coded video stream to the second terminal through the server. The second terminal decodes the seventh video stream CMV7 and the eighth video stream CMV8 using the fourth decoding rate and the third decoding rate, respectively, to obtain a seventh decoded video stream and an eighth decoded video stream, displays the eighth decoded video stream in the first display area of the second display screen, and displays the seventh decoded video stream in the second display area of the second display screen. The specific implementation steps refer to the aforementioned flow of fig. 8, and are not described herein again.
In some possible implementation processes, after the server regulates and controls the encoding rates of the two video streams of the first terminal through the adaptive link, the first terminal also receives an adjustment request for switching the display area, which is sent by the second terminal. For example, referring to the flowchart described in fig. 8, when the server monitors that the network channel bandwidth is decreased, an adjustment request is sent to the first terminal, and after the first terminal receives the adjustment request, the first video stream collected by the first camera is encoded by the first encoding module using the third encoding rate to obtain the fifth video stream CMV5 and sent to the second terminal, and accordingly, the first terminal sends the second video stream collected by the second camera to the second terminal by the second encoding module using the fourth encoding rate to obtain the sixth video stream CMV6 and sends the sixth video stream to the second terminal. And after receiving the fifth coded video stream and the sixth coded video stream, the second terminal decodes and displays the fifth coded video stream and the sixth coded video stream in the first display area and the second display area respectively. After that, the second terminal may further receive a signal for switching the display area, generate a first adjustment request, and send the first adjustment request to the first terminal, after the first terminal receives the first adjustment request, the first video stream collected by the first camera is encoded by the first encoding module using the fourth encoding rate to obtain a ninth video stream CMV9 and sent to the second terminal, and correspondingly, the first terminal sends the second video stream collected by the second camera to the second terminal, and the second video stream obtained by the second encoding module using the third encoding rate to obtain a tenth video stream CMV10 and sends the tenth video stream to the second terminal. And after receiving the ninth coded video stream and the tenth coded video stream, the second terminal decodes and displays the ninth coded video stream and the tenth coded video stream in the second display area and the first display area respectively. The specific implementation steps refer to the aforementioned flow of fig. 6, and are not described herein again.
Optionally, after the second terminal displays the first decoded video stream in the first display area of the second display screen and displays the second decoded video stream in the second display area of the second display screen, the second terminal may capture the audio stream HA2 through a microphone or an audio capture device, encode the HA2 with an encoder of the second terminal to obtain an encoded audio stream CHA2, send the encoded audio stream CHA2 to the server through the transceiver, and forward the encoded audio stream CHA2 to the first terminal through the server. After the first terminal receives the encoded audio stream CHA2 sent by the second terminal through the transceiver, the first terminal decodes the encoded audio stream by the decoder to obtain a decoded audio stream DHA2, and plays the decoded audio stream DHA2 as audio. That is to say, after sharing the video picture to the second terminal, the first terminal also receives the audio stream fed back by the second terminal. After the first terminal receives the audio of the second terminal, the voice sharing of the user at the second terminal can be known, the interaction of video sharing between the first terminal user and the second terminal user is realized, and the flexibility and diversity of video interaction are enriched.
Optionally, the first video stream captured by the first capturing device is further used for displaying in a third display area of the first display screen of the first terminal, and the second video stream captured by the second capturing device is further used for displaying in a third display area of the first display screen of the first terminal, where the third display area is larger than the fourth display area. After the second terminal displays the first decoded video stream in the first display area of the second display screen and displays the second decoded video stream in the second display area of the second display screen, the second terminal may capture the feedback video stream RMV through the third camera module.
Referring to fig. 9, fig. 9 is a schematic diagram illustrating an operation of another video interaction method according to an embodiment of the present application, and includes a first terminal 901, a server 902, a second terminal 903, and a user 904 of the second terminal. The second terminal 903 encodes the feedback video stream RMV by an encoder of the second terminal to obtain an encoded feedback video stream CRMV, and sends the encoded feedback video stream CRMV to the server 902 through the transceiver, and the server 902 forwards the encoded feedback video stream CRMV to the first terminal 901. And after receiving the coded audio stream CRMV sent by the second middle terminal through the transceiver, the first terminal decodes the coded audio stream by the decoder to obtain a feedback decoded video stream DRMV, and displays the feedback decoded video stream DRMV in a fifth display area of a third display screen of the first terminal. Optionally, the fifth display area may be an area displayed in full screen, where the first display screen does not display the pictures of the first video stream and the second video stream; the fifth display region may also be a smaller display region than the third display region for display in the third display region in picture-in-picture format.
Optionally, the second terminal may include a plurality of camera modules, and the terminal receives a selection operation of a user on a third capturing device in the plurality of camera modules, and captures the feedback video stream RMV using the third capturing device.
Referring to fig. 9, it can be seen that, after the first terminal shares the video picture with the second terminal, the second terminal can also send the video picture of the local terminal to the first terminal, so that the user of the first terminal can also view the video stream shared by the second terminal, thereby effectively realizing the two-way interaction between the first terminal and the second terminal, satisfying the requirement of the user on the interactivity, enriching the flexibility and diversity of video sharing, and improving the experience of video interaction of the user.
Referring to fig. 10, fig. 10 is a schematic structural diagram of another video interactive system according to an embodiment of the present application, where the video interactive system includes a first terminal 1001 and a second terminal 1002.
The first terminal 1001 may include a first camera module 1011, a second camera module 1012, an encoder 1013, and a transceiver 1014. The first camera module 1011 and the second camera module 1012 are used to capture a first video stream MV1 and a second video stream MV2, respectively. The encoder 1013 encodes the first video stream MV1 into a first encoded video stream CMV1 and the second video stream MV2 into a second encoded video stream CMV 2. The first transmission module transmits the first encoded video stream CMV1 and the second encoded video stream CMV2 to the second terminal 112.
The second terminal 1002 comprises a transceiver 1021, a decoder 1022, and a second display 1023. After the second terminal 1002 receives the first encoded video stream CMV1 and the second encoded video stream CMV2 through the transceiver 1021, the decoder 1022 decodes the CMV1 and the CMV2 to obtain the first decoded video stream DMV1 and the second decoded video stream DMV2, respectively, displays the first decoded video stream DMV1 in the first display area of the second display screen 1023, and displays the second decoded video stream DMV2 in the second display area of the second display screen.
Optionally, the first terminal and the second terminal are connected through a local area network or other short-distance network connection mode, for example, the first terminal and the second terminal are connected through WIFI.
Optionally, the first terminal may be a mobile phone or other image capturing device, and the second terminal may be a large screen display or other display device.
The first camera module 1012 may be a camera, or a camera module formed by a plurality of cameras, or a camera module coupled in the first terminal, or a camera device connected to the first terminal, which is not limited in this application. Correspondingly, the second camera module 1013 may also be one camera, or a camera module composed of a plurality of cameras, or a camera module coupled in the first terminal, or a camera device connected to the first terminal, which is not limited in this application. The second camera module can be a front camera of the mobile phone. For example, taking the first terminal as a mobile phone as an example, the first camera may be a rear camera module of the mobile phone, and the second camera module may be a front camera module of the mobile phone.
The encoder 1013 may be an encoding apparatus integrating a first encoding module and a second encoding library, and encode two video streams using the first encoding module and the second encoding module respectively, or encode two video streams using different encoding units in the same encoder, or encode two video streams using different time slots, and only the manner in which the encoder integrates the first encoding module and the first encoding module is described below.
Referring to fig. 11, fig. 11 is a video interaction method provided by an embodiment of the present application, which may be implemented based on the video interaction system shown in fig. 2, and the method includes, but is not limited to, the following steps:
step S1101: the first terminal acquires a first video stream through the first camera module, and acquires a second video stream through the second camera module.
Specifically, the first camera module and the second camera module are functional modules capable of taking images, the first camera module can be used for acquiring the first video stream MV1, and the second camera module can be used for acquiring the second video stream MV 2. The first camera module and the second camera module collect video streams and transmit the first video stream MV1 and the second video stream MV2 to the encoder. Optionally, the first terminal may further include a microphone or other audio collecting module, and the audio collecting module collects the audio stream HA1 and transmits the audio stream HA1 to the encoder.
Step S1102: and the second video stream is encoded by the second encoding module by adopting a second encoding rate to obtain a second encoded video stream.
Specifically, the first terminal may encode different video streams using different encoding rates, such as 1080P, 720P, and so on. Wherein the first encoding rate is greater than the second encoding rate. In encoding, the first video stream MV1 is encoded using a higher encoding rate to obtain a first encoded video stream CMV1, and the second encoding rate MV2 is encoded using a lower encoding rate to obtain a second encoded video stream CMV2, such that the encoding rate of the first video stream is greater than the encoding rate of the second video stream. For example, the first terminal encodes the first video stream using a 1080P coding rate through the first encoding module, and encodes the second video stream MV2 using a 720P coding rate through the second encoding module. The first encoding module and the first encoding module may be two different encoding modules, or may be different modules integrated in the same encoding apparatus, or may be the same module integrated in the same encoding apparatus.
Optionally, if the first terminal collects the audio stream HA1, the HA1 is encoded by the encoder to obtain CHA1, and the CHA1 is transmitted to the transceiver.
Step S1103: the first terminal transmits the first encoded video stream and the second encoded video stream.
Specifically, the first terminal receives the first encoded video stream CMV1 and the second encoded video stream CMV2 transmitted by the encoding module, and transmits the first encoded video stream CMV1 and the second encoded video stream CMV2 to the second terminal. For example, the first terminal may transmit the first encoded video stream CMV1 and the second encoded video stream CMV2 to the second terminal using WIFI.
Optionally, the transceiver sends the encoded audio stream CHA1 to the server over a network channel that satisfies the audio transmission.
Step S1104: the second terminal receives the first encoded video stream and the second encoded video stream.
Specifically, the first video stream CMV1 and CMV2 transmitted by the first terminal of the second terminal passes the first encoded video stream CMV1 and the second encoded video stream CMV2 to the decoding module for decoding. Optionally, the second terminal further receives the audio stream CHA1 forwarded by the server and sent by the first terminal.
Step S1105: and decoding the first coded video stream by adopting a first decoding rate through a first decoding module to obtain a first decoded video stream, and decoding the second coded video stream by adopting a second decoding rate through a second decoding module to obtain a second decoded video stream.
Specifically, the second terminal may include a first decoding module and a second decoding module, which are used to parse a video stream that needs to be displayed on the display screen. After receiving the first encoded video stream CMV1 and the second encoded video stream CMV2, the decoding module of the second terminal decodes the first encoded video stream CMV1 in the first decoding module to obtain a first decoded video stream DMV1, wherein the decoding rate of the first decoding module is a first decoding rate via which the video stream encoded using the first encoding rate can be decoded. The second terminal decodes the second encoded video stream CMV2 in a second decoding module having a decoding rate that is a second decoding rate via which the video stream encoded using the second encoding rate may be decoded to obtain a second decoded video stream DMV 2. For example, a first decoding rate may decode a video stream encoded using 1080P, a second decoding rate may decode a video stream encoded using 720P, the first encoded video stream is encoded using a 1080P coding rate, the first encoded video stream is decoded using the first decoding rate, and similarly, the first encoded video stream is decoded using the second decoding rate if the first encoded video stream is encoded using a 720P coding rate. The decoding module transmits the decoded video stream to a display.
Optionally, the second terminal decodes the encoded audio stream CHA 1.
Optionally, the first decoding module and the first decoding module may be two different decoding modules, or may be different modules integrated in the same decoding apparatus, or may be the same module integrated in the same encoding apparatus.
Step S1106: the second terminal displays the first decoded video stream in the first display area and the second decoded video stream in the second display area.
Specifically, the second terminal displays the first decoded video stream DMV1 in the first display area of the second display screen, and displays the second decoded video stream DMV2 in the second display area of the second display screen. For example, the second terminal displays the first decoded video stream as a large screen, and displays the second decoded video stream in a picture-in-picture form at the upper left corner or the upper right corner of the video, so that a user of the second terminal can view a scene of the first video stream shot by the first terminal on the large screen and can view a scene of the second video stream at the same time. Correspondingly, because the second display area is smaller, the video stream of the second display area can properly reduce the encoding rate, so that the user experience is not influenced, and the network resources are saved. Optionally, the second terminal plays the decoded audio stream DHA1 as audio through a speaker or other audio devices, so that the user can also hear the sound shared by the first terminal when watching a scene.
Some possible embodiments are as described in the video interaction method of fig. 3, and are not described herein again.
In the method described in fig. 11, the first terminal and the second terminal pass through the local area network, and the first terminal collects the first video stream and the second video stream, encodes the first video stream and the second video stream using the first encoding rate and the second encoding rate, respectively, and then sends the first video stream and the second video stream to the second terminal. Correspondingly, the second terminal receives the two paths of video streams, displays the first coded video stream with higher coding rate in the first display area of the display screen, and displays the second coded video stream in the second display area of the display screen, so that a user of the second terminal can watch a plurality of video pictures sent by the first terminal; correspondingly, because the second display area is smaller, the video stream of the second display area can properly reduce the encoding rate, so that the user experience is not influenced, and the calculation pressure of the second terminal is reduced; therefore, when the user shares videos, the video interaction method enables the first terminal and the second terminal to be directly connected to share two paths of videos, the sharing effect of video pictures shot by the two cameras can be improved, and the experience of video interaction of the user is improved.
The method of the embodiments of the present application is set forth above in detail and the apparatus of the embodiments of the present application is provided below.
Referring to fig. 12, fig. 12 is a schematic structural diagram of a terminal 120 according to an embodiment of the present disclosure, where the terminal 120 may include a memory 1201, a processor 1202, a transceiver 1203, an encoder 1204, a first camera module 1205, and a second camera module 1206, and details of each unit are described below. The memory 1201, the processor 1202, the transceiver 1203, the encoder 1204, the first video module 1205 and the second video module 1206 may be connected by a bus or in other manners, and the embodiment of the present application takes the bus connection as an example.
Wherein, the Memory 1201(Memory) is a Memory device in the terminal, the Memory 1201 includes but is not limited to a Random Access Memory (RAM), a read-only Memory (ROM), an erasable programmable read-only Memory (EPROM), or a portable read-only Memory (CD-ROM), and the Memory 1201 is used to provide a storage space and store programs and data, and may include but is not limited to: android system, iOS system, windows phone system, etc., which are not limited in this application. It is understood that the memory 1201 herein may include both the built-in memory of the terminal and, of course, the expansion memory supported by the terminal.
The processor 1202 may be one or more Central Processing Units (CPUs), and in the case where the processor 1202 is one CPU, the CPU may be a single-core CPU or a multi-core CPU. The processor 1202 is a computing core and a control core of the terminal, and can analyze various instructions in the terminal and process various data of the terminal, such as: processor 1202 may communicate various types of interaction data between internal structures of the terminal, and so on.
The transceiver 1203 may optionally include a standard wired communicator or interface, a wireless communicator or interface (e.g., WI-FI module, mobile communications interface, etc.), and may be controlled by the processor 1202 for transceiving data; the transceiver 1203 may also be used for transmission and interaction of signaling or instructions internal to the terminal.
The encoder 1204 integrates two encoding modules, which are a first encoding module and a second encoding module, and the two encoding modules may be two different encoders, or different modules integrated in the same encoder, or encoding modules in the same encoder.
The first camera module 1205 may be a camera with a camera function in the first terminal, or may be a camera device connected by using another connection method, for example, a high-definition camera connected by using a USB connection line.
The second camera module 1206 may be a camera with a camera function in the first terminal, or may be a camera device connected by using another connection method, for example, a high-definition camera connected by using a USB connection line.
In an embodiment of the application, the processor 1202 runs an executable computer program in the memory 1201 for performing the following operations:
coding a first video stream acquired by a first camera module through a first coding module to obtain a first coded video stream; coding a second video stream acquired by a second camera module through a second coding module to obtain a second coded video stream; the coding rate of the first coding module is a first coding rate, the coding rate of the second coding module is a second coding rate, and the first coding rate is greater than the second coding rate;
transmitting, by the transceiver, the first encoded video stream and the second encoded video stream to the second terminal; wherein the video stream encoded at the first encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the second encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
In an alternative, the processor 1202 is further configured to receive a first adjustment request through the transceiver after the first encoded video stream and the second encoded video stream are transmitted to the second terminal through the transceiver;
the processor 1202 is further configured to adjust the coding rate of the first coding module to the second coding rate and adjust the coding rate of the second coding module to the first coding rate according to the first adjustment request;
the processor 1202 is further configured to encode the first video stream acquired by the first camera module through the first encoding module to obtain a third encoded video stream; the processor is further configured to encode a second video stream acquired by the second camera module through the second encoding module to obtain a fourth encoded video stream;
the processor 1202 is further configured to transmit the third encoded video stream and the fourth encoded video stream to the second terminal through the transceiver.
In an alternative, the processor 1202 is further configured to receive, through the transceiver, a second adjustment request sent by the server after the first encoded video stream and the second encoded video stream are sent to the second terminal through the transceiver;
the processor 1202 is further configured to adjust the coding rate of the first coding module to a third coding rate and adjust the coding rate of the second coding module to a fourth coding rate according to the second adjustment request, where the third coding rate is greater than the fourth coding rate;
the processor 1202 is further configured to encode the first video stream acquired by the first camera module through the first encoding module to obtain a fifth encoded video stream; the processor is further configured to encode a second video stream acquired by the second camera module through the second encoding module to obtain a sixth encoded video stream; the fifth video stream and the sixth encoded video stream satisfy the network channel bandwidth indicated by the second adjustment request;
the processor 1202 is further configured to transmit the fifth encoded video stream and the sixth encoded video stream to a second terminal through the transceiver; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal.
In an alternative, the processor 1202 is further configured to receive, through the transceiver, a second adjustment request sent by the server after the first encoded video stream and the second encoded video stream are sent to the second terminal through the transceiver;
the processor 1202 is further configured to adjust the coding rate of the first coding module to a fourth coding rate according to the second adjustment request; adjusting the coding rate of the second coding module to a third coding rate, wherein the third coding rate is greater than a fourth coding rate;
the processor 1202 is further configured to encode the first video stream acquired by the first camera module through the first encoding module to obtain a seventh encoded video stream; the processor 1202 is further configured to encode a second video stream acquired by the second camera module through a second encoding module to obtain an eighth encoded video stream, where the third encoding rate is greater than the fourth encoding rate; the seventh encoded video stream and the eighth encoded video stream satisfy the network channel bandwidth indicated by the second adjustment request;
the processor 1202 is further configured to transmit the seventh encoded video stream and the eighth encoded video stream to the second terminal through the transceiver; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal.
In an alternative, the processor 1202 is further configured to receive a first adjustment request through the transceiver after the fifth encoded video stream and the sixth encoded video stream are transmitted to the second terminal through the transceiver;
the processor 1202 is further configured to adjust the coding rate of the first coding module to a fourth coding rate and adjust the coding rate of the second coding module to a third coding rate according to the first adjustment request;
the processor 1202 is further configured to encode the first video stream acquired by the first camera module through the first encoding module to obtain a ninth encoded video stream; the processor is further configured to encode a second video stream acquired by the second camera module through the second encoding module to obtain a tenth encoded video stream; the decoding rate of the first decoding module is a third decoding rate, and the decoding rate of the second decoding module is a fourth decoding rate;
the processor 1202 is further configured to transmit the ninth encoded video stream and the tenth encoded video stream to the second terminal through the transceiver; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal.
In an optional scheme, the terminal 120 further includes a first player 1207; the processor 1202 is further configured to receive, through the transceiver, a first audio stream transmitted by the second terminal after transmitting the first encoded video stream and the second encoded video stream to the second terminal through the transceiver;
the processor 1202 is further configured to play audio according to the first audio stream through the player.
In an optional scheme, the terminal further includes a first display 1208 and a decoder 1209; the processor 1202 is further configured to display the first video stream acquired by the first camera module in a third display area of the first display screen after the first video stream acquired by the first camera module is encoded by the first encoding module to obtain a first encoded video stream;
the processor 1202 is further configured to display the second video stream acquired by the second camera module in a fourth display area of the first display screen after the second video stream acquired by the second camera module is encoded by the second encoding module to obtain a second encoded video stream, where the third display area is larger than the fourth display area;
the processor 1202 is further configured to receive, by the transceiver, a feedback encoded video stream sent by the second terminal after sending the first video stream and the second video stream to the second terminal, where the feedback encoded video stream is a video stream obtained by encoding a feedback video stream collected by the second terminal through the third camera module;
the processor 1202 is further configured to decode the feedback encoded video stream by a decoder to obtain a feedback decoded video stream;
the processor 1202 is further configured to display the feedback decoded video stream in a third display area of the first display screen, and the processor 1202 is further configured to adjust the first video stream acquired by the first camera module to be displayed in a fifth display area of the first display screen, where the fifth display area of the first display screen is smaller than the third display area.
In an optional scheme, the terminal further includes a first audio acquisition module; the first terminal collects audio streams through the audio collection module, and the audio streams are sent to the second terminal after being coded, so that the second terminal can hear the sound sent by the first terminal.
It should be noted that the implementation of each operation may also correspond to the corresponding description of any one of the method embodiments shown in fig. 3 or the method embodiment shown in fig. 11. The terminal 120 is a first terminal in any of the method embodiments shown in fig. 3 or the method embodiments shown in fig. 11.
In the terminal 120 illustrated in fig. 12, a first terminal collects a first video stream and a second video stream, encodes the first video stream and the second video stream using a first encoding rate and a second encoding rate, respectively, and transmits the encoded first video stream and the encoded second video stream to a second terminal. The second terminal receives the two paths of video streams, displays the first coded video stream with higher coding rate in a first display area of the display screen, and displays the second coded video stream in a second display area of the display screen, so that a user of the second terminal can watch a plurality of video pictures sent by the first terminal; correspondingly, because the second display area is smaller, the video stream of the second display area can properly reduce the encoding rate, so that the user experience is not influenced, and the network resources are saved; therefore, when a user shares videos, the video interaction method provided by the embodiment of the application can improve the sharing effect of the video pictures shot by the two cameras, and improve the experience of video interaction of the user.
Referring to fig. 13, fig. 13 is a schematic structural diagram of a terminal 130 according to an embodiment of the present disclosure, where the terminal 130 may include a memory 1301, a processor 1302, a transceiver 1303, a decoder 1304, and a second display 1305, and details of each unit are described as follows. The memory 1301, the processor 1302, the transceiver 1303, the decoder 1304, and the second display screen 1305 may be connected through a bus or in other manners, and in this embodiment of the application, the connection through the bus is taken as an example.
The Memory 1301(Memory) is a Memory device in the terminal, the Memory 6001 includes, but is not limited to, a Random Access Memory (RAM), a read-only Memory (ROM), an erasable programmable read-only Memory (EPROM), or a portable read-only Memory (CD-ROM), and the Memory 1301 is used to provide a storage space for storing programs and data, and may include, but is not limited to: android system, iOS system, windows phone system, etc., which are not limited in this application. It is understood that the memory 6001 can include both the internal memory of the terminal and, of course, the expansion memory supported by the terminal.
The processor 1302 may be one or more Central Processing Units (CPUs), and in the case that the processor 1302 is one CPU, the CPU may be a single-core CPU or a multi-core CPU. The processor 1302 is a computing core and a control core of the terminal, and can analyze various instructions in the terminal and process various data of the terminal, such as: the processor 1302 may transmit various types of interaction data between the internal structures of the terminal, and so on.
The transceiver 1303 may optionally include a standard wired communicator or interface, a wireless communicator or interface (e.g., WI-FI module, mobile communication interface, etc.), which under the control of the processor 1302 may be used for transceiving data; the transceiver 1303 can also be used for transmission and interaction of signaling or instructions internal to the terminal.
The decoder 1304 integrates two decoding modules, namely a first decoding module and a second decoding module, which may be two different decoders, or different modules integrated in the same decoder, or a decoding module in the same decoder.
In an embodiment of the application, the processor 1302 runs an executable computer program in the memory 1301 for performing the following operations:
receiving a first coded video stream and a second coded video stream sent by a first terminal through a transceiver, wherein the first coded video stream is a video stream obtained by coding the first video stream acquired by the first terminal at a first coding rate, the second coded video stream is a video stream obtained by coding the second video stream acquired by the first terminal at a second coding rate, and the first coding rate is greater than the second coding rate;
the first coding video stream is decoded by a first decoding module to obtain a first decoding video stream, and the second coding video stream is decoded by a second decoding module at a second decoding rate to obtain a second decoding video stream; the decoding rate of the first decoding module is a first decoding rate, and the decoding rate of the second decoding module is a second decoding rate;
the video stream encoded at the first encoding rate is displayed in a first display area of a second display screen, and the video stream encoded at the second encoding rate is displayed in a second display area of the second display screen, the first display area being larger than the second display area.
In one possible implementation, the processor 1302 is further configured to receive, by the transceiver, a switch command of a display area after displaying the first decoded video stream encoded at the first encoding rate in the first display area of the second display screen and displaying the second decoded video stream encoded at the second encoding rate in the second display area of the second display screen, generate a first adjustment request according to the switch command, and send, by the transceiver, the first adjustment request to the first terminal;
the processor 1302 is further configured to receive a third encoded video stream and a fourth encoded video stream sent by the first terminal, where the third encoded video stream is a video stream sent by the first terminal after receiving the first adjustment request and encoded at the second encoding rate, and the fourth encoded video stream is a video stream sent by the first terminal after receiving the first adjustment request and encoded at the first encoding rate;
the processor 1302 is further configured to obtain a third decoded video stream by decoding the third encoded video stream through the second decoding module, and obtain a fourth decoded video stream by decoding the fourth encoded video stream through the first decoding module;
the processor 1302 is further configured to display the video stream encoded at the first encoding rate in a first display area of a second display screen, and display the video stream encoded at the second encoding rate in a second display area of the second display screen, where the first display area is larger than the second display area.
In a possible implementation, the processor 1302 is further configured to receive, by the transceiver, a fifth encoded video stream and a sixth encoded video stream after displaying the first decoded video stream in the first display area of the second display screen and displaying the second decoded video stream in the second display area of the second display screen, where the fifth encoded video stream is a video stream that is transmitted by the first terminal through the third encoding rate after receiving the second adjustment request, and the sixth encoded video stream is a video stream that is encoded by the first terminal through the fourth encoding rate after receiving the second adjustment request; the second adjustment request is sent to the first terminal by the server, and the fifth coded video stream and the sixth coded video stream meet the network channel bandwidth indicated by the second adjustment request;
the processor 1302 is further configured to decode the fifth encoded video stream by the first decoding module to obtain a fifth decoded video stream, and decode the sixth encoded video stream by the second decoding module to obtain a sixth decoded video stream; the decoding rate of the first decoding module is a third decoding rate, and the decoding rate of the second decoding module is a fourth decoding rate;
the processor 1302 is further configured to display the video stream encoded at the third encoding rate in a first display area of a second display screen, and display the video stream encoded at the third encoding rate in a second display area of the second display screen, where the first display area is larger than the second display area.
In an optional scheme, the processor 1302 is further configured to, after receiving the third encoded video stream and the fourth encoded video stream sent by the first terminal, further include:
the processor 1302 is further configured to receive, through the transceiver, a seventh encoded video stream and an eighth encoded video stream, where the seventh encoded video stream is a video stream that is encoded and transmitted by the first terminal at a fourth encoding rate after receiving the second adjustment request, and the eighth encoded video stream is a video stream that is encoded by the first terminal at the fourth encoding rate after receiving the second adjustment request; the second adjustment request is sent to the second terminal by the server, and the seventh coded video stream and the eighth coded video stream meet the network channel bandwidth indicated by the second adjustment request;
the processor 1302 is further configured to decode the seventh encoded video stream by the second decoding module to obtain a seventh decoded video stream, and decode the eighth encoded video stream by the first decoding module to obtain an eighth decoded video stream; the decoding rate of the first decoding module is a third decoding rate, and the decoding rate of the second decoding module is a fourth decoding rate;
the processor 1302 is further configured to display the video stream encoded at the third encoding rate in a first display area of a second display screen, and display the video stream encoded at the fourth encoding rate in a second display area of the second display screen, where the first display area is larger than the second display area.
In a possible implementation, the processor 1302 is further configured to receive, through the transceiver, a switching command for the display area after receiving the fifth encoded video stream and the sixth encoded video stream, generate a first adjustment request according to the switching command, and send the first adjustment request to the first terminal through the transceiver;
the processor 1302 is further configured to receive a ninth encoded video stream and a tenth encoded video stream sent by the first terminal, where the ninth encoded video stream is a video stream sent by the first terminal after receiving the first adjustment request and encoded at a fourth encoding rate, and the tenth encoded video stream is a video stream sent by the first terminal after receiving the first adjustment request and encoded at a third encoding rate;
the processor 1302 is further configured to obtain a third decoded video stream by decoding the ninth encoded video stream through the second decoding module, and obtain a fourth decoded video stream by decoding the tenth encoded video stream through the first decoding module; the decoding rate of the first decoding module is a third decoding rate, and the decoding rate of the second decoding module is a fourth decoding rate;
the processor 1302 is further configured to display the video stream encoded at the third encoding rate in a first display area of a second display screen, and display the video stream encoded at the fourth encoding rate in a second display area of the second display screen, where the first display area is larger than the second display area.
In a possible embodiment, the terminal 130 further includes a second audio capture module 1306; the processor 1302 is further configured to capture a first audio stream by the audio capture module after displaying the video stream encoded at the first encoding rate in the first display area of the second display screen and displaying the second decoded video stream of the video stream encoded at the second encoding rate in the second display area of the second display screen;
the processor 1302 is further configured to send a first audio stream to the first terminal through the transceiver, where the first audio stream is used for the first terminal to play audio.
In a possible embodiment, the terminal further comprises a third camera module 1307 and an encoder 1308; the first video stream is also used for the first terminal to display in a third display area of the first display screen, and the second video stream is also used for the first terminal to display in a fourth display area of the first display screen; the processor 1302 is further configured to capture a feedback video stream through the third camera module after displaying the video stream encoded at the first encoding rate in the first display area of the second display screen and displaying the second decoded video stream of the video stream encoded at the second encoding rate in the second display area of the second display screen;
the processor 1302 is further configured to display the feedback video stream in a sixth display area of the second display screen, where the sixth display area is smaller than the first display area;
the processor 1302 is further configured to encode the feedback video stream by an encoder to obtain a feedback encoded video stream;
the processor 1302 is further configured to send a feedback encoded video stream to the first terminal through the transceiver, where the feedback encoded video stream is used for the first terminal to display in a third display area of the first display screen, and the first video stream is also used for displaying in a fifth display area of the first display screen of the first terminal, and the fifth display area is smaller than the third display area.
The third camera module 1307 may be a camera with a camera function in the second terminal, or may be an image pickup device connected by using another connection method, for example, a high-definition camera connected by using a USB connection line.
After the second terminal receives the video picture shared by the first terminal, the second terminal collects the video picture through the camera module of the second terminal and sends the video picture to the first terminal, so that the user of the first terminal can also view the video stream shared by the second terminal, the two-way interaction between the first terminal and the second terminal is effectively realized, the requirement of the user on the interactivity is met, the flexibility and diversity of video sharing are enriched, and the experience of video interaction of the user is improved.
It should be noted that the implementation of each operation may also correspond to the corresponding description of any one of the method embodiments shown in fig. 3 or the method embodiment shown in fig. 11. The terminal 13 is a second terminal in any of the method embodiments shown in fig. 3 or fig. 11.
In the terminal 130 illustrated in fig. 13, a first terminal collects a first video stream and a second video stream, encodes the first video stream and the second video stream using a first encoding rate and a second encoding rate, respectively, and transmits the encoded first video stream and the encoded second video stream to a second terminal. The second terminal receives the two paths of video streams, displays the first coded video stream with higher coding rate in a first display area of the display screen, and displays the second coded video stream in a second display area of the display screen, so that a user of the second terminal can watch a plurality of video pictures sent by the first terminal; correspondingly, because the second display area is smaller, the video stream of the second display area can properly reduce the encoding rate, so that the user experience is not influenced, and the network resources are saved; therefore, when a user shares videos, the video interaction method provided by the embodiment of the application can improve the sharing effect of the video pictures shot by the two cameras, and improve the experience of video interaction of the user.
Referring to fig. 14, fig. 14 is a schematic structural diagram of a server 80 according to an embodiment of the present application, wherein the terminal 140 may include a memory 1401, a processor 1402, and a transceiver 1403, and the details of each unit are described below. The memory 1401, the processor 1402, and the transceiver 1403 may be connected by a bus or other means, and the embodiment of the present application is exemplified by being connected by a bus.
The Memory 1401(Memory) is a Memory device in the terminal, and the Memory 1401 includes, but is not limited to, a Random Access Memory (RAM), a read-only Memory (ROM), an erasable programmable read-only Memory (EPROM), or a portable read-only Memory (CD-ROM), and is used to provide a storage space and store programs and data, and may include, but is not limited to: android system, iOS system, windows phone system, etc., which are not limited in this application. It is understood that the memory 6001 can include both the internal memory of the terminal and, of course, the expansion memory supported by the terminal.
The processor 1402 may be one or more Central Processing Units (CPUs), and in the case that the processor 1402 is one CPU, the CPU may be a single-core CPU or a multi-core CPU. The processor 1402 is a computing core and a control core of the terminal, and can analyze various instructions in the terminal and process various data of the terminal, such as: the processor 1402 may transmit various types of interaction data between internal structures of the terminal, and the like.
Transceiver 1403 optionally may include a standard wired communicator or interface, a wireless communicator or interface (e.g., WI-FI module, mobile communications interface, etc.), which under the control of processor 1402 may be used for transceiving data; the transceiver 1403 may also be used for transmission and interaction of terminal internal signaling or instructions.
In an embodiment of the present application, the processor 1402 runs an executable computer program in the memory 1401 for performing the following operations:
receiving a first coded video stream and a second coded video stream sent by a first terminal through a transceiver, wherein the first coded video stream is a video stream obtained by coding the first video stream collected by the first terminal at a first coding rate, the second coded video stream is a video stream obtained by coding the second video stream collected by the first terminal at a second coding rate, and the first coding rate is greater than the second coding rate;
forwarding, by the transceiver, the first encoded video stream and the second encoded video stream to a second terminal; wherein the video stream encoded at the first encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the second encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
In a possible implementation, the processor 1402 is further configured to receive, through the transceiver, a first adjustment request sent by the second terminal after receiving, through the transceiver, the first encoded video stream and the second encoded video stream sent by the first terminal;
the processor 1402 is further configured to forward, via the transceiver, a first adjustment request to the first terminal, where the first adjustment request is used for the first terminal to adjust the coding rate;
the processor 1402 is further configured to receive, through the transceiver, a third encoded video stream and a fourth encoded video stream transmitted by the first terminal, where the third encoded video stream is a video stream encoded by the first terminal at the second encoding rate after receiving the first adjustment request, and the fourth encoded video stream is a video stream encoded by the first terminal at the first encoding rate after receiving the first adjustment request;
a processor further configured to transmit, via the transceiver, the third encoded video stream and the fourth encoded video stream to the second terminal; wherein the video stream encoded at the first encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the second encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
In a possible implementation, the processor 1402 is further configured to analyze the network channel bandwidth to obtain a second adjustment request after receiving, through the transceiver, the first encoded video stream and the second encoded video stream transmitted by the first terminal;
the processor 1402 is further configured to send a second adjustment request to the first terminal through the transceiver, where the second adjustment request is used for the first terminal to adjust the coding rate;
the processor 1402 is further configured to receive, through the transceiver, a fifth encoded video stream and a sixth encoded video stream, where the fifth encoded video stream is a video stream that is encoded and transmitted by the first terminal at the third encoding rate after receiving the second adjustment request, and the sixth encoded video stream is a video stream that is encoded by the first terminal at the fourth encoding rate after receiving the second adjustment request; wherein the third encoding rate is greater than the fourth encoding rate, and the fifth encoded video stream and the sixth encoded video stream satisfy the network channel bandwidth indicated by the second adjustment request;
the processor 1402, further configured to forward the fifth encoded video stream and the sixth encoded video stream to the second terminal through the transceiver; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
In a possible embodiment, the server 140 may further include a network detector; the processor 1402, further configured to analyze, by the network detector, the network bandwidth information to obtain a second adjustment request after the third encoded video stream and the fourth encoded video stream sent by the first terminal are received by the transceiver;
the processor 1402 is further configured to send a second adjustment request to the first terminal through the transceiver, where the second adjustment request is used for the first terminal to adjust the coding rate;
the processor 1402, further configured to receive, through the transceiver, a seventh encoded video stream and an eighth encoded video stream, where the seventh encoded video stream is a video stream that is encoded and transmitted by the first terminal at a fourth encoding rate after receiving the second adjustment request, and the eighth encoded video stream is a video stream that is encoded by the first terminal at a third encoding rate after receiving the second adjustment request; wherein the third encoding rate is greater than the fourth encoding rate, and the seventh encoded video stream and the eighth encoded video stream satisfy the network channel bandwidth indicated by the second adjustment request;
the processor 1402, further configured to forward the seventh encoded video stream and the eighth encoded video stream to the second terminal through the transceiver; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
In a possible implementation, the processor 1402 is further configured to receive, through the transceiver, the first adjustment request sent by the second terminal after forwarding the fifth encoded video stream and the sixth encoded video stream to the second terminal through the transceiver;
the processor 1402 is further configured to forward, via the transceiver, a first adjustment request to the first terminal, where the first adjustment request is used for the first terminal to adjust the coding rate;
the processor 1402 is further configured to receive, through the transceiver, a ninth encoded video stream and a tenth encoded video stream transmitted by the first terminal, where the ninth encoded video stream is a video stream encoded by the first terminal at a fourth encoding rate after receiving the first adjustment request, and the tenth encoded video stream is a video stream encoded by the first terminal at a third encoding rate after receiving the first adjustment request;
the processor 1402, further configured to transmit the ninth encoded video stream and the tenth encoded video stream to the second terminal through the transceiver; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
In a possible implementation, the processor 1402 is further configured to, after the forwarding the first encoded video stream and the second encoded video stream to the second terminal through the transceiver, further include:
the processor 1402 is further configured to receive, through the transceiver, a first audio stream transmitted by the second terminal;
the processor 1402 is further configured to forward the first audio stream to the first terminal through the transceiver, where the first audio stream is used for playing audio by the first terminal.
In a possible embodiment, the first video stream is further used for the first terminal to display in a third display area of the first display screen, and the second video stream is further used for the first terminal to display in a fourth display area of the first display screen; the processor 1402 is further configured to receive, by the transceiver, a feedback encoded video stream sent by the second terminal after forwarding the first encoded video stream and the second encoded video stream to the second terminal through the transceiver, where the feedback encoded video stream is a video stream obtained by encoding a feedback video stream collected by the second terminal;
the processor 1402 is further configured to forward a feedback encoded video stream to the first terminal through the transceiver, where the feedback video stream is used for the first terminal to display in a third display area of the first display screen, the first video stream is also used for the first terminal to display in a fifth display area of the first display screen, and the fifth display area is smaller than the third display area
It should be noted that the implementation of each operation may also correspond to the corresponding description of any one of the method embodiments shown in fig. 3 or the method embodiment shown in fig. 11. The terminal 13 is a server in any one of the method embodiments shown in fig. 3 or the method embodiment shown in fig. 11.
In the terminal 140 depicted in fig. 14, the server 140 forwards the first audio stream and the second audio stream captured by the first terminal to the second terminal. The second terminal receives the two paths of video streams, displays the first coded video stream with higher coding rate in a first display area of the display screen, and displays the second coded video stream in a second display area of the display screen, so that a user of the second terminal can watch a plurality of video pictures sent by the first terminal. Therefore, when a user shares videos, the video interaction method provided by the embodiment of the application can improve the sharing effect of the video pictures shot by the two cameras, and improve the experience of video interaction of the user.
Referring to fig. 15, fig. 15 is a schematic structural diagram of a terminal 150 according to an embodiment of the present disclosure, where the terminal 150 may include a first image capturing unit 1501, a second image capturing unit 1502, an encoding unit 1503 and a communication unit 1504, where the encoding unit 1503 integrates two encoding units, which are respectively a first encoding unit and a second encoding unit, and the two encoding units may be two different encoders, or different modules integrated in the same encoder, or encoding modules in the same encoder. The above-mentioned detailed description of each unit of the terminal 150 is as follows:
the first encoding unit is configured to encode a first video stream acquired by the first camera unit to obtain a first encoded video stream; the second encoding unit is configured to encode a second video stream acquired by the second camera unit to obtain a second encoded video stream; wherein, the coding rate of the first coding unit is a first coding rate, the coding rate of the second coding unit is a second coding rate, and the first coding rate is larger than the second coding rate;
the communication unit is configured to send the first encoded video stream and the second encoded video stream to the second terminal; wherein the video stream encoded at the first encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the second encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
In a possible implementation, the terminal further includes a decision unit; the communication unit is further configured to receive a first adjustment request after transmitting the first encoded video stream and the second encoded video stream to the second terminal;
the decision unit is further configured to adjust the coding rate of the first coding unit to a second coding rate and adjust the coding rate of the second coding unit to a first coding rate according to the first adjustment request;
the first encoding unit is further configured to encode the first video stream acquired by the first camera unit to obtain a third encoded video stream; the second encoding unit is further configured to encode a second video stream acquired by the second camera unit to obtain a fourth encoded video stream;
the communication unit is further configured to transmit the third encoded video stream and the fourth encoded video stream to the second terminal.
In a possible embodiment, the terminal further comprises a decision unit 1505; the communication unit is further configured to receive a second adjustment request sent by the server after sending the first encoded video stream and the second encoded video stream to the second terminal;
the decision unit is further configured to adjust the coding rate of the first coding unit to a third coding rate and adjust the coding rate of the second coding unit to a fourth coding rate according to the second adjustment request, where the third coding rate is greater than the fourth coding rate;
the first encoding unit is further configured to encode the first video stream acquired by the first camera unit through the first encoding unit to obtain a fifth encoded video stream; the second encoding unit is further configured to encode a second video stream acquired by the second camera unit through the second encoding unit to obtain a sixth encoded video stream; the fifth video stream and the sixth encoded video stream satisfy the network channel bandwidth indicated by the second adjustment request;
the communication unit is further configured to send the fifth encoded video stream and the sixth encoded video stream to the second terminal; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal.
In a possible embodiment, the terminal further comprises a decision unit 1505; the communication unit is further configured to receive a second adjustment request sent by the server after sending the first encoded video stream and the second encoded video stream to the second terminal;
the decision unit is further configured to adjust the coding rate of the first coding unit to a fourth coding rate according to the second adjustment request; adjusting the coding rate of the second coding unit to a third coding rate, the third coding rate being greater than a fourth coding rate;
the first encoding unit is further configured to encode the first video stream acquired by the first camera unit to obtain a seventh encoded video stream; the second encoding unit is further configured to encode a second video stream acquired by the second camera unit to obtain an eighth encoded video stream, where the third encoding rate is greater than the fourth encoding rate; the seventh encoded video stream and the eighth encoded video stream satisfy the network channel bandwidth indicated by the second adjustment request;
the communication unit is further configured to send a seventh encoded video stream and an eighth encoded video stream to the second terminal; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal.
In a possible embodiment, the terminal further comprises a decision unit 1505; the communication unit is further configured to receive a first adjustment request after transmitting the first encoded video stream and the second encoded video stream to the second terminal;
the decision unit is further configured to adjust the coding rate of the first coding unit to a fourth coding rate and adjust the coding rate of the second coding unit to a third coding rate according to the first adjustment request;
the first encoding unit is further configured to encode the first video stream acquired by the first camera unit to obtain a ninth encoded video stream; the second encoding unit is further configured to encode a second video stream acquired by the second camera unit through the second encoding unit to obtain a tenth encoded video stream; the decoding rate of the first decoding unit is a third decoding rate, and the decoding rate of the second decoding unit is a fourth decoding rate;
the communication unit is further configured to send a ninth encoded video stream and a tenth encoded video stream to the second terminal; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal.
In a possible embodiment, the terminal further includes an output unit 1507; the communication unit is further configured to receive a first audio stream transmitted by the second terminal after the first encoded video stream and the second encoded video stream are transmitted to the second terminal through the communication unit;
the output unit is used for playing the audio according to the first audio stream.
In a possible implementation, the terminal further includes a decision unit 1505, a first display 1507, and a decoding unit 1508; the decision unit is further configured to display the first video stream acquired by the first camera unit in a third display area of the first display screen after the first video stream acquired by the first camera unit is encoded by the first encoding unit to obtain a first encoded video stream;
the decision unit is further configured to display the second video stream acquired by the second camera unit in a fourth display area of the first display screen after the second video stream acquired by the second camera unit is encoded by the second encoding unit to obtain a second encoded video stream, where the third display area is larger than the fourth display area;
the communication unit is further configured to receive a feedback encoded video stream sent by the second terminal after the first video stream and the second video stream are sent to the second terminal, where the feedback encoded video stream is a video stream obtained by the second terminal after the feedback video stream acquired by the third camera unit is encoded;
the decoding unit is further configured to decode the feedback encoded video stream by the decoding unit to obtain a feedback decoded video stream;
the decision unit is further configured to display the feedback decoded video stream in a third display area of the first display screen, and the processing unit is further configured to adjust the first video stream acquired by the first camera unit to be displayed in a fifth display area of the first display screen, where the fifth display area of the first display screen is smaller than the third display area.
It should be noted that the implementation of each operation may also correspond to the corresponding description of any one of the method embodiments shown in fig. 3 or the method embodiment shown in fig. 11. The terminal 150 is a first terminal in any of the method embodiments shown in fig. 3 or fig. 11.
Fig. 15 illustrates a terminal 150 in which a first terminal captures a first video stream and a second video stream, encodes the first video stream and the second video stream using a first encoding rate and a second encoding rate, respectively, and transmits the encoded first video stream and the encoded second video stream to a second terminal. The second terminal receives the two paths of video streams, displays the first coded video stream with higher coding rate in a first display area of the display screen, and displays the second coded video stream in a second display area of the display screen, so that a user of the second terminal can watch a plurality of video pictures sent by the first terminal; correspondingly, because the second display area is smaller, the video stream of the second display area can properly reduce the encoding rate, so that the user experience is not influenced, and the network resources are saved; therefore, when a user shares videos, the video interaction method provided by the embodiment of the application can improve the sharing effect of the video pictures shot by the two cameras, and improve the experience of video interaction of the user.
Referring to fig. 16, fig. 16 is a schematic structural diagram of another terminal 160 according to an embodiment of the present disclosure, where the terminal 160 may include a communication 1601, a decoding unit 1602, a decision unit 1603, and a second display 1604, where the decoding unit 1602 integrates two decoding units, which are a first decoding unit and a second decoding unit, respectively, and the two decoding units may be two different decoders, may also be different modules integrated in the same decoder, or may be decoding modules in the same decoder. The above-mentioned detailed description of each unit of the terminal 160 is as follows:
the communication unit is configured to receive a first encoded video stream and a second encoded video stream sent by a first terminal, where the first encoded video stream is a video stream obtained by encoding the first video stream collected by the first terminal at a first encoding rate, the second encoded video stream is a video stream obtained by encoding the second video stream collected by the first terminal at a second encoding rate, and the first encoding rate is greater than the second encoding rate;
the first decoding unit is configured to decode the first encoded video stream to obtain a first decoded video stream; the second decoding unit is configured to decode a second encoded video stream to obtain a second decoded video stream; the decoding rate of the first decoding unit is a first decoding rate, and the decoding rate of the second decoding unit is a second decoding rate;
the decision unit is configured to display the video stream encoded at the first encoding rate in a first display area of a second display screen, and display a second decoded video stream of the video stream encoded at a second encoding rate in a second display area of the second display screen, where the first display area is larger than the second display area.
In a possible implementation, the communication unit is further configured to receive a switching command of the display area after the decision unit displays the first decoded video stream encoded at the first encoding rate in the first display area of the second display screen and displays the second decoded video stream encoded at the second encoding rate in the second display area of the second display screen;
the decision unit is further configured to generate a first adjustment request according to the handover command, and send the first adjustment request to the first terminal through the communication unit;
the communication unit is further configured to receive a third encoded video stream and a fourth encoded video stream sent by the first terminal, where the third encoded video stream is a video stream sent by the first terminal after receiving the first adjustment request and encoded at the second encoding rate, and the fourth encoded video stream is a video stream sent by the first terminal after receiving the first adjustment request and encoded at the first encoding rate;
the second decoding unit is further configured to obtain a third decoded video stream by decoding the third encoded video stream; the fourth decoding unit is further configured to decode the fourth encoded video stream by the first decoding unit to obtain a fourth decoded video stream;
the decision unit is further configured to display the video stream encoded at the first encoding rate in a first display area of a second display screen, and display the video stream encoded at the second encoding rate in a second display area of the second display screen, where the first display area is larger than the second display area.
In a possible implementation, the communication unit is further configured to receive a fifth encoded video stream and a sixth encoded video stream after the decision unit displays the first decoded video stream in the first display area of the second display screen and displays the second decoded video stream in the second display area of the second display screen, where the fifth encoded video stream is a video stream that is transmitted by the first terminal through the third encoding rate after receiving the second adjustment request, and the sixth encoded video stream is a video stream that is encoded by the first terminal through the fourth encoding rate after receiving the second adjustment request; the second adjustment request is sent to the first terminal by the server, and the fifth coded video stream and the sixth coded video stream meet the network channel bandwidth indicated by the second adjustment request;
the first decoding unit is further configured to decode a fifth encoded video stream to obtain a fifth decoded video stream; the second decoding unit is further configured to decode the sixth encoded video stream by the second decoding unit to obtain a sixth decoded video stream; the decoding rate of the first decoding unit is a third decoding rate, and the decoding rate of the second decoding unit is a fourth decoding rate;
the decision unit is further configured to display the video stream encoded at the third encoding rate in a first display area of the second display screen, and display the video stream encoded at the third encoding rate in a second display area of the second display screen, where the first display area is larger than the second display area.
In a possible implementation, the communication unit is further configured to receive a seventh encoded video stream and an eighth encoded video stream after receiving the third encoded video stream and the fourth encoded video stream transmitted by the first terminal, where the seventh encoded video stream is a video stream transmitted by the first terminal after receiving the second adjustment request and the eighth encoded video stream is a video stream transmitted by the first terminal after receiving the second adjustment request and encoded by the fourth encoding rate; the second adjustment request is sent to the second terminal by the server, and the seventh coded video stream and the eighth coded video stream meet the network channel bandwidth indicated by the second adjustment request;
the second decoding unit is further configured to decode the seventh encoded video stream to obtain a seventh decoded video stream; the first decoding unit is further configured to decode the eighth encoded video stream by the first decoding unit to obtain an eighth decoded video stream; the decoding rate of the first decoding unit is a third decoding rate, and the decoding rate of the second decoding unit is a fourth decoding rate;
the decision unit is further configured to display the video stream encoded at the third encoding rate in a first display area of the second display screen, and display the video stream encoded at the fourth encoding rate in a second display area of the second display screen, where the first display area is larger than the second display area.
In a possible implementation, the communication unit is further configured to receive a switching command of the display area after receiving the fifth encoded video stream and the sixth encoded video stream through the communication unit;
the decision unit is further configured to generate a first adjustment request according to the handover command, and send the first adjustment request to the first terminal through the communication unit;
the above communication unit is further configured to receive a ninth encoded video stream and a tenth encoded video stream sent by the first terminal, where the ninth encoded video stream is a video stream sent by the first terminal after receiving the first adjustment request and encoded at a fourth encoding rate, and the tenth encoded video stream is a video stream sent by the first terminal after receiving the first adjustment request and encoded at a third encoding rate;
the second decoding unit is further configured to obtain a third decoded video stream by decoding the ninth encoded video stream through the second decoding unit; the first decoding unit is further configured to decode the tenth encoded video stream by the first decoding unit to obtain a fourth decoded video stream; the decoding rate of the first decoding unit is a third decoding rate, and the decoding rate of the second decoding unit is a fourth decoding rate;
the decision unit is further configured to display the video stream encoded at the third encoding rate in a first display area of the second display screen, and display the video stream encoded at the fourth encoding rate in a second display area of the second display screen, where the first display area is larger than the second display area.
In a possible embodiment, the terminal further includes an audio acquisition unit 1605; the audio acquiring unit is further configured to acquire the first audio stream after the decision unit displays the video stream encoded at the first encoding rate in the first display area of the second display screen and displays the second decoded video stream of the video stream encoded at the second encoding rate in the second display area of the second display screen;
the communication unit is further configured to send a first audio stream to the first terminal, where the first audio stream is used for the first terminal to play audio.
In a possible embodiment, the terminal further comprises a third camera 1606 and an encoding unit 1607; the first video stream is also used for the first terminal to display in a third display area of the first display screen, and the second video stream is also used for the first terminal to display in a fourth display area of the first display screen; the third camera unit is further configured to capture a feedback video stream after displaying the video stream encoded at the first encoding rate in the first display area of the second display screen and displaying the second decoded video stream of the video stream encoded at the second encoding rate in the second display area of the second display screen;
the decision unit is further configured to display the feedback video stream in a sixth display area of the second display screen, where the sixth display area is smaller than the first display area;
the encoding unit is further configured to encode the feedback video stream to obtain a feedback encoded video stream;
the communication unit is further configured to send a feedback encoded video stream to the first terminal, where the feedback encoded video stream is used for the first terminal to display in a third display area of the first display screen, and the first video stream is also used for displaying in a fifth display area of the first display screen of the first terminal, where the fifth display area is smaller than the third display area.
It should be noted that the implementation of each operation may also correspond to the corresponding description of any one of the method embodiments shown in fig. 3 or the method embodiment shown in fig. 11. The terminal 160 is a second terminal in any of the method embodiments shown in fig. 3 or fig. 11.
In the terminal 160 illustrated in fig. 16, a first terminal collects a first video stream and a second video stream, encodes the first video stream and the second video stream using a first encoding rate and a second encoding rate, respectively, and transmits the encoded first video stream and the encoded second video stream to a second terminal. The second terminal receives the two paths of video streams, displays the first coded video stream with higher coding rate in a first display area of the display screen, and displays the second coded video stream in a second display area of the display screen, so that a user of the second terminal can watch a plurality of video pictures sent by the first terminal; correspondingly, because the second display area is smaller, the video stream of the second display area can properly reduce the encoding rate, so that the user experience is not influenced, and the network resources are saved; therefore, when a user shares videos, the video interaction method provided by the embodiment of the application can improve the sharing effect of the video pictures shot by the two cameras, and improve the experience of video interaction of the user.
Referring to fig. 17, fig. 17 is a schematic structural diagram of a server 170 according to an embodiment of the present disclosure, where the server 170 may include a communication unit 1701, where details of each unit are as follows:
the communication unit is configured to receive a first encoded video stream and a second encoded video stream sent by a first terminal, where the first encoded video stream is a video stream obtained by encoding the first video stream collected by the first terminal at a first encoding rate, the second encoded video stream is a video stream obtained by encoding the second video stream collected by the first terminal at a second encoding rate, and the first encoding rate is greater than the second encoding rate;
the communication unit is further configured to forward the first encoded video stream and the second encoded video stream to the second terminal; wherein the video stream encoded at the first encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the second encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
In a possible implementation, the communication unit is further configured to receive, after receiving the first encoded video stream and the second encoded video stream transmitted by the first terminal through the communication unit, a first adjustment request transmitted by the second terminal;
the communication unit is further configured to forward a first adjustment request to the first terminal, where the first adjustment request is used for the first terminal to adjust the coding rate;
the communication unit is further configured to receive a third encoded video stream and a fourth encoded video stream sent by the first terminal, where the third encoded video stream is a video stream encoded by the first terminal at the second encoding rate after receiving the first adjustment request, and the fourth encoded video stream is a video stream encoded by the first terminal at the first encoding rate after receiving the first adjustment request;
the communication unit is further configured to send a third encoded video stream and a fourth encoded video stream to the second terminal; wherein the video stream encoded at the first encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the second encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
In a possible embodiment, the server further includes a decision unit 1702; the decision unit is configured to analyze a network channel bandwidth to obtain a second adjustment request after the communication unit receives the first encoded video stream and the second encoded video stream sent by the first terminal;
the communication unit is further configured to send a second adjustment request to the first terminal, where the second adjustment request is used for the first terminal to adjust the coding rate;
the communication unit is further configured to receive a fifth encoded video stream and a sixth encoded video stream, where the fifth encoded video stream is a video stream that is encoded and transmitted by the first terminal at the third encoding rate after receiving the second adjustment request, and the sixth encoded video stream is a video stream that is encoded by the first terminal at the fourth encoding rate after receiving the second adjustment request; wherein the third encoding rate is greater than the fourth encoding rate, and the fifth encoded video stream and the sixth encoded video stream satisfy the network channel bandwidth indicated by the second adjustment request;
the communication unit is further configured to forward the fifth encoded video stream and the sixth encoded video stream to the second terminal; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
In a possible embodiment, the server further includes a decision unit; the decision unit is configured to analyze network bandwidth information to obtain a second adjustment request after the communication unit receives a third encoded video stream and a fourth encoded video stream sent by the first terminal;
the communication unit is further configured to send a second adjustment request to the first terminal, where the second adjustment request is used for the first terminal to adjust the coding rate;
the communication unit is further configured to receive a seventh encoded video stream and an eighth encoded video stream, where the seventh encoded video stream is a video stream that is encoded and transmitted by the first terminal at the fourth encoding rate after receiving the second adjustment request, and the eighth encoded video stream is a video stream that is encoded by the first terminal at the third encoding rate after receiving the second adjustment request; wherein the third encoding rate is greater than the fourth encoding rate, and the seventh encoded video stream and the eighth encoded video stream satisfy the network channel bandwidth indicated by the second adjustment request;
the communication unit is further configured to forward the seventh encoded video stream and the eighth encoded video stream to the second terminal; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
In a possible implementation, the communication unit is further configured to receive a first adjustment request sent by the second terminal after forwarding the fifth encoded video stream and the sixth encoded video stream to the second terminal through the communication unit;
the communication unit is further configured to forward a first adjustment request to the first terminal, where the first adjustment request is used for the first terminal to adjust the coding rate;
the above-mentioned communication unit is further configured to receive a ninth encoded video stream and a tenth encoded video stream sent by the first terminal, where the ninth encoded video stream is a video stream encoded by the first terminal at a fourth encoding rate after receiving the first adjustment request, and the tenth encoded video stream is a video stream encoded by the first terminal at a third encoding rate after receiving the first adjustment request;
the communication unit is further configured to send a ninth encoded video stream and a tenth encoded video stream to the second terminal; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
In a possible implementation, the communication unit is further configured to receive a first audio stream sent by a second terminal after forwarding the first encoded video stream and the second encoded video stream to the second terminal;
the communication unit is further configured to forward a first audio stream to the first terminal, where the first audio stream is used for the first terminal to play audio.
It should be noted that the implementation of each operation may also correspond to the corresponding description of any one of the method embodiments shown in fig. 3 or the method embodiment shown in fig. 11. The terminal 150 is a first terminal in any of the method embodiments shown in fig. 3 or fig. 11.
Fig. 17 illustrates a terminal 170 in which a first terminal captures a first video stream and a second video stream, encodes the first video stream and the second video stream using a first encoding rate and a second encoding rate, respectively, and transmits the encoded first video stream and the encoded second video stream to a second terminal. The second terminal receives the two paths of video streams, displays the first coded video stream with higher coding rate in a first display area of the display screen, and displays the second coded video stream in a second display area of the display screen, so that a user of the second terminal can watch a plurality of video pictures sent by the first terminal; correspondingly, because the second display area is smaller, the video stream of the second display area can properly reduce the encoding rate, so that the user experience is not influenced, and the network resources are saved; therefore, when a user shares videos, the video interaction method provided by the embodiment of the application can improve the sharing effect of the video pictures shot by the two cameras, and improve the experience of video interaction of the user.
Embodiments of the present application further provide a computer-readable storage medium, where instructions are stored, and when the instructions are executed on a processor, the instructions implement operations performed by a first terminal in the embodiment shown in fig. 3, or implement operations performed by a second terminal in the embodiment shown in fig. 3, or implement operations performed by a server in the embodiment shown in fig. 3, or implement operations performed by the first terminal in the embodiment shown in fig. 11, or implement operations performed by the second terminal in the embodiment shown in fig. 11, or implement operations performed by the server in the embodiment shown in fig. 5.
Embodiments of the present application further provide a computer program product, which when executed on a processor, implements operations performed by the first terminal in the embodiment shown in fig. 3, or implements operations performed by the second terminal in the embodiment shown in fig. 3, or implements operations performed by the server in the embodiment shown in fig. 3, or implements operations performed by the first terminal in the embodiment shown in fig. 11, or implements operations performed by the second terminal in the embodiment shown in fig. 11, or implements operations performed by the server in the embodiment shown in fig. 11.
One of ordinary skill in the art will appreciate that all or part of the processes in the methods of the above embodiments may be implemented by hardware related to instructions of a computer program, which may be stored in a computer-readable storage medium, and when executed, may include the processes of the above method embodiments. And the aforementioned storage medium includes: various media capable of storing program codes, such as ROM or RAM, magnetic or optical disks, etc.

Claims (37)

1. A video interaction method is characterized by being applied to a first terminal with a first camera module, a second camera module and a first display screen, wherein the first terminal and the second terminal carry out real-time video interaction; the method comprises the following steps:
the first terminal encodes the first video stream acquired by the first camera module through a first encoding module to obtain a first encoded video stream; the first terminal encodes the second video stream acquired by the second camera module through a second encoding module to obtain a second encoded video stream; wherein the coding rate of the first coding module is a first coding rate, the coding rate of the second coding module is a second coding rate, and the first coding rate is greater than the second coding rate;
the first terminal sending the first encoded video stream and the second encoded video stream to the second terminal; wherein the video stream encoded at the first encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the second encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
2. The method of claim 1, wherein after the first terminal transmits the first encoded video stream and the second encoded video stream to a second terminal, further comprising:
the first terminal receives a first adjustment request;
the first terminal adjusts the coding rate of the first coding module to the second coding rate and adjusts the coding rate of the second coding module to the first coding rate according to the first adjustment request;
the first terminal encodes the first video stream acquired by the first camera module through the first encoding module to obtain a third encoded video stream; the first terminal encodes the second video stream acquired by the second camera module through the second encoding module to obtain a fourth encoded video stream;
the first terminal sends the third encoded video stream and the fourth encoded video stream to the second terminal.
3. The method of claim 1, wherein after the first terminal transmits the first encoded video stream and the second encoded video stream to a second terminal, further comprising:
the first terminal receives a second adjustment request sent by the server;
according to the second adjustment request, the first terminal adjusts the coding rate of the first coding module to the third coding rate and adjusts the coding rate of the second coding module to the fourth coding rate, wherein the third coding rate is greater than the fourth coding rate;
the first terminal encodes the first video stream acquired by the first camera module through the first encoding module to obtain a fifth encoded video stream; the first terminal encodes the second video stream acquired by the second camera module through the second encoding module to obtain a sixth encoded video stream; the fifth video stream and the sixth encoded video stream satisfy the network channel bandwidth indicated by the second adjustment request;
the first terminal transmitting the fifth encoded video stream and the sixth encoded video stream to the second terminal; wherein the video stream encoded at the third encoding rate is for display in a first display area of the second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal.
4. The method of claim 2, wherein after the first terminal transmits the third encoded video stream and the fourth encoded video stream to the second terminal, further comprising:
the first terminal receives a second adjustment request sent by the server;
the first terminal adjusts the coding rate of the first coding module to a fourth coding rate according to the second adjustment request; adjusting the coding rate of the second coding module to a third coding rate, the third coding rate being greater than the fourth coding rate;
the first terminal encodes the first video stream acquired by the first camera module through the first encoding module to obtain a seventh encoded video stream; the first terminal encodes the second video stream acquired by the second camera module through the second encoding module to obtain an eighth encoded video stream, wherein the third encoding rate is greater than the fourth encoding rate; the seventh encoded video stream and the eighth encoded video stream satisfy the network channel bandwidth indicated by the second adjustment request;
the first terminal transmitting the seventh encoded video stream and the eighth encoded video stream to the second terminal; wherein the video stream encoded at the third encoding rate is for display in the first display area of the second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in the second display area of the second display screen of the second terminal.
5. The method of any of claims 1-4, further comprising, after the first terminal transmits the first encoded video stream and the second encoded video stream to a second terminal:
the first terminal receives a first audio stream sent by the second terminal;
and the first terminal plays audio according to the first audio stream.
6. The method according to any one of claims 1 to 5, wherein after the first terminal encodes the first video stream captured by the first camera module through a first encoding module to obtain a first encoded video stream, the method further comprises:
the first terminal displays a first video stream acquired by the first camera module in a third display area of a first display screen;
after the first terminal encodes the second video stream acquired by the second camera module through the second encoding module to obtain a second encoded video stream, the method further includes:
the first terminal displays a second video stream acquired by the second camera module in a fourth display area of the first display screen, wherein the third display area is larger than the fourth display area;
after the first terminal sends the first encoded video stream and the second encoded video stream to the second terminal, the method further includes:
the first terminal receives a feedback coding video stream sent by the second terminal, wherein the feedback coding video stream is a video stream obtained by coding the feedback video stream acquired by the second terminal through a third camera module;
the first terminal decodes the feedback coding video stream to obtain a feedback decoding video stream;
the first terminal displays the feedback decoding video stream in a third display area of the first display screen, and adjusts the first video stream acquired by the first camera module to be displayed in a fifth display area of the first display screen, wherein the fifth display area of the first display screen is smaller than the third display area.
7. A video interaction method is applied to a second terminal with a second display screen, wherein the second terminal performs real-time video interaction with a first terminal, and the method comprises the following steps:
the second terminal receives a first coded video stream and a second coded video stream sent by the first terminal, wherein the first coded video stream is a video stream obtained by coding the first video stream collected by the first terminal at a first coding rate, the second coded video stream is a video stream obtained by coding the second video stream collected by the first terminal at a second coding rate, and the first coding rate is greater than the second coding rate;
the second terminal decodes the first coded video stream through a first decoding module to obtain a first decoded video stream, and decodes the second coded video stream through a second decoding module to obtain a second decoded video stream; the decoding rate of the first decoding module is a first decoding rate, and the decoding rate of the second decoding module is a second decoding rate;
and the second terminal displays the first decoded video stream in a first display area of the second display screen, and displays the second decoded video stream in a second display area of the second display screen, wherein the first display area is larger than the second display area.
8. The method according to claim 7, wherein after the second terminal displays the first decoded video stream on the first display area of the second display screen and the second terminal displays the second decoded video stream on the second display area of the second display screen, the method further comprises:
the second terminal receives a switching command of a display area, generates a first adjustment request according to the switching command, and sends the first adjustment request to the first terminal;
the second terminal receives a third coded video stream and a fourth coded video stream sent by the first terminal, wherein the third coded video stream is a video stream coded by the first terminal at a second coding rate after receiving the first adjustment request, and the fourth coded video stream is a video stream coded by the first terminal at a first coding rate after receiving the first adjustment request;
the second terminal decodes the third coded video stream through the second decoding module to obtain a third decoded video stream, and decodes the fourth coded video stream through the first decoding module to obtain a fourth decoded video stream;
the second terminal displays the fourth decoded video stream in a first display area of the second display screen, and displays the third decoded video stream in a second display area of the second display screen, wherein the first display area is larger than the second display area.
9. The method according to claim 7, wherein after the second terminal displays the first decoded video stream on the first display area of the second display screen and the second terminal displays the second decoded video stream on the second display area of the second display screen, the method further comprises:
the second terminal receives a fifth coded video stream and a sixth coded video stream, wherein the fifth coded video stream is a video stream which is sent by the first terminal after receiving the second adjustment request through the third coding rate, and the sixth coded video stream is a video stream which is coded by the first terminal after receiving the second adjustment request through the fourth coding rate; the second adjustment request is sent to the first terminal by the server, and the fifth coded video stream and the sixth coded video stream meet the network channel bandwidth indicated by the second adjustment request;
the second terminal decodes the fifth coded video stream through the first decoding module to obtain a fifth decoded video stream, and decodes the sixth coded video stream through the second decoding module to obtain a sixth decoded video stream; the decoding rate of the first decoding module is a third decoding rate, and the decoding rate of the second decoding module is a fourth decoding rate;
and the second terminal displays the fifth decoded video stream in a first display area of the second display screen, and the second terminal displays the sixth decoded video stream in a second display area of the second display screen, wherein the first display area is larger than the second display area.
10. The method according to claim 8, wherein after the second terminal receives the third encoded video stream and the fourth encoded video stream transmitted by the first terminal, the method further comprises:
the second terminal receives a seventh coded video stream and an eighth coded video stream, wherein the seventh coded video stream is a video stream sent by the first terminal through a fourth coding rate after receiving the second adjustment request, and the eighth coded video stream is a video stream coded by the first terminal through a third coding rate after receiving the second adjustment request; the second adjustment request is sent to a second terminal by a server, and the seventh coded video stream and the eighth coded video stream meet the network channel bandwidth indicated by the second adjustment request;
the second terminal decodes the seventh coded video stream through the second decoding module to obtain a seventh decoded video stream, and decodes the eighth coded video stream through the first decoding module to obtain an eighth decoded video stream; the decoding rate of the first decoding module is a third decoding rate, and the decoding rate of the second decoding module is a fourth decoding rate;
and the second terminal displays the eighth decoded video stream in a first display area of the second display screen, and the second terminal displays the seventh decoded video stream in a second display area of the second display screen, wherein the first display area is larger than the second display area.
11. The method according to any one of claims 7 to 10, wherein after the second terminal displays the first decoded video stream in the first display area of the second display screen and the second terminal displays the second decoded video stream in the second display area of the second display screen, further comprising:
the second terminal collects a first audio stream;
and the second terminal sends the first audio stream to the first terminal, and the first audio stream is used for the first terminal to play audio.
12. The method of any of claims 7 to 11, wherein the first video stream is also for display by the first terminal in a third display area of a first display screen, and wherein the second video stream is also for display by the first terminal in a fourth display area of the first display screen; after the second terminal displays the first decoded video stream in the first display area of the second display screen and the second terminal displays the second decoded video stream in the second display area of the second display screen, the method further includes:
the second terminal acquires a feedback video stream through a third camera module;
the second terminal displays the feedback video stream in a sixth display area of the second display screen, wherein the sixth display area is smaller than the first display area;
the second terminal encodes the feedback video stream to obtain a feedback encoded video stream;
and the second terminal sends the feedback coded video stream to the first terminal, wherein the feedback coded video stream is used for being displayed in a third display area of the first display screen by the first terminal, the first video stream is also used for being displayed in a fifth display area of the first display screen of the first terminal, and the fifth display area is smaller than the third display area.
13. A video interaction method is characterized in that when the method is applied to a first terminal and a second terminal to carry out real-time video interaction, the method comprises the following steps:
a server receives a first coded video stream and a second coded video stream sent by a first terminal, wherein the first coded video stream is a video stream obtained by coding the first video stream acquired by the first terminal at a first coding rate, the second coded video stream is a video stream obtained by coding the second video stream acquired by the first terminal at a second coding rate, and the first coding rate is greater than the second coding rate;
the server forwards the first encoded video stream and the second encoded video stream to the second terminal; wherein the video stream encoded at the first encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the second encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
14. The method according to claim 13, wherein after the server receives the first encoded video stream and the second encoded video stream transmitted by the first terminal, the method further comprises:
the server receives a first adjustment request sent by the second terminal;
the server forwards the first adjustment request to the first terminal, wherein the first adjustment request is used for the first terminal to adjust the coding rate;
the server receives a third coded video stream and a fourth coded video stream sent by the first terminal, wherein the third coded video stream is a video stream coded by the first terminal at a second coding rate after receiving the first adjustment request, and the fourth coded video stream is a video stream coded by the first terminal at a first coding rate after receiving the first adjustment request;
transmitting the third encoded video stream and the fourth encoded video stream to the second terminal; wherein the video stream encoded at the first encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the second encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
15. The method according to claim 13, wherein after the server receives the first encoded video stream and the second encoded video stream transmitted by the first terminal, the method further comprises:
the server analyzes the bandwidth of the network channel to obtain a second adjustment request;
the server sends the second adjustment request to a first terminal, wherein the second adjustment request is used for the first terminal to adjust the coding rate;
the server receives a fifth coded video stream and a sixth coded video stream, wherein the fifth coded video stream is a video stream which is sent by the first terminal after receiving the second adjustment request through coding at a third coding rate, and the sixth coded video stream is a video stream which is sent by the first terminal after receiving the second adjustment request through coding at a fourth coding rate; wherein the third encoding rate is greater than the fourth encoding rate, and the fifth encoded video stream and the sixth encoded video stream satisfy the network channel bandwidth indicated by the second adjustment request;
the server forwards the fifth encoded video stream and the sixth encoded video stream to the second terminal; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
16. The method according to claim 14, wherein after the server receives the first encoded video stream and the second encoded video stream transmitted by the first terminal, the method further comprises:
the server analyzes the bandwidth of the network channel to obtain a second adjustment request;
the server sends the second adjustment request to a first terminal, wherein the second adjustment request is used for the first terminal to adjust the coding rate;
the server receives a seventh coded video stream and an eighth coded video stream, wherein the seventh coded video stream is a video stream which is sent by the first terminal after receiving the second adjustment request through a fourth coding rate, and the eighth coded video stream is a video stream which is sent by the first terminal after receiving the second adjustment request through a third coding rate; wherein the third encoding rate is greater than the fourth encoding rate, and the seventh encoded video stream and the eighth encoded video stream satisfy the network channel bandwidth indicated by the second adjustment request;
the server forwards the seventh encoded video stream and the eighth encoded video stream to the second terminal; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
17. The method according to any one of claims 13 to 16, further comprising, after the server receives the first encoded video stream and the second encoded video stream transmitted by the first terminal:
the server receives a first audio stream sent by the second terminal;
and the server forwards the first audio stream to the first terminal, wherein the first audio stream is used for the first terminal to play audio.
18. The method of any of claims 13 to 17, wherein the first video stream is also for display by the first terminal in a third display area of a first display screen, and wherein the second video stream is also for display by the first terminal in a fourth display area of the first display screen; after the server forwards the first encoded video stream and the second encoded video stream to the second terminal, the method further includes:
receiving a feedback coding video stream sent by the second terminal, wherein the feedback coding video stream is a video stream obtained by coding the feedback video stream collected by the second terminal;
and forwarding the feedback coding video stream to the first terminal, wherein the feedback video stream is used for the first terminal to display in a third display area of the first display screen, the first video stream is also used for the first terminal to display in a fifth display area of the first display screen, and the fifth display area is smaller than the third display area.
19. A terminal comprising a memory, a processor, a transceiver, a first camera module, a second camera module, a first encoding module, and a second encoding module, the memory storing a program, the processor invoking the program for performing the following operations:
coding a first video stream acquired by the first camera module through a first coding module to obtain a first coded video stream; the first terminal encodes the second video stream acquired by the second camera module through a second encoding module to obtain a second encoded video stream; wherein the coding rate of the first coding module is a first coding rate, the coding rate of the second coding module is a second coding rate, and the first coding rate is greater than the second coding rate;
transmitting, by the transceiver, the first encoded video stream and the second encoded video stream to the second terminal; wherein the video stream encoded at the first encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the second encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
20. The terminal of claim 19, wherein the processor is further configured to:
receiving, by the transceiver, a first adjustment request after transmitting the first encoded video stream and the second encoded video stream to the second terminal through the transceiver;
adjusting the coding rate of the first coding module to the second coding rate and the coding rate of the second coding module to the first coding rate according to the first adjustment request;
coding a first video stream acquired by the first camera module through the first coding module to obtain a third coded video stream; the processor is further configured to encode a second video stream acquired by the second camera module through the second encoding module to obtain a fourth encoded video stream;
transmitting, by the transceiver, the third encoded video stream and the fourth encoded video stream to the second terminal.
21. The terminal of claim 19, wherein the processor is further configured to:
receiving, by the transceiver, a second adjustment request transmitted by a server after transmitting the first encoded video stream and the second encoded video stream to the second terminal through the transceiver;
adjusting the coding rate of the first coding module to the third coding rate and the coding rate of the second coding module to the fourth coding rate according to the second adjustment request, wherein the third coding rate is greater than the fourth coding rate;
coding the first video stream acquired by the first camera module through the first coding module to obtain a fifth coded video stream; the processor is further configured to encode a second video stream acquired by the second camera module through the second encoding module to obtain a sixth encoded video stream; the fifth video stream and the sixth encoded video stream satisfy the network channel bandwidth indicated by the second adjustment request;
transmitting, by the transceiver, the fifth encoded video stream and the sixth encoded video stream to the second terminal; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal.
22. The terminal of claim 20, wherein the processor is further configured to:
receiving, by the transceiver, a second adjustment request transmitted by the server after transmitting the third encoded video stream and the fourth encoded video stream to the second terminal through the transceiver;
adjusting the coding rate of the first coding module to a fourth coding rate according to the second adjustment request; adjusting the coding rate of the second coding module to a third coding rate, the third coding rate being greater than the fourth coding rate;
the first video stream collected by the first camera module is coded by controlling a first coding module to obtain a seventh coded video stream; the processor is further configured to encode a second video stream acquired by the second camera module by controlling a second encoding module to obtain an eighth encoded video stream, where the third encoding rate is greater than the fourth encoding rate; the seventh encoded video stream and the eighth encoded video stream satisfy the network channel bandwidth indicated by the second adjustment request;
transmitting, by the transceiver, the seventh encoded video stream and the eighth encoded video stream to the second terminal; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal.
23. A terminal according to any of claims 19 to 22, characterized in that the terminal further comprises a player; the processor is further configured to:
after the first encoded video stream and the second encoded video stream are transmitted to the second terminal through the transceiver, receiving a first audio stream transmitted by the second terminal through the transceiver;
and playing audio according to the first audio stream through the player.
24. A terminal according to any of claims 19 to 22, characterized in that the terminal further comprises a first display and a decoder; the processor is further configured to:
after a first video stream acquired by the first camera module is coded by a first coding module to obtain a first coded video stream, the first video stream acquired by the first camera module is displayed in a third display area of a first display screen;
after a second video stream acquired by the second camera module is encoded by a second encoding module to obtain a second encoded video stream, displaying the second video stream acquired by the second camera module in a fourth display area of the first display screen, wherein the third display area is larger than the fourth display area;
after the first video stream and the second video stream are sent to a second terminal, receiving a feedback coding video stream sent by the second terminal through the transceiver, wherein the feedback coding video stream is a video stream obtained by coding the feedback video stream collected by the second terminal through a third camera module;
decoding the feedback coding video stream through the decoder to obtain a feedback decoding video stream;
and the processor is further configured to adjust the first video stream acquired by the first camera module to be displayed in a fifth display area of the first display screen, where the fifth display area of the first display screen is smaller than the third display area.
25. A terminal, comprising a memory, a processor, a transceiver, a first decoding module, a second decoding module, and a second display screen, wherein the memory stores a program, and the processor calls the program to perform the following operations:
receiving, by the transceiver, a first encoded video stream and a second encoded video stream sent by the first terminal, where the first encoded video stream is a video stream obtained by encoding the first video stream acquired by the first terminal at a first encoding rate, the second encoded video stream is a video stream obtained by encoding the second video stream acquired by the first terminal at a second encoding rate, and the first encoding rate is greater than the second encoding rate;
decoding the first coded video stream by a first decoding module to obtain a first decoded video stream, and decoding the second coded video stream by a second decoding rate of a second decoding module to obtain a second decoded video stream; the decoding rate of the first decoding module is a first decoding rate, and the decoding rate of the second decoding module is a second decoding rate;
displaying the first decoded video stream in a first display area of the second display screen, and displaying the second decoded video stream in a second display area of the second display screen, wherein the first display area is larger than the second display area.
26. The terminal of claim 25, wherein the processor is further configured to:
after the first decoded video stream is displayed in a first display area of the second display screen and the second decoded video stream is displayed in a second display area of the second display screen, receiving a switching command of the display areas through the transceiver, generating a first adjustment request according to the switching command, and sending the first adjustment request to the first terminal through the transceiver;
receiving a third encoded video stream and a fourth encoded video stream sent by the first terminal, wherein the third encoded video stream is a video stream sent by the first terminal after receiving the first adjustment request and encoded at a second encoding rate, and the fourth encoded video stream is a video stream sent by the first terminal after receiving the first adjustment request and encoded at a first encoding rate;
decoding the third coded video stream by the second decoding module to obtain a third decoded video stream, and decoding the fourth coded video stream by the first decoding module to obtain a fourth decoded video stream;
displaying the fourth decoded video stream in a first display area of the second display screen, and displaying the third decoded video stream in a second display area of the second display screen, wherein the first display area is larger than the second display area.
27. The terminal of claim 25, wherein the processor is further configured to:
receiving, by the transceiver, a fifth encoded video stream and a sixth encoded video stream after displaying the first decoded video stream in the first display area of the second display screen and displaying the second decoded video stream in the second display area of the second display screen, the fifth encoded video stream being a video stream transmitted by the first terminal through encoding at a third encoding rate after receiving a second adjustment request, and the sixth encoded video stream being a video stream transmitted by the first terminal through encoding at a fourth encoding rate after receiving the second adjustment request; the second adjustment request is sent to the first terminal by the server, and the fifth coded video stream and the sixth coded video stream meet the network channel bandwidth indicated by the second adjustment request;
decoding the fifth coded video stream through the first decoding module to obtain a fifth decoded video stream, and decoding the sixth coded video stream through the second decoding module to obtain a sixth decoded video stream; the decoding rate of the first decoding module is a third decoding rate, and the decoding rate of the second decoding module is a fourth decoding rate;
displaying the fifth decoded video stream in a first display area of the second display screen, and displaying the sixth decoded video stream in a second display area of the second display screen, wherein the first display area is larger than the second display area.
28. The terminal of claim 26, wherein the processor is further configured to:
after receiving the third encoded video stream and the fourth encoded video stream transmitted by the first terminal, the method further includes:
receiving, by the transceiver, a seventh encoded video stream and an eighth encoded video stream, the seventh encoded video stream being a video stream transmitted by the first terminal at a fourth encoding rate after receiving a second adjustment request, the eighth encoded video stream being a video stream encoded by the first terminal at the fourth encoding rate after receiving the second adjustment request; the second adjustment request is sent to a second terminal by a server, and the seventh coded video stream and the eighth coded video stream meet the network channel bandwidth indicated by the second adjustment request;
decoding the seventh coded video stream through the second decoding module to obtain a seventh decoded video stream, and decoding the eighth coded video stream through the first decoding module to obtain an eighth decoded video stream; the decoding rate of the first decoding module is a third decoding rate, and the decoding rate of the second decoding module is a fourth decoding rate;
displaying the eighth decoded video stream in a first display area of the second display screen, and displaying the seventh decoded video stream in a second display area of the second display screen, wherein the first display area is larger than the second display area.
29. The terminal according to any of claims 25 to 28, characterized in that the terminal further comprises an audio acquisition module; the processor is further configured to:
capturing, by the audio capture module, a first audio stream after the video stream encoded at the first encoding rate is displayed in the first display area of the second display screen and the second decoded video stream encoded at the second encoding rate is displayed in the second display area of the second display screen;
and sending the first audio stream to the first terminal through the transceiver, wherein the first audio stream is used for playing audio by the first terminal.
30. A terminal according to any of claims 25 to 29, further comprising a third camera module, an encoder; the first video stream is also used for the first terminal to display in a third display area of the first display screen, and the second video stream is also used for the first terminal to display in a fourth display area of the first display screen; the processor is further configured to:
after the video stream coded by the first coding rate is displayed in a first display area of the second display screen, the second decoded video stream coded by the second coding rate is displayed in a second display area of the second display screen, a feedback video stream is collected by the third camera module;
displaying the feedback video stream in a sixth display area of the second display screen, wherein the sixth display area is smaller than the first display area;
encoding the feedback video stream by the encoder to obtain a feedback encoded video stream;
and sending the feedback coded video stream to the first terminal through a transceiver, wherein the feedback coded video stream is used for the first terminal to display in a third display area of the first display screen, the first video stream is also used for displaying in a fifth display area of the first display screen of the first terminal, and the fifth display area is smaller than the third display area.
31. A server, comprising a memory, a processor, and a transceiver, wherein the memory stores a program, and the processor invokes the program to perform the operations of:
receiving, by the transceiver, a first encoded video stream and a second encoded video stream sent by the first terminal, where the first encoded video stream is a video stream obtained by encoding the first video stream acquired by the first terminal at a first encoding rate, the second encoded video stream is a video stream obtained by encoding the second video stream acquired by the first terminal at a second encoding rate, and the first encoding rate is greater than the second encoding rate;
forwarding, by the transceiver, the first encoded video stream and the second encoded video stream to the second terminal; wherein the video stream encoded at the first encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the second encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
32. The server according to claim 31, wherein the processor is further configured to:
after receiving a first encoded video stream and a second encoded video stream transmitted by the first terminal through the transceiver, receiving a first adjustment request transmitted by a second terminal through the transceiver;
forwarding, by the transceiver, the first adjustment request to the first terminal, the first adjustment request for the first terminal to adjust an encoding rate;
receiving, by the transceiver, a third encoded video stream and a fourth encoded video stream transmitted by the first terminal;
receiving, by the transceiver, a third encoded video stream and a fourth encoded video stream transmitted by the first terminal, where the third encoded video stream is a video stream encoded by the first terminal at a second encoding rate after receiving the first adjustment request, and the fourth encoded video stream is a video stream encoded by the first terminal at a first encoding rate after receiving the first adjustment request;
transmitting, by the transceiver, the third encoded video stream and the fourth encoded video stream to the second terminal; wherein the video stream encoded at the first encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the second encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
33. The server according to claim 31, wherein the processor is further configured to:
after receiving a first coded video stream and a second coded video stream sent by the first terminal through the transceiver, analyzing a network channel bandwidth to obtain a second adjustment request;
transmitting, by the transceiver, the second adjustment request to a first terminal, the second adjustment request being for the first terminal to adjust a coding rate;
receiving, by the transceiver, a fifth encoded video stream and a sixth encoded video stream, the fifth encoded video stream being a video stream transmitted by the first terminal at a third encoding rate after receiving a second adjustment request, the sixth encoded video stream being a video stream encoded by the first terminal at a fourth encoding rate after receiving the second adjustment request; wherein the third encoding rate is greater than the fourth encoding rate, and the fifth encoded video stream and the sixth encoded video stream satisfy the network channel bandwidth indicated by the second adjustment request;
forwarding, by the transceiver, the fifth encoded video stream and the sixth encoded video stream to the second terminal; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
34. The server according to claim 31, wherein the processor is further configured to:
after the third coded video stream and the fourth coded video stream sent by the first terminal are received through the transceiver, analyzing network bandwidth information to obtain a second adjustment request;
transmitting, by the transceiver, the second adjustment request to a first terminal, the second adjustment request being for the first terminal to adjust a coding rate;
receiving, by the transceiver, a seventh encoded video stream and an eighth encoded video stream, the seventh encoded video stream being a video stream transmitted by the first terminal at a fourth encoding rate after receiving a second adjustment request, the eighth encoded video stream being a video stream encoded by the first terminal at a third encoding rate after receiving the second adjustment request; wherein the third encoding rate is greater than the fourth encoding rate, and the seventh encoded video stream and the eighth encoded video stream satisfy the network channel bandwidth indicated by the second adjustment request;
forwarding, by the transceiver, the seventh encoded video stream and the eighth encoded video stream to the second terminal; wherein the video stream encoded at the third encoding rate is for display in a first display area of a second display screen of the second terminal and the video stream encoded at the fourth encoding rate is for display in a second display area of the second display screen of the second terminal, the first display area being larger than the second display area.
35. A server as claimed in any one of claims 31 to 34, wherein the processor is further configured to:
after said forwarding, by said transceiver, said first encoded video stream and said second encoded video stream to a second terminal, further comprising:
receiving, by the transceiver, a first audio stream transmitted by the second terminal;
and forwarding the first audio stream to the first terminal through the transceiver, wherein the first audio stream is used for playing audio by the first terminal.
36. The server according to any one of claims 31 to 35, wherein the first video stream is further for display by the first terminal in a third display area of a first display screen, and the second video stream is further for display by the first terminal in a fourth display area of the first display screen; the processor is further configured to:
after the first coded video stream and the second coded video stream are forwarded to a second terminal through the transceiver, receiving a feedback coded video stream sent by the second terminal through the transceiver, wherein the feedback coded video stream is a video stream obtained by coding a feedback video stream collected by the second terminal;
forwarding the feedback encoded video stream to the first terminal through the transceiver, where the feedback video stream is used for the first terminal to display in a third display area of the first display screen, and the first video stream is also used for the first terminal to display in a fifth display area of the first display screen, where the fifth display area is smaller than the third display area.
37. A computer-readable storage medium, in which a computer program is stored which, when run on a processor, implements the method of any one of claims 1 to 19.
CN201910947188.3A 2019-09-29 2019-09-29 Video interaction method and related equipment Pending CN110650341A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910947188.3A CN110650341A (en) 2019-09-29 2019-09-29 Video interaction method and related equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910947188.3A CN110650341A (en) 2019-09-29 2019-09-29 Video interaction method and related equipment

Publications (1)

Publication Number Publication Date
CN110650341A true CN110650341A (en) 2020-01-03

Family

ID=68993635

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910947188.3A Pending CN110650341A (en) 2019-09-29 2019-09-29 Video interaction method and related equipment

Country Status (1)

Country Link
CN (1) CN110650341A (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20010073665A (en) * 2000-01-19 2001-08-01 오길록 Apparatus and method of controlling time code modification and rate buffer level for low-delay video communication
CN101742223A (en) * 2010-01-19 2010-06-16 上海华平信息技术股份有限公司 Intelligent video stream bandwidth regulating method
CN103024342A (en) * 2012-12-25 2013-04-03 广东威创视讯科技股份有限公司 Bandwidth control method and system for video conference system
CN104270597A (en) * 2010-04-07 2015-01-07 苹果公司 Establishing A Video Conference During A Phone Call
CN107528938A (en) * 2017-07-26 2017-12-29 维沃移动通信有限公司 A kind of video call method, terminal and computer-readable recording medium
CN109195010A (en) * 2018-08-15 2019-01-11 咪咕视讯科技有限公司 A kind of code rate adjustment method and device

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20010073665A (en) * 2000-01-19 2001-08-01 오길록 Apparatus and method of controlling time code modification and rate buffer level for low-delay video communication
CN101742223A (en) * 2010-01-19 2010-06-16 上海华平信息技术股份有限公司 Intelligent video stream bandwidth regulating method
CN104270597A (en) * 2010-04-07 2015-01-07 苹果公司 Establishing A Video Conference During A Phone Call
CN103024342A (en) * 2012-12-25 2013-04-03 广东威创视讯科技股份有限公司 Bandwidth control method and system for video conference system
CN107528938A (en) * 2017-07-26 2017-12-29 维沃移动通信有限公司 A kind of video call method, terminal and computer-readable recording medium
CN109195010A (en) * 2018-08-15 2019-01-11 咪咕视讯科技有限公司 A kind of code rate adjustment method and device

Similar Documents

Publication Publication Date Title
US10728594B2 (en) Method and apparatus for transmitting data of mobile terminal
US11051015B2 (en) Low bitrate encoding of panoramic video to support live streaming over a wireless peer-to-peer connection
US11184584B2 (en) Method for image decoding, method for image encoding, apparatus for image decoding, apparatus for image encoding
US10237548B2 (en) Video transmission based on independently encoded background updates
US20110285863A1 (en) Live television broadcasting system for the internet
CN103888699A (en) Projection device with video function and method for video conference by using same
US20170163934A1 (en) Data, multimedia & video transmission updating system
KR19990064087A (en) Video speech decoding apparatus, video speech coding apparatus and information transmission system
CN103686219A (en) Method, device and system for recording and broadcasting video conference
KR20180035089A (en) Providing virtual reality service considering region of interest
US6526100B1 (en) Method for transmitting video images, a data transmission system and a multimedia terminal
CN108696720B (en) Video scheduling system and method suitable for satellite communication
CN117176972B (en) Cloud conference audio and video transmission system and method based on WebRTC technology
US20130291011A1 (en) Transcoding server and method for overlaying image with additional information therein
JP6466638B2 (en) Terminal, system, program, and method for thinning frames of a captured moving image according to a motion change amount
CN111083427B (en) Data processing method of embedded terminal and 4K video conference system
KR20120009948A (en) Apparatus and method for generating a three-dimension image data in portable terminal
CN110650341A (en) Video interaction method and related equipment
CN107332818B (en) Method, device and system for processing media data, storage medium and processor
CN115706829A (en) Multi-window video communication method, device and system
US10616620B1 (en) Low bitrate encoding of spherical video to support live streaming over a high latency and/or low bandwidth network
CN112272281B (en) Regional distributed video conference system
KR20180096399A (en) Method, device and non-trnasitory computer-readable recording media for supporting relay broadcasting
CN104702970A (en) Video data synchronization method, device and system
KR100233531B1 (en) Active picture size controlling method and apparatus according to channel status

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20200103

RJ01 Rejection of invention patent application after publication