WO2019071808A1 - 视频画面显示的方法、装置、系统、终端设备及存储介质 - Google Patents
视频画面显示的方法、装置、系统、终端设备及存储介质 Download PDFInfo
- Publication number
- WO2019071808A1 WO2019071808A1 PCT/CN2017/116628 CN2017116628W WO2019071808A1 WO 2019071808 A1 WO2019071808 A1 WO 2019071808A1 CN 2017116628 W CN2017116628 W CN 2017116628W WO 2019071808 A1 WO2019071808 A1 WO 2019071808A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- user
- data
- terminal
- video
- speaking
- Prior art date
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
- H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/15—Conference systems
Definitions
- the present invention relates to the field of real-time video communication, and in particular, to a video picture display method, device, system, terminal device and storage medium.
- Video communication is a very wide range of applications in existing instant messaging applications.
- the user video chats with one or more other users through the terminal device.
- the chat process only the multi-person video window or the full-screen display may be displayed on the screen of the terminal device.
- a person's video preview when the video screen needs to be switched, the user needs to display the corresponding person's video window through the toggle button.
- an object of the present invention is to provide a video picture display method, apparatus, system, terminal device and storage medium, which can improve the flexibility of remote video picture switching and simplify user operations.
- the present invention provides a method for displaying a video picture, comprising the following steps:
- Corresponding interface is displayed according to user data in the user list.
- the displaying the corresponding interface according to the user data in the user list specifically:
- the user list is displayed on the current interface
- the compressed first video data corresponding to the user data generates a small window video preview on the current interface.
- the method further includes:
- the invention also provides a method for displaying a video picture, comprising the following steps:
- the audio feature comprises at least one of the following: a decibel value, a timbre, a tone, a voiceprint.
- the audio feature is a decibel value
- the determining, according to the audio feature, the speaking state of each user terminal, and generating the terminal state change notification according to the speaking state of each user terminal specifically:
- the speaking state of the user terminal is marked as being speaking
- a terminal state change notification is generated according to the speaking state of each user terminal.
- valid audio data is selected from the audio data according to a speaking state of each user terminal, and the valid audio data is transmitted to each user terminal.
- the multimedia data further includes video data uploaded by each user terminal, and the method for switching the remote video image further includes:
- the video data corresponding to the user terminal is subjected to compression processing, and the first video data obtained by the compression processing is sent to the respective user terminals;
- the second video data is transmitted to the user terminal when receiving a request from one of the user terminals to transmit the second video data that is not subjected to the compression processing corresponding to the user data in the user list of the user terminal.
- the present invention also provides an apparatus for displaying a video picture, comprising:
- a status receiving module configured to receive a terminal status change notification from the server
- an update module configured to update the user list according to the received terminal state change notification, where the user list includes user data that is speaking;
- a display control module configured to display a corresponding interface according to the user data in the user list.
- the present invention also provides an apparatus for displaying a video picture, comprising:
- a data receiving module configured to receive multimedia data uploaded from each user terminal; wherein the multimedia data includes audio data;
- An extracting module configured to extract audio features of audio data uploaded by each user terminal
- a determining module configured to determine a speaking state of each user terminal according to the audio feature, and generate a terminal state change notification according to a speaking state of each user terminal;
- a sending module configured to send the terminal status change notification to the user terminals, so that the user terminals update the user list according to the received terminal status change notification, and according to the user data in the user list The corresponding interface is displayed.
- the present invention also provides a system for displaying a video picture, comprising at least two user terminals and at least one server, wherein:
- the user terminal is configured to send the collected multimedia data to a server; the multimedia data includes audio data;
- the server is configured to receive the multimedia data uploaded from each user terminal, extract audio features of the audio data in the multimedia data, and determine each user terminal according to the audio feature. After the terminal state change notification is generated according to the speaking state of each user, the terminal state change notification is sent to the respective user terminals;
- the user terminal is further configured to receive a terminal status change notification from the server, update the user list according to the terminal status change notification, and display a corresponding interface according to the user data in the user list; wherein the user list Includes user data that is being spoken.
- the present invention also provides a terminal device including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, the processor executing the computer program as described above A method of displaying a video picture as described in any of the above.
- the present invention also provides a computer readable storage medium comprising a stored computer program, wherein a device in which the computer readable storage medium is located is executed while the computer program is running to perform any of the above The method of video screen display as described in the item.
- the present invention provides a video screen display method, device, system, terminal device and storage medium.
- the user terminal updates the user list according to the terminal status change notification from the server, and displays the corresponding interface according to the user data in the user list.
- FIG. 1 is a schematic flow chart of a method for displaying a video screen according to a first embodiment of the present invention.
- FIG. 2 is a schematic diagram of a display interface of a method for displaying a video screen according to a first embodiment of the present invention.
- FIG. 3 is a schematic diagram of another display interface of a method for displaying a video screen according to a first embodiment of the present invention.
- FIG. 4 is a flow chart showing the display result of the method for displaying a video screen according to the second embodiment of the present invention.
- FIG. 5 is a schematic diagram of a display interface of a method for displaying a video screen according to a second embodiment of the present invention.
- FIG. 6 is a schematic flowchart diagram of a method for displaying a video screen according to a fourth embodiment of the present invention.
- FIG. 7 is a schematic structural diagram of an apparatus for displaying a video screen according to a seventh embodiment of the present invention.
- FIG. 8 is a schematic structural diagram of an apparatus for displaying a video screen according to an eighth embodiment of the present invention.
- a first embodiment of the present invention provides a video screen display method, which can be executed on a terminal device, and includes the following steps:
- the terminal device may be an electronic terminal having an interactive screen, such as a smart phone, a tablet computer, a personal computer, or a multimedia player.
- each of the terminal devices uploads audio data and video data collected by itself to a server, and the server extracts audio features (such as decibels) according to audio data in each terminal device. And generating a terminal state change notification according to the audio characteristics of each terminal device and transmitting to each of the terminal devices, each of the terminal devices receiving a terminal state change notification from the server.
- the terminal state change notification refers to that the speaking state of one or more terminals is speaking, or the terminal state change notification refers to that the speaking state of one or more terminals is a stop speech.
- each of the terminal devices maintains a user list.
- the server detects that the speaking state of one or more terminal devices is speaking, the server sends a notification to the terminal device, and after receiving the notification, the terminal device receives the notification.
- the user data is updated in its own user list, that is, the user data that is being spoken is added to the user list.
- the received terminal state change notification is that the speaking state of one or more terminal devices is a stop speech, the user data of the stop speaking is deleted from the user list.
- the current interface is directly The list of users is displayed.
- a small window video preview is generated on the current interface according to the compressed first video data corresponding to the user data from the server.
- a user who has multiple terminal devices currently speaks at the same time, that is, a user list of each terminal includes a plurality of user data. It can be understood that, at this time, it is not necessary to have a video screen of all users who are speaking. Displaying on the screen of the terminal device only needs to display a list of users including the plurality of user data on the current interface, as shown in FIG. 2 .
- a user of only one terminal device (hereinafter referred to as a first terminal device) is in a period of time, that is, only the user data of the first terminal device is included in the user list on each terminal device.
- the user corresponding to the currently displayed full screen video screen is the user who is speaking, the user does not need to perform any operation, and does not need to generate a corresponding small window video preview.
- the current interface of the terminal device may have the following three display schemes:
- the system defaults to display the host's video screen on the current interface in full screen.
- the host in general, the host is relatively the core of the conference. Therefore, during the conference, the moderator The terminal device uploads the video data of the host to the server, and the server sends the received video data of the moderator to each terminal device, and each of the terminal devices according to the received video data of the moderator The corresponding video screen is displayed in full screen.
- the system can display the background of the main screen on the current interface by default. Of course, the user can also customize the background image displayed on the current interface.
- the system can also display the video screen corresponding to the video preview of the small window on the current interface according to the user's wishes. For example, when the user clicks on the small window video preview, the user terminal requests video data corresponding to the small window video preview from the server and receives the video data, and then displays the video screen corresponding to the video data in full screen. .
- the present invention provides a video screen display method, in which a user terminal updates a user list according to a terminal state change notification from a server and displays a corresponding interface according to the user list, thereby improving remote video screen switching. Flexibility, effectively reducing user operations.
- the small window video preview is generated on the current interface, and the video screen of the speaking user is displayed through the small window video preview mode, thereby realizing automatic switching and displaying the video window, and fully utilizing the screen space.
- the server detects that another user of one or more terminal devices also starts to speak, and sends a new terminal state change notification to the respective terminal device.
- Each of the terminal devices updates the user list according to the received new terminal state change notification, that is, the user list of each terminal device newly adds user data of other terminal devices that start to speak, if the first terminal device The user continues to speak, that is, the user data of the first terminal device is also displayed on the user list, and the second user list including the newly added user data is displayed below the first small window video preview, as shown in FIG. 5. Shown.
- the user list may be a list of names of users who are speaking or a list of device names whose status is the user terminal that is speaking.
- the method further includes:
- the second terminal device when the user of the second terminal device (which may be the first terminal device or other terminal device) clicks the small input device (mouse or keyboard, etc.) or a touch screen or the like
- the second terminal device sends a request for complete video data corresponding to the small window video preview to the server, where the service is
- the second video data uploaded by the terminal device corresponding to the small window video preview is completely transmitted to the second terminal device without any compression processing.
- the second terminal device After receiving the second video data, displays a video screen corresponding to the second video data in full screen on the screen.
- the video screen displayed by the terminal device in full screen is still the video image corresponding to the first user, and is not automatically Switch the video screen on the current interface, for example, it will not switch back to the host's video screen.
- the video presentation surface of the speaking user is displayed first, and then the user decides whether the video screen needs to be displayed in full screen according to his or her own desire, thereby ensuring the accuracy of the video screen switching and realizing the interaction.
- a fourth embodiment of the present invention further provides a video screen display method, which can be executed on a server, and includes the following steps:
- S21 Collect audio data and video data of itself uploaded from each user terminal.
- each user terminal can upload audio data and video data by wire or wirelessly.
- each user terminal participating in the remote video collects the user's voice through the microphone of the terminal device or an external microphone, and collects the video image of the user through the camera provided by the terminal device or an external camera, and the user is recorded in real time.
- the sound (audio data) and the video picture (video data) are uploaded to a server, which can store or process the audio data and video data.
- the audio feature comprises at least one of the following: a decibel value, a timbre, a tone, and a voiceprint.
- the terminal device transmits a voice of the user (hereinafter referred to as first audio data) to the server, and the server pairs the collected first audio data.
- a detection is performed, volume information in the first audio data is extracted, and the volume information is converted into an audio feature (eg, a decibel value).
- the audio feature is a decibel value, specifically,
- the speaking state of the user terminal is marked as being speaking
- a terminal state change notification is generated according to the speaking state of each user terminal.
- the specific threshold, the shortest talk duration, and the maximum silence duration are preset in the storage unit in the server, and the server extracts the decibel value corresponding to each terminal device and the specific threshold, The shortest talk duration and the maximum silence duration are compared, and then the state change of each terminal device is judged, and the terminal state change notification can be generated according to the state change of each terminal device.
- the server detects that the volume of the sound in the audio data uploaded by the third terminal device is large (the decibel value is higher than the specific Threshold) and the duration of the user's speaking exceeds the shortest speaking duration, marking the state of the third terminal device as being speaking, and generating a notification that the user of the third terminal device is speaking; of course, assuming that the user of the fourth terminal device is finished After speaking, the server stops talking.
- the server detects that there is no sound in the audio data uploaded by the fourth terminal device or the volume of the sound is small (the decibel value is lower than a certain threshold) and lasts for a period of time (greater than the maximum silence duration). Then, the status of the fourth terminal device is marked as stopping the speaking, and a notification that the user of the fourth terminal device has stopped speaking is generated.
- the server can judge whether the sound emitted by the human or the vocalized object is generated according to different timbres. sound.
- the server may preset a specific frequency, and the server compares the frequency of the tone extracted from the audio data with the specific frequency to determine the user who is speaking.
- the server can also determine the user who is speaking according to the degree of matching of the voiceprint.
- the terminal state change notification is sent to the user terminals, so that the user terminals notify the update user list according to the received terminal state change notification, and display the corresponding interface according to the user data in the user list.
- the server sends the first terminal state change notification to each terminal device by wire or wireless manner, the each terminal After receiving the first terminal state change notification, adding user data of the third terminal device to the user list; of course, if the second terminal state change notification is that the user of the fourth terminal device stops speaking, the server will The second terminal state change notification is sent to each terminal device, and each of the terminal devices after receiving the second terminal state change notification The user data of the fourth terminal device is deleted on the user list.
- the server sends the audio data collected by each terminal device to each terminal device, and the terminal device plays the audio data, but in some cases, the audio data collected by some terminal devices may be invalid. Audio data (such as noise), if the terminal device plays the audio data, it will affect the user's listening experience.
- the method further includes:
- the valid audio data is selected from the audio data according to the speaking state of each user terminal, and the valid audio data is transmitted to each user terminal.
- the server may determine whether the audio data video is junk audio data according to the extracted audio features, and may block or delete the junk audio data. For example, if the server detects that the decibel value of the extracted second audio data exceeds a preset noise standard value, recording the second audio data as junk audio data, and shielding the junk audio data, the video may be improved. The quality of a chat or video conference. The server may mark other audio data than the junk audio data as valid audio data.
- the valid audio data may refer to audio data corresponding to all users who are speaking.
- the server extracts an audio feature (such as a decibel value) from the audio data uploaded from each user terminal to mark the current speaking state as the user terminal that is speaking, and marks the audio data corresponding to the speaking user terminal as valid audio data. Transmitting the valid video data to each terminal device, and the respective terminal devices play the received valid audio data. For example, when the server marks that the speaking state of the third terminal device is speaking, the audio data corresponding to the third terminal device is marked as the first valid audio data, and the first valid audio data is sent to each Terminal Equipment. Each of the terminal devices plays the received first valid audio data, that is, plays the sound of the third terminal device user.
- an audio feature such as a decibel value
- the method further includes:
- the video data corresponding to the user terminal is subjected to compression processing, and the first video data obtained by the compression processing is sent to the respective user terminals;
- the video image displayed on the small window video preview does not require high resolution, which can effectively save network resources and system resources.
- the server can compress the video data through a video encoder.
- the server compresses the complete video data uploaded by the first terminal device through the video encoder, and sends the first video data obtained by the compression process to each terminal.
- the device, and each terminal device generates a first small window video preview of the video picture of the first video data. If the user of the second terminal device clicks on the first small window video preview, the second terminal device requests the server for complete video data of the first terminal device, and after receiving the request, the server will use the first terminal.
- the complete video data of the device is sent to the second terminal device, and the second terminal device displays the received complete video data in full screen.
- a seventh embodiment of the present invention further provides an apparatus for displaying a video screen, including:
- the status receiving module 11 is configured to receive a terminal status change notification from the server.
- the updating module 12 is configured to update the user list according to the received terminal status change notification, where the user list displays the user data that is speaking;
- the display control module 13 is configured to display a corresponding interface according to user data in the user list.
- the display control module 13 specifically includes:
- a first display control unit configured to display the user list on the current interface when the number of user data in the user list is greater than one
- a second display control unit configured to generate a small window video on the current interface according to the compressed first processed video data corresponding to the user data when the number of user data in the user list is equal to Preview.
- the display control module 13 is further configured to:
- the display control module 13 is further configured to:
- an eighth embodiment of the present invention further provides an apparatus for displaying a video screen, including:
- the data receiving module 21 is configured to receive multimedia data uploaded from each user terminal, where the multimedia data includes audio data;
- the extracting module 22 is configured to extract audio features of the audio data uploaded by the user terminals;
- a determining module 23 configured to determine a speaking state of each user terminal according to the audio feature, and generate a terminal state change notification according to a speaking state of each user terminal;
- the sending module 24 is configured to send the terminal status change notification to the user terminals, so that the respective user terminals notify the update user list according to the received terminal status change, and according to the user in the user list.
- the data shows the corresponding interface.
- the audio feature is a decibel value
- the determining module 23 specifically includes:
- a first marking unit configured to mark a speaking state of the user terminal as being speaking when the decibel value is higher than a specific threshold and the duration is greater than a shortest speaking duration
- a second marking unit configured to mark a speaking state of the user terminal as stopping the speaking when the decibel value is lower than a specific threshold and the duration is greater than a maximum silence duration
- the notification unit is configured to generate a terminal state change notification according to the speaking state of each user terminal.
- the determining module 23 is further configured to:
- the valid audio data is obtained according to the speaking state of each user terminal, and the valid audio data is transmitted to each user terminal.
- the multimedia data further includes video data uploaded by each user terminal, and the sending module 24 is further configured to:
- the video data corresponding to the user terminal is compressed, and the compressed first video data is sent to the user terminals;
- the second video data is transmitted to the user terminal when receiving a request from one of the user terminals to transmit the second video data that is not subjected to the compression processing corresponding to the user data in the user list of the user terminal.
- the audio feature comprises at least one of the following: a decibel value, a timbre, a tone, a voiceprint.
- a ninth embodiment of the present invention further provides a system for displaying a video picture, comprising at least two user terminals and at least one server, wherein:
- the user terminal is configured to send the collected multimedia data to a server; the multimedia data Including audio data;
- the server is configured to receive the multimedia data uploaded from each user terminal, extract audio features of the audio data in the multimedia data, determine a speaking state of each user terminal according to the audio feature, and according to the user After the terminal state generates the terminal state change notification, the terminal state change notification is sent to the respective user terminals;
- the user terminal is further configured to receive a terminal status change notification from the server, update the user list according to the terminal status change notification, and display a corresponding interface according to the user data in the user list; wherein the user list Includes user data that is being spoken.
- the audio feature is a decibel value
- the server is further configured to mark the speaking state of the user terminal as being speaking when the decibel value is higher than a specific threshold and the duration is greater than a shortest speaking duration;
- the decibel value is lower than a certain threshold and the duration is greater than the maximum silence duration, the speaking state of the user terminal is marked as stopping the speech; and the terminal state change notification is generated according to the speaking state of each user terminal.
- the multimedia data further includes video data uploaded by each user terminal, and the server is further configured to: when the number of user terminals in which the speaking state is being uttered is 1, the video corresponding to the user terminal is Performing a compression process on the data, and transmitting the first video data obtained by the compression process to the user terminals;
- the user terminal is further configured to: when the number of user data in the user list is greater than 1, display the user list on the current interface; when the number of user data in the user list is equal to 1, according to the server
- the compressed first video data corresponding to the user data generates a small window video preview on the current interface.
- the user terminal is further configured to: when the user list is updated according to the user terminal change notification from the server, detecting that the user data corresponding to the current small window video preview does not stop speaking and detecting the user list If the number of user data in the user is greater than 1, the user list containing the newly added user data is displayed on the current interface.
- the user terminal is further configured to: when detecting an event of clicking the small window video preview, requesting, by the server, second video data that is not subjected to compression processing corresponding to the small window video preview;
- the server is further configured to send the second video data when receiving a request from one of the user terminals to send the second video data that is not subjected to the compression processing corresponding to the user data in the user list of the user terminal To the user terminal;
- the user terminal is further configured to receive second video data returned by the server according to the request, and generate a full screen display screen according to the second video data.
- the server is further configured to: select valid audio data from the audio data according to a speaking state of each user terminal, and send the valid audio data to each user terminal.
- the audio feature comprises at least one of the following: a decibel value, a timbre, a tone, a voiceprint.
- a tenth embodiment of the present invention further provides a terminal device for displaying a video screen.
- the terminal device of the video screen display of this embodiment includes a processor, a display, a memory, and a program stored in the computer program that can be executed on the processor, such as a video screen display.
- the processor implements the steps in the embodiments of the respective video picture display methods when the computer program is executed, such as step S11 shown in FIG.
- the processor executes the computer program, the functions of each unit in the foregoing device embodiments are implemented, such as the status receiving module 11 shown in FIG.
- the computer program can be partitioned into one or more modules that are stored in the memory and executed by the processor to perform the present invention.
- the one or more modules may be a series of computer program instruction segments capable of performing a particular function, the instruction segments being used to describe the execution of the computer program in the terminal device displayed on the video screen.
- the terminal device displayed on the video screen may be a computing device such as a desktop computer, a notebook, a palmtop computer, and a cloud server.
- the terminal device displayed by the video screen may include, but is not limited to, a processor, a memory, and a display. It can be understood by those skilled in the art that the schematic diagram is only an example of a terminal device for video screen display, does not constitute a limitation of a terminal device for displaying a video screen, may include more or less components than illustrated, or may combine some Some components, or different components, such as the terminal device of the video screen display, may also include input and output devices, network access devices, buses, and the like.
- the so-called processor can be a central processing unit (CPU), or other general-purpose processor, digital signal processor (DSP), application specific integrated circuit (ASIC), ready-made Field-Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc.
- the general purpose processor may be a microprocessor or the processor may be any conventional processor or the like, which is a control center of the terminal device of the video screen display, and connects the entire video by using various interfaces and lines. The various parts of the terminal device displayed on the screen.
- the memory can be used to store the computer program and/or module, by executing or executing a computer program and/or module stored in the memory, and calling the memory stored in the memory Data, implementing various functions of the terminal device displayed by the video screen.
- the memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application required for at least one function (such as a sound playing function, a text conversion function, etc.), and the like; the storage data area may be stored. Data created based on the use of the mobile phone (such as audio data, text message data, etc.).
- the memory may include a high-speed random access memory, and may also include non-volatile memory such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SMC), and a Secure Digital (SD) card.
- non-volatile memory such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SMC), and a Secure Digital (SD) card.
- Flash Card at least one disk storage device, flash memory device, or other volatile solid-state storage device.
- the terminal device integrated module displayed by the video screen can be stored in a computer readable storage medium if it is implemented in the form of a software functional unit and sold or used as a separate product. Based on such understanding, the present invention implements all or part of the processes in the foregoing embodiments, and may also be completed by a computer program to instruct related hardware.
- the computer program may be stored in a computer readable storage medium. The steps of the various method embodiments described above may be implemented when the program is executed by the processor.
- the computer program comprises computer program code, which may be in the form of source code, object code form, executable file or some intermediate form.
- the computer readable medium may include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a removable hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM). , random access memory (RAM, Random Access Memory), electrical carrier signals, telecommunications signals, and software distribution media. It should be noted that the content contained in the computer readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in a jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, computer readable media Does not include electrical carrier signals and telecommunication signals.
- the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical. Units can be located in one place or distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
- the connection relationship between the modules indicates that there is a communication connection between them, and specifically, one or more communication buses or signal lines can be realized.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- General Engineering & Computer Science (AREA)
- Information Transfer Between Computers (AREA)
- Telephonic Communication Services (AREA)
Abstract
Description
Claims (14)
- 一种视频画面显示的方法,其特征在于,包括:接收来自服务器的终端状态变化通知;根据接收的所述终端状态变化通知更新用户列表,所述用户列表包括正在发言的用户数据;根据所述用户列表中的用户数据显示对应界面。
- 根据权利要求1所述的视频画面显示的方法,其特征在于,所述根据所述用户列表中的用户数据显示对应界面,具体包括:当所述用户列表中的用户数据的数目大于1时,在当前界面显示所述用户列表;当所述用户列表中的用户数据的数目等于1时,根据来自服务器的与该用户数据对应的压缩处理过的第一视频数据,在当前界面上生成小窗视频预览。
- 根据权利要求2所述的视频画面显示的方法,其特征在于,在当所述用户列表中的用户数据的数目等于1时,根据来自服务器的与该用户数据对应的视频画面,在当前界面上生成小窗视频预览之后,还包括:当在根据来自服务器的用户终端变化通知更新所述用户列表时,检测到与当前小窗视频预览对应的用户数据未停止发言并且检测到所述用户列表中的用户数据的数目大于1,则将包含新增的用户数据的用户列表在当前界面进行显示。
- 根据权利要求2所述的切换远程视频的方法,其特征在于,还包括:当检测到点击所述小窗视频预览的事件时,向所述服务器请求与所述小窗视频预览对应的未经过压缩处理的第二视频数据;接收来自服务器根据请求返回的第二视频数据,并根据所述第二视频数据生成全屏显示画面。
- 一种视频画面显示的方法,其特征在于,包括:接收来自各个用户终端上传的多媒体数据;其中,所述多媒体数据包括音频数据;提取所述各个用户终端上传的音频数据的音频特征;根据所述音频特征确定各个用户终端的发言状态,并根据各个用户终端的发言状态生成终端状态变化通知;将所述终端状态变化通知发送给所述各个用户终端,以使所述各个用户终端根据接收的所述终端状态变化通知更新用户列表,并根据所述用户列表中的用户数据显示对应界面。
- 根据权利要求5所述的视频画面显示的方法,其特征在于,所述音频特征至少包括以下其中之一:分贝值、音色、音调、声纹。
- 根据权利要求5所述的视频画面显示的方法,其特征在于,所述音频特征为分贝值,则所述根据所述音频特征确定各个用户终端的发言状态,并根据各个用户终端的发言状态生成终端状态变化通知,具体包括:当所述分贝值高于特定阈值且持续时间大于最短说话时长时,标记此用户终端的发言状态为正在发言;当所述分贝值低于特定阈值且持续时间大于最大沉默时长时,标记此用户终端的发言状态为停止发言;根据各个用户终端的发言状态生成终端状态变化通知。
- 根据权利要求5至7任意一项所述的视频画面显示的方法,其特征在于,还包括:根据各个用户终端的发言状态从所述音频数据中选取出有效音频数据,并将所述有效音频数据发送到各个用户终端。
- 根据权利要求5所述的视频画面显示的方法,其特征在于,所述多媒体数据还包括各个用户终端上传的视频数据,则所述视频画面显示的方法还包括:当检测到发言状态为正在发言的用户终端的数量为1时,将与该用户终端对应的视频数据进行压缩处理,并将所述压缩处理得到的第一视频数据发送到所述各个用户终端;当接收到来自其中一个用户终端发送与该用户终端的用户列表中的用户数据对应的未进行压缩处理的第二视频数据的请求时,将所述第二视频数据发送到该用户终端。
- 一种视频画面显示的装置,其特征在于,包括:状态接收模块,用于接收来自服务器的终端状态变化通知;更新模块,用于根据接收的所述终端状态变化通知更新用户列表,所述用户列表包括正在发言的用户数据;显示控制模块,用于根据所述用户列表中的用户数据显示对应界面。
- 一种视频画面显示的装置,其特征在于,包括:数据接收模块,用于接收来自各个用户终端上传的自身的音频数据和视频数据;提取模块,用于提取所述各个用户终端上传的音频数据的音频特征;确定模块,用于根据所述音频特征确定各个用户终端的发言状态,并根据各个用户终端的发言状态生成终端状态变化通知;发送模块,用于将所述终端状态变化通知发送给所述各个用户终端,以使所述各个用户终端根据接收的所述终端状态变化通知更新用户列表,并根据所述用户列表中的用户数据显示对应界面。
- 一种视频画面显示的系统,其特征在于,包括至少两个用户终端及至少一个服务器,其中:所述用户终端,用于将采集的多媒体数据发送给服务器;所述多媒体数据包括音频数据;所述服务器,用于接收来自各个用户终端上传的所述多媒体数据,提取所述多媒体数据中的音频数据的音频特征,根据所述音频特征确定各个用户终端的发言状态,并根据所述各个用户的发言状态生成终端状态变化通知后,将所述终端状态变化通知发送给所述各个用户终端;所述用户终端,还用于接收来自所述服务器的终端状态变化通知,根据所述终端状态变化通知更新用户列表,并根据所述用户列表中的用户数据显示对应界面;其中,所述用户列表包括正在发言的用户数据。
- 一种终端设备,包括处理器、存储器以及存储在所述存储器中且被配置为由所述处理器执行的计算机程序,所述处理器执行所述计算机程序时实现如权利要求1至9中任意一项所述的视频画面显示的方法。
- 一种计算机可读存储介质,其特征在于,所述计算机可读存储介质包 括存储的计算机程序,其中,在所述计算机程序运行时控制所述计算机可读存储介质所在设备执行如权利要求1至9中任意一项所述的视频画面显示的方法。
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710949829.XA CN107682752B (zh) | 2017-10-12 | 2017-10-12 | 视频画面显示的方法、装置、系统、终端设备及存储介质 |
CN201710949829.X | 2017-10-12 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2019071808A1 true WO2019071808A1 (zh) | 2019-04-18 |
Family
ID=61139936
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2017/116628 WO2019071808A1 (zh) | 2017-10-12 | 2017-12-15 | 视频画面显示的方法、装置、系统、终端设备及存储介质 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN107682752B (zh) |
WO (1) | WO2019071808A1 (zh) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20230065847A1 (en) * | 2021-08-31 | 2023-03-02 | International Business Machines Corporation | Network bandwidth conservation during video conferencing |
Families Citing this family (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109963107B (zh) * | 2019-02-20 | 2021-10-08 | 视联动力信息技术股份有限公司 | 一种音视频数据的显示方法和系统 |
CN110996021A (zh) * | 2019-11-30 | 2020-04-10 | 咪咕文化科技有限公司 | 导播切换方法、电子设备和计算机可读存储介质 |
CN113784151B (zh) * | 2020-06-10 | 2024-05-17 | 腾讯科技(深圳)有限公司 | 一种数据处理方法、装置、计算机设备及存储介质 |
CN112383738B (zh) * | 2020-11-11 | 2023-03-03 | 浙江讯盟科技有限公司 | 一种省流量及资源消耗低的多人音视频会议方法和系统 |
CN114697732A (zh) * | 2020-12-30 | 2022-07-01 | 华为技术有限公司 | 一种拍摄方法、系统及电子设备 |
CN113596349B (zh) * | 2021-07-26 | 2024-06-04 | 世邦通信股份有限公司 | 发言位自动联动视频的会议方法及系统、装置与存储介质 |
CN113923359A (zh) * | 2021-10-13 | 2022-01-11 | 宁波米福软件有限公司 | 一种庭审现场画面与远程画面的融合方法及融合系统 |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101442654A (zh) * | 2008-12-26 | 2009-05-27 | 深圳华为通信技术有限公司 | 视频通信中视频对象切换的方法、装置及系统 |
CN102404542A (zh) * | 2010-09-09 | 2012-04-04 | 华为终端有限公司 | 多屏视频会议中对与会者图像显示进行调整的方法及装置 |
CN102647578A (zh) * | 2011-02-17 | 2012-08-22 | 鸿富锦精密工业(深圳)有限公司 | 视频切换系统及方法 |
US20130010049A1 (en) * | 2011-07-08 | 2013-01-10 | Adel Mostafa | Negotiate multi-stream continuous presence |
CN103297743A (zh) * | 2012-03-05 | 2013-09-11 | 联想(北京)有限公司 | 一种视频会议显示窗口调整方法及视频会议服务设备 |
CN104038725A (zh) * | 2010-09-09 | 2014-09-10 | 华为终端有限公司 | 多屏视频会议中对与会者图像显示进行调整的方法及装置 |
CN105791738A (zh) * | 2014-12-15 | 2016-07-20 | 深圳Tcl新技术有限公司 | 视频会议中视频窗口的调整方法及装置 |
CN106063255A (zh) * | 2014-02-27 | 2016-10-26 | 谷歌公司 | 显示视频会议期间的演讲者 |
-
2017
- 2017-10-12 CN CN201710949829.XA patent/CN107682752B/zh active Active
- 2017-12-15 WO PCT/CN2017/116628 patent/WO2019071808A1/zh active Application Filing
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101442654A (zh) * | 2008-12-26 | 2009-05-27 | 深圳华为通信技术有限公司 | 视频通信中视频对象切换的方法、装置及系统 |
CN102404542A (zh) * | 2010-09-09 | 2012-04-04 | 华为终端有限公司 | 多屏视频会议中对与会者图像显示进行调整的方法及装置 |
CN104038725A (zh) * | 2010-09-09 | 2014-09-10 | 华为终端有限公司 | 多屏视频会议中对与会者图像显示进行调整的方法及装置 |
CN102647578A (zh) * | 2011-02-17 | 2012-08-22 | 鸿富锦精密工业(深圳)有限公司 | 视频切换系统及方法 |
US20130010049A1 (en) * | 2011-07-08 | 2013-01-10 | Adel Mostafa | Negotiate multi-stream continuous presence |
CN103297743A (zh) * | 2012-03-05 | 2013-09-11 | 联想(北京)有限公司 | 一种视频会议显示窗口调整方法及视频会议服务设备 |
CN106063255A (zh) * | 2014-02-27 | 2016-10-26 | 谷歌公司 | 显示视频会议期间的演讲者 |
CN105791738A (zh) * | 2014-12-15 | 2016-07-20 | 深圳Tcl新技术有限公司 | 视频会议中视频窗口的调整方法及装置 |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20230065847A1 (en) * | 2021-08-31 | 2023-03-02 | International Business Machines Corporation | Network bandwidth conservation during video conferencing |
Also Published As
Publication number | Publication date |
---|---|
CN107682752A (zh) | 2018-02-09 |
CN107682752B (zh) | 2020-07-28 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2019071808A1 (zh) | 视频画面显示的方法、装置、系统、终端设备及存储介质 | |
US11683278B2 (en) | Spectrogram and message bar generation based on audio data in an instant messaging application | |
US8970662B2 (en) | Output management for electronic communications | |
EP3282669A2 (en) | Private communications in virtual meetings | |
US7822050B2 (en) | Buffering, pausing and condensing a live phone call | |
US11474775B2 (en) | Sound effect adjustment method, device, electronic device and storage medium | |
US9544703B2 (en) | Detection of device configuration | |
US20220076688A1 (en) | Method and apparatus for optimizing sound quality for instant messaging | |
US20220291897A1 (en) | Method and device for playing voice, electronic device, and storage medium | |
CN106664433B (zh) | 多媒体信息播放方法及系统、标准化服务器、直播终端 | |
CN103973542B (zh) | 一种语音信息处理方法及装置 | |
US8868419B2 (en) | Generalizing text content summary from speech content | |
CN109120947A (zh) | 一种直播间的语音私聊方法及客户端 | |
US10313502B2 (en) | Automatically delaying playback of a message | |
US20190221226A1 (en) | Electronic apparatus and echo cancellation method applied to electronic apparatus | |
CN114845144B (zh) | 一种投屏方法、辅助投屏装置及存储介质 | |
CN111797271A (zh) | 多人听音乐实现方法、装置、存储介质及电子设备 | |
CN110162255B (zh) | 单机程序的运行方法、装置、设备及存储介质 | |
CN110767203B (zh) | 音频处理方法、装置及移动终端及存储介质 | |
WO2017101300A1 (zh) | 一种通话方法、装置及终端 | |
CN113284500B (zh) | 音频处理方法、装置、电子设备及存储介质 | |
TWI811692B (zh) | 用於場景音轉換的方法與裝置及電話系統 | |
CN111352605A (zh) | 一种音频播放、发送的方法及装置 | |
JP7417272B2 (ja) | 端末装置、サーバ装置、配信方法、学習器取得方法、およびプログラム | |
WO2023216119A1 (zh) | 音频信号编码方法、装置、电子设备和存储介质 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 17928307 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 17928307 Country of ref document: EP Kind code of ref document: A1 |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 21.10.2020) |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 17928307 Country of ref document: EP Kind code of ref document: A1 |