CN112312059A

CN112312059A - Information processing method and device for multimedia conference and electronic equipment

Info

Publication number: CN112312059A
Application number: CN202010368249.3A
Authority: CN
Inventors: 陈满砚; 丁力; 陈杰; 周敏明; 卢姝含; 朱斌; 许一峰; 喻宙; 何景煜
Original assignee: Beijing ByteDance Network Technology Co Ltd
Current assignee: Beijing ByteDance Network Technology Co Ltd
Priority date: 2020-04-30
Filing date: 2020-04-30
Publication date: 2021-02-02

Abstract

The embodiment of the disclosure discloses an information processing method and device for a multimedia conference and electronic equipment. One embodiment of the method comprises: in the process of the multimedia conference, whether a request of a user for applying preprocessing to the multimedia information stream of the local terminal is received or not is determined; in response to determining that the request is received, applying preprocessing corresponding to the request to the multimedia information stream of the user; transmitting the multimedia information stream to which the preprocessing is applied in real time. Therefore, the participating users can preprocess their own multimedia information streams in the multimedia conference. The operation of preprocessing the multimedia information flow of the participating users is simplified. The user experience may be improved.

Description

Information processing method and device for multimedia conference and electronic equipment

Technical Field

The embodiment of the disclosure relates to the technical field of internet, in particular to an information processing method and device for a multimedia conference and electronic equipment.

Background

The network multimedia conference can realize real-time transmission of information in the forms of moving images, voice, application data (electronic whiteboards, graphics) and the like between two points and multiple points. The method is suitable for teleconferencing, network discussion and the like. People at two or more locations can interact through a network multimedia conference.

The network multimedia conference mainly carries out information communication through voice and video, namely when a user participates in the multimedia conference, the terminal equipment sends the collected audio information and video information of the participating user to the server side, and the server side forwards the audio information and video information of the participating user.

Disclosure of Invention

This disclosure presents concepts in a simplified form that are described in detail below in the detailed description. This disclosure is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

Some embodiments of the present disclosure propose an information processing method, apparatus and electronic device for multimedia conferencing.

In a first aspect, some embodiments of the present disclosure provide an information processing method for a multimedia conference, including: in the process of the multimedia conference, whether a request of a user for applying preprocessing to the multimedia information stream of the local terminal is received or not is determined; in response to determining that the request is received, applying preprocessing corresponding to the request to the multimedia information stream of the user; transmitting the multimedia information stream to which the preprocessing is applied in real time.

In a second aspect, some embodiments of the present disclosure provide an information processing apparatus for multimedia conferencing, comprising: the determining unit is used for determining whether a request of a user for applying preprocessing to the multimedia information stream of the local terminal is received or not in the process of the multimedia conference; the preprocessing unit is used for responding to the received request and applying preprocessing corresponding to the request to the multimedia information flow of the user; a sending unit, configured to send the multimedia information stream to which the preprocessing is applied in real time.

In a third aspect, some embodiments of the present disclosure provide an electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon which, when executed by one or more processors, cause the one or more processors to perform a method according to the first aspect.

In a fourth aspect, some embodiments of the disclosure provide a computer readable medium having a computer program stored thereon, wherein the program when executed by a processor implements the method of the first aspect.

According to the information processing method, the information processing device and the electronic equipment for the multimedia conference, whether a request of a user for applying preprocessing to a multimedia information stream of a local terminal is received or not is determined in the process of the multimedia conference; in response to determining that the request is received, applying preprocessing corresponding to the request to the multimedia information stream of the user; transmitting the multimedia information stream to which the preprocessing is applied in real time. Therefore, the participating users can preprocess their own multimedia information streams in the multimedia conference. The operation of preprocessing the multimedia information flow of the participating users is simplified. The user experience may be improved.

Drawings

The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and features are not necessarily drawn to scale.

Fig. 1 is a flow diagram of some embodiments of an information processing method for multimedia conferencing according to the present disclosure;

FIGS. 2A-2B are schematic diagrams of an application scenario of an information display method according to the present disclosure;

FIG. 3 is a flow diagram of further embodiments of an information processing method for multimedia conferencing according to the present disclosure;

FIG. 4 is a schematic block diagram of some embodiments of an information processing apparatus for multimedia conferencing according to the present disclosure;

fig. 5 is an exemplary system architecture to which an information processing method for a multimedia conference of one embodiment of the present disclosure may be applied;

FIG. 6 is a schematic structural diagram of an electronic device suitable for use in implementing some embodiments of the present disclosure.

Detailed Description

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.

It should be understood that the various steps recited in the method embodiments of the present disclosure may be performed in a different order, and/or performed in parallel. Moreover, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this respect.

The term "include" and variations thereof as used herein are open-ended, i.e., "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions for other terms will be given in the following description.

It should be noted that the terms "first", "second", and the like in the present disclosure are only used for distinguishing different devices, modules or units, and are not used for limiting the order or interdependence relationship of the functions performed by the devices, modules or units.

It is noted that references to "a", "an", and "the" modifications in this disclosure are intended to be illustrative rather than limiting, and that those skilled in the art will recognize that "one or more" may be used unless the context clearly dictates otherwise.

The names of messages or information exchanged between devices in the embodiments of the present disclosure are for illustrative purposes only, and are not intended to limit the scope of the messages or information.

Referring to fig. 1, a flow diagram of some embodiments of an information processing method for multimedia conferencing according to the present disclosure is shown. The information processing method for the multimedia conference comprises the following steps:

step 101, in the process of the multimedia conference, determining whether a request of a user for applying preprocessing to the multimedia information stream of the local terminal is received.

In this embodiment, the participating users can participate in the multimedia conference through the terminal devices. A multimedia conference herein may include a plurality of participating users. Multiple participating users may not be in the same conference room. The terminal device may include, but is not limited to, a smart phone, a tablet computer, a laptop portable computer, a desktop computer, and the like.

The multimedia conference can be an audio conference or a video conference.

The information stream may comprise an audio data stream and/or a video data stream corresponding to the multimedia conference.

In a multimedia conference, terminal devices of participating users can collect multimedia information streams of the participating users through a multimedia information collection device. The multimedia information acquisition device can be a voice information acquisition device such as a microphone and a video information acquisition device such as a camera. The multimedia information acquisition device can be a multimedia information acquisition device which is built in the terminal equipment; or a multimedia information acquisition device which is arranged outside the terminal equipment and realizes wired or wireless communication connection with the terminal equipment.

In practice, a client of a multimedia conference application for conducting a multimedia conference may be installed in a terminal device. The user can participate in the multimedia conference through the client.

In the prior art, a user may set a relevant parameter of the preprocessing function in a parameter setting in a home page of a client (i.e., not in a multimedia conference). Under the control of the relevant parameters, relevant preprocessing can be carried out on a plurality of multimedia conferences using the client. That is, the parameters of the preprocessing of the global multimedia conference need to be set outside the multimedia conference in the prior art. The preprocessing parameters can enable the multimedia data streams corresponding to the multimedia conferences to be subjected to the same preprocessing according to the same preprocessing parameters in the multimedia conferences. In the above scheme, if the user forgets to set the relevant parameters of the preprocessing before the multimedia conference, the user needs to quit the multimedia conference, and the relevant parameters of the preprocessing are set in the relevant parameter setting items of the client. Therefore, for the participating users in the multimedia conference, the operation of the participating users is complicated by setting the relevant parameters of the preprocessing through the scheme. According to the scheme, the participating users can not flexibly set the relevant preprocessed parameters in the multimedia conference according to the requirements of the participating users.

In this embodiment, the participating users may display the information of the preprocessing function on the display interface of the multimedia conference during the multimedia conference. The user performs selection operation on the preprocessing function information, and then performs corresponding setting on the parameters for preprocessing the multimedia conference information stream, so as to request to preprocess the multimedia information stream, or cancel the preprocessing of the multimedia information stream.

The pre-processing may include pre-processing audio data for the multimedia conference and/or pre-processing video data for the multimedia conference. Further, the preprocessing of the audio data includes: reducing the noise of the environment; the preprocessing of the video data comprises beautifying and background blurring.

Specifically, an entry for starting/closing the preprocessing related parameters may be set in a presentation interface of the multimedia conference during the multimedia conference. After the user selects the above-mentioned entry, an option to set the start/stop preprocessing may be presented.

In some application scenarios, a preprocessed portal may be exposed on an exposure interface of the multimedia conference. After clicking on the above entry, a pre-processing control may be exposed.

In these application scenarios, the step 101 may include the following sub-steps:

firstly, monitoring whether the user performs selection operation on a preprocessing control displayed on a display interface corresponding to the multimedia conference or not in the process of the multimedia conference;

and secondly, in response to the fact that the user is monitored to perform selection operation on the preprocessing control displayed on the display interface corresponding to the multimedia conference in the process of the multimedia conference, determining that a request of the user for preprocessing the multimedia information stream of the local terminal is received.

The preprocessing control here may include, for example: a voice preprocessing control and a video preprocessing control. The preprocessing control may include an option button. The selection preprocessing can be realized by executing selection operation on the option button; or the pre-processing that has been selected is deselected to cancel the pre-processing.

In these application scenarios, when the user performs a selection operation on the selection button of the preprocessing control, it can be considered that the user wishes to preprocess the multimedia information stream at the home terminal. That is, a request for the user to pre-process the multimedia information stream of the home terminal is received.

Step 102, in response to determining that the request is received, applying a preprocessing corresponding to the request to the multimedia information stream of the user.

In step 101, after determining that a request for preprocessing a multimedia information stream of a local terminal by a user is received, preprocessing corresponding to the request may be applied to the multimedia information stream of the user acquired in real time.

For example, each video frame collected in real time is preprocessed by beautifying, blurring background, and the like. And for example, noise reduction processing is performed on audio data acquired in real time. The beautifying is performed by the collected face image of the user. Beautifying may include: adjusting a skin color of the user, adjusting a size of eyes of the user, adjusting an outline of a face of the user, and the like.

The background blurring is to blur and display a portion of the captured video frame including the user image, from which the user image is removed, so as to highlight the user image in the video frame.

The existing beautifying tools can be utilized to beautify the face image of the user, and various existing background blurring tools can be utilized to perform blurring setting on the background in the video frame.

In some optional implementations of this embodiment, the preprocessing is applied to the multimedia information stream of the local end according to the request only in the current multimedia conference.

That is, when the user initiates a request for applying the preprocessing to the multimedia information stream of the home terminal, and the user does not cancel the preprocessing in the multimedia conference, the duration of the preprocessing performed on the multimedia data may be the duration of the multimedia conference. After the multimedia conference is finished, the preprocessing of the multimedia data is invalid.

And 103, transmitting the multimedia information stream to which the preprocessing is applied in real time.

After applying the preprocessing corresponding to the request to the multimedia information stream of the user collected in real time in step 102, the terminal device may send the preprocessed multimedia information stream to the server. The server side can merge the preprocessed multimedia information stream of the user with the multimedia information streams of other users, and send the merged multimedia information stream to the terminal devices of a plurality of participating users. In the information processing method for the multimedia conference provided by this embodiment, it is determined whether a request for a user to preprocess a multimedia information stream of a home terminal is received or not in the process of the multimedia conference; in response to determining that the request is received, applying preprocessing corresponding to the request to the multimedia information stream of the user collected in real time; and sending the preprocessed multimedia information stream in real time, so that the participating users can preprocess the multimedia information stream of the participating users in the multimedia conference. The operation of preprocessing the multimedia information flow of the participating users is simplified. The user experience may be improved.

Please refer to fig. 2A to 2B, which are schematic diagrams of an application scenario of the information display method according to the present disclosure.

The participating user a1 uses the terminal device to participate in the multimedia conference. A presentation interface 202 of the multimedia conference can be presented in the terminal device display window 201.

Taking the multimedia conference as an example of a video conference, the video information of the participating users of the multimedia conference who speak currently can be displayed in the display interface 202. The participating users that are currently speaking may include participating user a 1. A function selection navigation bar 203 related to the multimedia conference can also be displayed in the display interface 202. The function selection navigation bar 203 may include special effects entries that preprocess the multimedia data stream of the participating user a 1. The specific entry shown in the figure is a schematic representation only. The special effects entry may be a special effects identifier used to characterize clicking where a special effects selection may be presented. The special effect identifier may include characters, graphics, symbols, and the like. When the user clicks the 'special effects entrance', at least one special effects selection item can be displayed on the display interface. Such as "sound noise reduction", "background blurring", "beauty" and other special effect options in fig. 2B. The user may select one or more special effects options. It will be appreciated that selection buttons may be displayed to the left or right of the special effect selection item described above. The user can select the special effect selection item by clicking the selection button or perform reverse selection operation on the selected special effect selection item.

When the user selects a special effects selection, the terminal device may determine that a request for preprocessing the multimedia stream of the home terminal is received from the participating user a 1. The terminal device may apply the preprocessing corresponding to the special effect selection item to the multimedia information stream of the user collected in real time. And sending the multimedia information stream of the participant user A1 to which the preprocessing is applied to a server side so that the server side forwards the multimedia information stream of the participant user A1 to which the preprocessing is applied.

The terminal devices here may be laptop portable computers and desktop computers. Due to the inconvenience of the movement of laptop portable computers and desktop computers, the angle at which the camera provided in the above-mentioned computers captures images is relatively large. When the terminal equipment is used for carrying out the multimedia conference, the noise ratio of the collected multimedia data stream is larger. For example, the environmental noise in the audio data stream is relatively high, and the background image in each video data frame includes images of other users, so that when the terminal device inconvenient to move is used for a multimedia conference, the noise in the multimedia information stream is relatively high, and the preprocessing is applied to the ongoing multimedia conference in the terminal device according to the request, thereby being beneficial to improving the signal-to-noise ratio of the multimedia data of the users acquired by using the terminal device and improving the effect of the multimedia conference.

In some optional implementations of this embodiment, before step 101, the information processing method for a multimedia conference further includes: displaying information of a preprocessing function in a display interface of the multimedia conference displayed by a client according to opening information which is sent by a server in advance and is used for opening the preprocessing function to the client of the user for carrying out the multimedia conference; and

the step 101 may include: and in the process of the multimedia conference, determining whether a request of the user for applying the preprocessing to the multimedia information stream of the local terminal is received or not according to whether the user executes preset operation on the information of the preprocessing function or not.

In these optional implementations, the server may select the client corresponding to the user to open the function of preprocessing the multimedia data stream.

After the client receives the opening information, the information of the preprocessing function can be displayed in a display interface of the multimedia conference displayed by the client. The information of the preprocessing function may for example comprise an entry of the preprocessing function. When the user selects the entry of the preprocessing function, a preprocessing option may be presented in the presentation interface. The pre-processing options herein may include pre-processing options for audio data and/or pre-processing options for video data.

When the server does not open the function of preprocessing the multimedia data stream to the client corresponding to the user, the user cannot send a request for preprocessing the multimedia information stream application of the home terminal to the client.

Referring to fig. 3, a flow diagram of some embodiments of an information processing method for multimedia conferencing according to the present disclosure is shown. The information processing method for the multimedia conference comprises the following steps:

step 301, in the process of the multimedia conference, determining whether a request of the user for applying the preprocessing to the multimedia information stream of the local terminal is received.

The multimedia conference can be an audio conference and also can be a video conference.

Step 301 may be the same as or similar to step 101 in the embodiment shown in fig. 1, and is not described herein again.

Step 302, in response to receiving the request by monitoring, loading the dynamic link library required by the preprocessing.

Step 303, applying the preprocessing corresponding to the request to the multimedia information stream based on the dynamic link library.

In some application scenarios, the server may send a Dynamic Link Library (DLL) for preprocessing the multimedia information stream to the terminal device in advance, and the terminal device stores the DLL. Such as storing the DLLs described above in an application package of a client for implementing multimedia conferences. After the user initiates a preprocessing request for the multimedia information stream of the user through the client, the terminal device can load the dynamic link library.

The dynamic link library may include functions for processing audio data (e.g., denoising audio data), functions for processing video data (e.g., beautifying face images in video frames, blurring background), and the like.

After receiving a request from a user to preprocess a multimedia information stream of a home terminal in step 301, the client for performing a multimedia conference may load the dynamic link library.

The dynamic link library is used for preprocessing the multimedia data stream of the user, and the dynamic link library is loaded when the user needs to preprocess the multimedia data stream. When the user does not need to start the client for the multimedia conference, the function for preprocessing the multimedia data stream is loaded in the code corresponding to the client, so that the time required for starting the client can be reduced.

The terminal device may apply the preprocessing corresponding to the request to the multimedia information stream of the user collected in real time by using the dynamic link library.

Step 304, the multimedia information stream to which the above preprocessing is applied is transmitted in real time.

Step 304 may be the same as or similar to step 103 in the embodiment shown in fig. 1, and is not described herein again.

Compared with the embodiment shown in fig. 1, the scheme provided in this embodiment highlights that the dynamic link library is loaded after a request for preprocessing the multimedia information stream of the home terminal by the user is received, and the multimedia information stream of the user is preprocessed by using the dynamic link library, so that the time for cold start of the client can be reduced, and the memory occupation of the client when the user does not have special effect requirements can be reduced.

In an optional implementation manner of some embodiments, before step 301, the information processing method for a multimedia conference further includes the following steps:

first, when a client for performing the multimedia conference is started, it is determined whether the client includes a preprocessing function.

The function corresponding to the preprocessing function may be set in a configuration file (manifest file) corresponding to the client.

Here, the terminal device may detect whether the configuration file (manifest file) corresponding to the client includes an identifier of the preprocessing function. And when the configuration file is detected to comprise the identification of the preprocessing function, determining that the client comprises the preprocessing function. Otherwise, determining that the client does not comprise the preprocessing function. When it is determined that the client includes the preprocessing function, the preprocessing function may be displayed on a display interface of the client for displaying the multimedia conference information.

In an optional implementation manner of some embodiments, the preprocessing function is opened to the user by the service end based on that the user attribute of the user satisfies a preset condition.

In an optional implementation manner of some embodiments, the user attribute includes at least one of: the frequency of the client used by the user and the setting parameters of the historical multimedia conference of the user. Correspondingly, when the user attribute includes the frequency of using the client by the user, the preset condition may be that the frequency of using the client by the user is greater than a first preset frequency threshold. And when the user attribute comprises the setting parameters of the historical multimedia conference of the user, the frequency of applying the preprocessing to the multimedia information stream in the historical setting parameters of the historical multimedia conference of the user is greater than a second preset frequency threshold value. The first preset frequency threshold and the second preset frequency threshold may be any integer greater than or equal to 1.

In some optional implementations of this embodiment, the user attribute may further include audio definition and video definition of the user history multimedia conference. The preset conditions include: the audio clarity of the user's historical multimedia conferences is less than a first clarity threshold and/or the video clarity of the user's historical video conferences is less than a second clarity threshold.

As an example, when it is detected that the audio definition of one or more historical multimedia conferences of the user is less than the first definition threshold, the preprocessing function may be opened to the user in order to ensure the effect of the multimedia conferences. The information of the preprocessing function can be displayed on a display interface of the multimedia conference, for example, an identifier of the preprocessing function is displayed. The preprocessing function here may be, for example, to mute ambient noise in the audio data of the user collected at the home terminal. That is, the noise reduction processing is performed on the audio data of the user. The audio clarity can be characterized by the signal-to-noise ratio of the audio data.

As another example, when it is detected that the video definition of one or more historical multimedia conferences of the user is less than the second definition threshold, the preprocessing function may be opened to the user in order to ensure the effect of the multimedia conferences. The information of the preprocessing function can be displayed on a display interface of the multimedia conference. The pre-processing function may include, for example, a background blurring function that blurs a background image in the multimedia video frame.

That is, when the user frequently uses the client, the server may open the preprocessing function to the user. For example, the server may send the function related to the preprocessing function to the terminal device, and the terminal device may update the client according to the function. And writing the preprocessing corresponding function into the configuration file corresponding to the client. The user can display the selection items of the preprocessing function on the display interface of the client. The first preset frequency threshold may be any integer greater than zero, and is not limited herein.

When the frequency of the audio and/or video preprocessed by the user in the third-party application is greater than a second preset frequency threshold, the function corresponding to the preprocessing can be opened for the user. The second preset frequency threshold here may be any integer greater than zero. That is, when the user preprocesses the audio and/or video information in the third-party application for a plurality of times, the server may open the function corresponding to the preprocessing to the user in the client for performing the multimedia conference, so as to facilitate the user to preprocess the multimedia data stream of the user.

Secondly, in response to determining that the client includes a preprocessing function, adding a path for loading the dynamic link library in an initialization list corresponding to the client.

The path for loading the dynamic link library may be a relative path stored in the dynamic link library, or may be an absolute path stored in the dynamic link library.

Further, the loading the dynamic link library required by the preprocessing in response to the monitoring receiving the request in step 302 includes:

and responding to the monitoring to receive the request, and loading the dynamic link library required by the preprocessing based on the path.

In these alternative implementations, the server may determine whether to open the preprocessing function to the user according to the user attribute of the user. In order to manage the multimedia information of the user.

In an optional implementation manner of some embodiments, the information processing method for a multimedia conference shown in fig. 1 and fig. 2 further includes: and stopping preprocessing the multimedia data stream collected in real time in response to receiving an instruction of closing the preprocessing from the user.

Here the user may choose to turn off the above pre-processing. When the user selects to close the preprocessing, the terminal device can not preprocess the real-time multimedia data stream any more, and only sends the original multimedia data stream acquired in real time to the server.

With further reference to fig. 4, as an implementation of the method shown in fig. 1 described above, the present disclosure provides some embodiments of an information processing apparatus for multimedia conferencing, which correspond to those of the method shown in fig. 1, and which may be applied in various electronic devices in particular.

As shown in fig. 4, an information processing apparatus for a multimedia conference of some embodiments includes: a determination unit 401, a preprocessing unit 402, and a transmission unit 403. The determining unit 401 is configured to determine, in the process of the multimedia conference, whether a request for applying preprocessing to a multimedia information stream of a local terminal by a user is received; a preprocessing unit 402, configured to, in response to determining that the request is received, apply preprocessing corresponding to the request to the multimedia information stream of the user; a sending unit 403, configured to send the multimedia information stream local end to which the preprocessing is applied in real time.

In this embodiment, specific processes of the determining unit 401, the preprocessing unit 402, and the sending unit 403 of the information processing apparatus for a multimedia conference and technical effects brought by the specific processes can refer to related descriptions of step 101, step 102, and step 103 in the corresponding embodiment of fig. 1, which are not described herein again.

In an optional implementation manner of some embodiments, the determining unit 401 is further configured to: monitoring the selection operation executed by the user on the preprocessing control displayed on the display interface corresponding to the multimedia conference in the process of the multimedia conference; and responding to the monitoring of the selection operation, and determining that a request of a user for applying preprocessing to the multimedia information stream of the local terminal is received.

In an optional implementation of some embodiments, the preprocessing unit 402 is further configured to: in response to receiving the request by monitoring, loading a dynamic link library required by the preprocessing; and applying the preprocessing corresponding to the request to the multimedia information stream based on the dynamic link library.

In an alternative implementation of some embodiments, the information processing apparatus for multimedia conferences further comprises a presentation unit (not shown in the figures). The display unit is used for displaying the information of the preprocessing function on a display interface of the multimedia conference displayed by the client according to opening information which is sent by a server side in advance and is used for opening the preprocessing function to the client of the user for carrying out the multimedia conference; and the determining unit 401 is further configured to: and in the process of the multimedia conference, determining whether a request of the user for applying the preprocessing to the multimedia information stream of the local terminal is received or not according to whether the user executes preset operation on the information of the preprocessing function or not.

In an optional implementation of some embodiments, the pre-processing function is opened to the user by the service end based on a user attribute of the user satisfying a preset condition.

In an optional implementation of some embodiments, the pre-processing comprises pre-processing of audio data and/or pre-processing of video data; wherein the pre-processing of the audio data comprises: noise reduction; the preprocessing of the video data comprises beautifying and background blurring.

In an alternative implementation of some embodiments, the information processing apparatus for multimedia conferencing further comprises a shutdown unit (not shown in the figures). The closing unit is used for: and stopping applying the preprocessing to the multimedia data stream collected in real time in response to receiving an instruction of a user to close the preprocessing.

The embodiment provides a pair of information processing devices for a multimedia conference, which determines whether a request of a user for applying preprocessing to a multimedia information stream of a local terminal is received or not in the process of the multimedia conference through a determination unit; the preprocessing unit responds to the request and applies preprocessing corresponding to the request to the multimedia information flow of the user acquired in real time; the transmitting unit transmits the multimedia information stream to which the preprocessing is applied in real time. Therefore, the participating users can preprocess their own multimedia information streams in the multimedia conference. The operation of preprocessing the multimedia information flow of the participating users is simplified. The user experience may be improved.

With further reference to fig. 5, fig. 5 illustrates an exemplary system architecture of an information processing method for a multimedia conference or an information processing apparatus for a multimedia conference to which some embodiments of the present disclosure may be applied.

As shown in fig. 5, the system architecture may include

terminal devices

501, 502, 503, a network 504, and a server 505. The network 504 serves to provide a medium for communication links between the

terminal devices

501, 502, 503 and the server 505. Network 504 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.

The

terminal devices

501, 502, 503 may interact with a server 505 over a network 504 to receive or send messages or the like. The

terminal devices

501, 502, 503 may have various client applications installed thereon, such as a web browser application, a search-type application, news information, a multimedia conference-type application, and the like. The client application in the

terminal device

501, 502, 503 may receive the instruction of the user, and complete the corresponding function according to the instruction of the user, for example, add the corresponding information in the information according to the instruction of the user.

The

terminal devices

501, 502, 503 may be hardware or software. When the

terminal devices

501, 502, 503 are hardware, they may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III, mpeg compression standard Audio Layer 3), MP4 players (Moving Picture Experts Group Audio Layer IV, mpeg compression standard Audio Layer 4), laptop portable computers, desktop computers, and the like. When the

terminal devices

501, 502, and 503 are software, they can be installed in the electronic devices listed above. It may be implemented as multiple pieces of software or software modules (e.g., software or software modules used to provide distributed services) or as a single piece of software or software module. And is not particularly limited herein.

The server 505 may be a server providing various services, for example, receiving information streams of the multimedia conference sent by the

terminal devices

501, 502, and 503, analyzing the information streams, and sending the analyzed information streams to the

terminal devices

501, 502, and 503.

It should be noted that the information processing method provided by the embodiment of the present disclosure may be executed by a terminal device, and accordingly, the information processing apparatus may be provided in the

terminal device

501, 502, 503. In addition, the information processing method provided by the embodiment of the present disclosure may also be executed by the server 505, and accordingly, an information processing apparatus may be provided in the server 505.

It should be understood that the number of terminal devices, networks, and servers in fig. 5 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

Referring now to fig. 6, a schematic diagram of an electronic device (e.g., the server or terminal device of fig. 4) suitable for use in implementing some embodiments of the present disclosure is shown. The terminal device in some embodiments of the present disclosure may include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a vehicle terminal (e.g., a car navigation terminal), and the like, and a fixed terminal such as a digital TV, a desktop computer, and the like. The terminal device/server shown in fig. 5 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.

As shown in fig. 6, the electronic device may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 601, which may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage means 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the electronic apparatus are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.

Generally, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 607 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; a storage device 608 including, for example, a magnetic tape, a hard disk, a memory card, or the like; and a communication device 609. The communication means 609 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. While fig. 6 illustrates an electronic device having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided. Each block shown in fig. 6 may represent one device or may represent multiple devices as desired.

In particular, according to some embodiments of the present disclosure, the processes described above with reference to the flow diagrams may be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In some such embodiments, the computer program may be downloaded and installed from a network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. The computer program, when executed by the processing device 601, performs the above-described functions defined in the methods of some embodiments of the present disclosure.

It should be noted that the computer readable medium described in some embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In some embodiments of the disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.

In some embodiments, the clients, servers may communicate using any currently known or future developed network Protocol, such as HTTP (HyperText Transfer Protocol), and may interconnect with any form or medium of digital data communication (e.g., a communications network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.

The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: in the process of the multimedia conference, whether a request of a user for applying preprocessing to the multimedia information stream of the local terminal is received or not is determined; in response to determining that the request is received, applying preprocessing corresponding to the request to the multimedia information stream of the user; transmitting the multimedia information stream to which the preprocessing is applied in real time.

Computer program code for carrying out operations for embodiments of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units described in some embodiments of the present disclosure may be implemented by software, and may also be implemented by hardware. The name of the unit does not in some cases constitute a limitation on the unit itself, and for example, the determining unit may also be described as a "unit that determines whether a request for preprocessing the multimedia information stream of the home terminal is received during the multimedia conference".

The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), systems on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is made without departing from the inventive concept as defined above. For example, the above features and (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure are mutually replaced to form the technical solution.

Claims

1. An information processing method for multimedia conferencing, comprising:

in the process of the multimedia conference, whether a request of a user for applying preprocessing to the multimedia information stream of the local terminal is received or not is determined;

in response to determining that the request is received, applying preprocessing corresponding to the request to the multimedia information stream of the user;

transmitting the multimedia information stream to which the preprocessing is applied in real time.

2. The method of claim 1, wherein determining whether a user request for preprocessing the multimedia information stream application of the local terminal is received during the multimedia conference comprises:

monitoring the selection operation executed by the user on the preprocessing control displayed on the display interface corresponding to the multimedia conference in the process of the multimedia conference; and

and responding to the monitored selection operation, and determining that a request of a user for applying preprocessing to the multimedia information stream of the local terminal is received.

3. The method of claim 1, wherein said applying a pre-processing corresponding to the request to the multimedia information stream of the user in response to determining that the request is received comprises:

in response to receiving the request by monitoring, loading a dynamic link library required by the preprocessing;

and applying the preprocessing corresponding to the request to the multimedia information stream based on the dynamic link library.

4. The method of claim 1, wherein before determining whether a user request for preprocessing of the multimedia stream application of the local terminal is received during the multimedia conference, the method further comprises:

displaying information of a preprocessing function in a display interface of the multimedia conference displayed by a client according to opening information which is sent by a server in advance and is used for opening the preprocessing function to the client of the user for carrying out the multimedia conference; and

in the process of the multimedia conference, determining whether a request of a user for preprocessing the multimedia information stream application of the local terminal is received comprises the following steps:

and in the process of the multimedia conference, determining whether a request of the user for applying the preprocessing to the multimedia information stream of the local terminal is received or not according to whether the user executes preset operation on the information of the preprocessing function or not.

5. The method of claim 1, wherein the pre-processing function is opened to the user by a service end based on a user attribute of the user satisfying a preset condition.

6. The method of claim 5, wherein the user attributes comprise at least one of: the frequency of the user using the client, the setting parameters of the historical multimedia conference of the user, and the audio definition and the video definition of the historical multimedia conference of the user.

7. The method according to claim 6, wherein the preset conditions include: the audio clarity of the user historical video conference is less than a first clarity threshold and/or the video clarity of the user historical video conference is less than a second clarity threshold.

8. The method of claim 1, wherein the pre-processing comprises pre-processing of audio data and/or pre-processing of video data;

the pre-processing of the audio data comprises: denoising the audio data;

the pre-processing of the video data includes at least one of beauty and background blurring.

9. The method of claim 1, wherein the preprocessing is applied to the local multimedia information stream only during the current multimedia conference, according to the request.

10. The method of claim 1, further comprising:

and stopping applying the preprocessing to the multimedia data stream collected in real time in response to receiving an instruction of a user to close the preprocessing.

11. An information processing apparatus for multimedia conferencing, comprising:

the determining unit is used for determining whether a request of a user for applying preprocessing to the multimedia information stream of the local terminal is received or not in the process of the multimedia conference;

the preprocessing unit is used for responding to the received request and applying preprocessing corresponding to the request to the multimedia information flow of the user;

a sending unit, configured to send the multimedia information stream to which the preprocessing is applied in real time.

12. An electronic device, comprising:

one or more processors;

a storage device for storing one or more programs,

when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-10.

13. A computer-readable medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-10.