CN115242757A - Data processing method and device, electronic equipment and storage medium - Google Patents
Data processing method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115242757A CN115242757A CN202211147902.9A CN202211147902A CN115242757A CN 115242757 A CN115242757 A CN 115242757A CN 202211147902 A CN202211147902 A CN 202211147902A CN 115242757 A CN115242757 A CN 115242757A
- Authority
- CN
- China
- Prior art keywords
- video conference
- conference terminal
- video
- reference original
- audio
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 26
- 230000005236 sound signal Effects 0.000 claims abstract description 234
- 238000000034 method Methods 0.000 claims abstract description 21
- 238000004590 computer program Methods 0.000 claims description 7
- 238000004891 communication Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/40—Support for services or applications
- H04L65/403—Arrangements for multi-party communication, e.g. for conferences
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
- G10L19/16—Vocoder architecture
- G10L19/167—Audio streaming, i.e. formatting and decoding of an encoded audio signal representation into a data stream for transmission or storage purposes
Landscapes
- Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Computer Networks & Wireless Communication (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Telephonic Communication Services (AREA)
Abstract
The application provides a data processing method, a data processing device, an electronic device and a storage medium, wherein the method comprises the following steps: for each video conference terminal accessed into the same video conference, the video conference terminal encodes the audio signal currently acquired by the video conference terminal to obtain an audio code stream; the video conference terminal sends the audio code stream to each reference video conference terminal except the video conference terminal; when the video conference terminal receives a reference audio code stream sent by a reference video conference terminal, for each reference audio code stream, the video conference terminal decodes the reference audio code stream to obtain a reference audio signal; if the reference audio signals comprise at least two reference audio signals, the video conference terminal performs audio mixing processing on the reference original audio signals to obtain mixed audio signals; the video conference terminal plays the mixed audio signal. The video conference can be developed without configuring the MCU, so that the front cost of the video conference is reduced.
Description
Technical Field
The present application relates to the field of communications technologies, and in particular, to a data processing method and apparatus, an electronic device, and a storage medium.
Background
In a video conference system, a Multipoint Controller (MCU) is required. The MCU is a multimedia information exchanger and is used for carrying out multi-point calling and connection, realizing the functions of video broadcasting, video selection, audio mixing, data broadcasting and the like, and finishing the junction and switching of signals among all terminals. Specifically, each participant (i.e., terminal) participating in the same video conference encodes local video and audio, and transmits an audio/video code stream obtained by encoding to the MCU through a transmission protocol. After receiving the audio and video signals of each participant, the MCU switches videos according to application requirements and sends appropriate videos to each participant; and after decoding the received audio signals, carrying out audio mixing, and sending the mixed audio to each party. Thus, each participant can see the video of the other participants and hear the audio of each participant at the same time.
The MCU is very powerful, and some MCUs with super-strong performance can even realize simultaneous processing of more than 300 paths of audios and videos (namely, realize that more than 300 terminals participate in the same video conference). However, the more powerful the MCU is, the higher the cost is, and it is practical to configure the MCU for a large-scale video conference, but for a small-scale video conference, such as a 3-party conference or a 5-party conference, configuring one MCU will greatly increase the front cost of the video conference.
Disclosure of Invention
In view of this, an object of the present application is to provide a data processing method, an apparatus, an electronic device, and a storage medium, which can implement the development of a video conference without configuring an MCU, thereby reducing the front cost of the video conference.
In a first aspect, an embodiment of the present application provides a data processing method, where the method includes:
for each video conference terminal in a plurality of video conference terminals accessed into the same video conference, the video conference terminal encodes the original audio signal currently acquired by the video conference terminal to obtain an audio code stream;
the video conference terminal sends the audio code stream to each reference video conference terminal except the video conference terminal;
under the condition that the video conference terminal receives at least one reference audio code stream sent by the reference video conference terminal, for each reference audio code stream, the video conference terminal decodes the reference audio code stream to obtain a reference original audio signal corresponding to the reference audio code stream;
if the reference original audio signals comprise at least two reference original audio signals, the video conference terminal performs audio mixing processing on the reference original audio signals to obtain mixed audio signals;
the video conference terminal plays the mixed audio signal.
In a possible implementation manner, before the video conference terminal encodes an original audio signal currently acquired by itself to obtain an audio code stream, the method further includes:
the video conference terminal judges whether the volume of the original audio signal reaches a preset threshold value;
the video conference terminal encodes the original audio signal currently acquired by the video conference terminal to obtain an audio code stream, and the method comprises the following steps:
and if the volume of the original audio signal reaches the preset threshold value, the video conference terminal encodes the original audio signal to obtain the audio code stream.
In a possible embodiment, each of the video conference terminals is provided with a priority; when the number of the reference original audio signals exceeds a preset number, the preset number is greater than 2, and the video conference terminal performs audio mixing processing on the reference original audio signals to obtain mixed audio signals, including:
the video conference terminal performs audio mixing processing on a target reference original audio signal to obtain the mixed audio signal, wherein the target reference original audio signal comprises: the reference original audio signals are all the reference original audio signals except the first target reference original audio signal, and for each reference original audio signal, the priority corresponding to the reference original audio signal is the priority of sending the reference audio code stream corresponding to the reference original audio signal to a reference video conference terminal of the video conference terminal.
In a possible implementation manner, for every two corresponding target reference original audio signals with different priorities, the weight occupied by the third target reference original audio signal during audio mixing is greater than the weight occupied by the fourth target reference original audio signal during audio mixing, where the third target reference original audio signal is a target reference original audio signal with a higher priority corresponding to the two corresponding target reference original audio signals with different priorities, and the fourth target reference original audio signal is a target reference original audio signal with a lower priority corresponding to the two corresponding target reference original audio signals with different priorities.
In one possible embodiment, the method further comprises:
the video conference terminal encodes the currently acquired original video signal to obtain a video code stream;
the video conference terminal sends the video code stream to each reference video conference terminal;
if the video conference terminal receives at least one reference video code stream sent by the reference video conference terminal, the video conference terminal decodes the reference video code stream for each reference video code stream to obtain a reference original video signal corresponding to the reference video code stream;
the video conference terminal allocates a playing area for each reference original video signal on a screen display area of the video conference terminal, wherein the playing area allocated to a target reference original video signal is larger than the playing area allocated to a candidate reference original video signal, the target reference original video signal is a reference original video signal with the highest corresponding priority, the candidate reference original video signal is a reference original video signal except the target reference original video signal in all the reference original video signals, and for each reference original video signal, the priority corresponding to the reference original video signal is the priority of sending a reference video code stream corresponding to the reference original video signal to a reference video conference terminal of the video conference terminal;
for each reference original video signal, the video conference terminal plays the reference original video signal on a playing area allocated for the reference original video signal.
In a second aspect, an embodiment of the present application further provides a data processing apparatus, where the apparatus includes:
the audio coding module is used for coding an original audio signal currently acquired by each video conference terminal in a plurality of video conference terminals accessed into the same video conference to obtain an audio code stream;
the audio sending module is used for sending the audio code stream to each reference video conference terminal except the video conference terminal in the plurality of video conference terminals;
the audio decoding module is used for decoding each reference audio code stream to obtain a reference original audio signal corresponding to the reference audio code stream under the condition that the video conference terminal receives at least one reference audio code stream sent by the reference video conference terminal;
the audio mixing module is used for carrying out audio mixing processing on the reference original audio signals to obtain mixed audio signals if the reference original audio signals comprise at least two reference original audio signals;
and the audio playing module is used for playing the mixed audio signal.
In a possible embodiment, the apparatus further comprises:
the judging module is used for judging whether the volume of the original audio signal reaches a preset threshold value before the audio coding module codes the original audio signal currently collected by the video conference terminal to obtain an audio code stream;
the audio encoding module is specifically configured to:
and if the volume of the original audio signal reaches the preset threshold value, encoding the original audio signal to obtain the audio code stream.
In a possible embodiment, each of the video conference terminals is provided with a priority; when the number of the reference original audio signals exceeds a preset number, where the preset number is greater than 2, the sound mixing module is specifically configured to:
performing audio mixing processing on a target reference original audio signal to obtain the mixed audio signal, wherein the target reference original audio signal includes: the audio signal processing method comprises the steps of obtaining a first target reference original audio signal and a second target reference original audio signal, wherein the first target reference original audio signal is a reference original audio signal with higher volume and reduced by one by the preset number, the second target reference original audio signal is a reference original audio signal with the highest priority in all the reference original audio signals except the first target reference original audio signal, and for each reference original audio signal, the priority corresponding to the reference original audio signal is the priority of sending a reference audio code stream corresponding to the reference original audio signal to a reference video conference terminal of the video conference terminal.
In a possible implementation manner, for every two corresponding target reference original audio signals with different priorities, a weight occupied by a third target reference original audio signal when performing audio mixing is greater than a weight occupied by a fourth target reference original audio signal when performing audio mixing, where the third target reference original audio signal is a target reference original audio signal with a higher priority corresponding to the two corresponding target reference original audio signals with different priorities, and the fourth target reference original audio signal is a target reference original audio signal with a lower priority corresponding to the two corresponding target reference original audio signals with different priorities.
In a possible embodiment, the apparatus further comprises:
the video coding module is used for coding the original video signal currently collected by the video conference terminal to obtain a video code stream;
the video sending module is used for sending the video code stream to each reference video conference terminal;
the video decoding module is used for decoding the reference video code stream for each reference video code stream to obtain a reference original video signal corresponding to the reference video code stream if the video conference terminal receives at least one reference video code stream sent by the reference video conference terminal;
the distribution module is used for distributing a playing area for each reference original video signal on a screen display area of the video conference terminal, wherein the playing area distributed to a target reference original video signal is larger than the playing area distributed to a candidate reference original video signal, the target reference original video signal is a reference original video signal with the highest corresponding priority, the candidate reference original video signal is a reference original video signal except the target reference original video signal in all the reference original video signals, and for each reference original video signal, the priority corresponding to the reference original video signal is the priority for sending a reference video code stream corresponding to the reference original video signal to the reference video conference terminal of the video conference terminal;
and the video playing module is used for playing the reference original video signal on a playing area allocated for the reference original video signal for each reference original video signal.
In a third aspect, an embodiment of the present application further provides an electronic device, including: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, when the electronic device runs, the processor and the storage medium communicate through the bus, and the processor executes the machine-readable instructions to execute the steps of the data processing method according to any one of the first aspect.
In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to perform the steps of the data processing method according to any one of the first aspect.
The data processing method, the data processing device, the electronic equipment and the storage medium can realize the development of the video conference without configuring the MCU, so that the front cost of the video conference is reduced.
Drawings
To more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.
Fig. 1 is a flowchart illustrating a data processing method provided in an embodiment of the present application;
FIG. 2 is a flow chart of another data processing method provided by the embodiment of the application;
fig. 3 is a schematic structural diagram illustrating a data processing apparatus according to an embodiment of the present application;
fig. 4 shows a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it should be understood that the drawings in the present application are only for illustration and description purposes and are not used to limit the protection scope of the present application. Additionally, it should be understood that the schematic drawings are not necessarily drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of the present application. It should be understood that the operations of the flow diagrams may be performed out of order, and steps without logical context may be performed in reverse order or simultaneously. One skilled in the art, under the guidance of this application, may add one or more other operations to, or remove one or more operations from, the flowchart.
In addition, the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that in the embodiments of the present application, the term "comprising" is used to indicate the presence of the features stated hereinafter, but does not exclude the addition of further features.
For facilitating understanding of the present embodiment, a data processing method, an apparatus, an electronic device, and a storage medium provided in the embodiments of the present application are described in detail.
Referring to fig. 1, a flowchart of a data processing method provided in an embodiment of the present application is shown, where the method includes:
s101, for each video conference terminal in a plurality of video conference terminals accessed into the same video conference, the video conference terminal encodes the currently acquired original audio signal to obtain an audio code stream.
For example, the number of the video conference terminals may be 2 to 5, and if the number of the video conference terminals accessing the same video conference is too large, the video conference terminals may be stuck (depending on the performance of the video conference terminals).
And S102, the video conference terminal sends the audio code stream to each reference video conference terminal except the video conference terminal.
For example, there are three video conference terminals including: the system comprises a video conference terminal A, a video conference terminal B and a video conference terminal C.
Then, for the video conference terminal a, the video conference terminal a sends an audio code stream o obtained by encoding the original video signal a currently acquired by the video conference terminal a to the video conference terminal B and the video conference terminal C;
for the video conference terminal B, the video conference terminal B sends an audio code stream p obtained by encoding an original video signal B currently acquired by the video conference terminal B to the video conference terminal A and the video conference terminal C;
for the video conference terminal C, the video conference terminal C sends an audio code stream q obtained by encoding an original video signal C currently collected by the video conference terminal C to the video conference terminal A and the video conference terminal B.
For each video conference terminal, the video conference terminal can receive n-1 reference audio code streams at most, wherein n is the number of the video conference terminals.
S103, under the condition that the video conference terminal receives at least one reference audio code stream sent by the reference video conference terminal, for each reference audio code stream, the video conference terminal decodes the reference audio code stream to obtain a reference original audio signal corresponding to the reference audio code stream.
And S104, if the reference original audio signals comprise at least two reference original audio signals, the video conference terminal performs audio mixing processing on the reference original audio signals to obtain mixed audio signals.
Namely, the audio mixing work originally performed by the MCU is changed into the local operation performed by the video conference terminal.
And S105, the video conference terminal plays the mixed audio signal.
If the reference original audio signal only comprises one reference original audio signal, the video conference terminal directly plays the reference original audio signal.
In a possible implementation manner, before the video conference terminal encodes an original audio signal currently acquired by itself to obtain an audio code stream, the method further includes:
and the video conference terminal judges whether the volume of the original audio signal reaches a preset threshold value.
The video conference terminal encodes the original audio signal currently acquired by the video conference terminal to obtain an audio code stream, and the method comprises the following steps:
and if the volume of the original audio signal reaches the preset threshold, the video conference terminal encodes the original audio signal to obtain the audio code stream.
That is, the video conference terminal performs the step of encoding the original audio signal only when the volume of the original audio signal reaches a certain level (i.e., a preset threshold) or more, and does not perform the step of encoding the original audio signal if the volume of the original audio signal does not reach the preset threshold.
In a possible embodiment, each of the video conference terminals is provided with a priority; when the number of the reference original audio signals exceeds a preset number, the preset number is greater than 2, and the video conference terminal performs audio mixing processing on the reference original audio signals to obtain mixed audio signals, including:
the video conference terminal performs audio mixing processing on a target reference original audio signal to obtain the mixed audio signal, wherein the target reference original audio signal comprises: the reference original audio signals are all the reference original audio signals except the first target reference original audio signal, and for each reference original audio signal, the priority corresponding to the reference original audio signal is the priority of sending the reference audio code stream corresponding to the reference original audio signal to a reference video conference terminal of the video conference terminal.
Illustratively, the reference original audio signals are 5 in number, and the preset number is 4; the priority comprises the following steps from high to low: class A, class B and class C; the reference original audio signal includes: the method comprises the steps of referring to an original audio signal 1 (the corresponding priority is C level, and the volume is 100 dB), referring to an original audio signal 2 (the corresponding priority is B level, and the volume is 80 dB), referring to an original audio signal 3 (the corresponding priority is A level, and the volume is 60 dB), referring to an original audio signal 4 (the corresponding priority is B level, and the volume is 70 dB), and referring to an original audio signal 5 (the corresponding priority is C level, and the volume is 120 dB).
Then, the first target reference original audio signal (i.e. the reference original audio signal with the larger volume reduced by one (since the foregoing example mentioned that the preset number is 4, the preset number reduced by one is 3)) includes: reference is made to the original audio signal 5 (with the highest volume of 120 db), to the original audio signal 1 (with the second highest volume of 100 db), and to the original audio signal 2 (with the third highest volume of 80 db).
The second target reference original audio signal is the reference original audio signal 3.
In some cases, the second target reference original audio signal includes two or more (i.e., there is a case where the priorities are the same), then the second target reference original audio signal with the highest volume is selected from all the second target reference original audio signals as the second target reference original audio signal to be mixed (i.e., when mixing, the second target reference original audio signal includes only one, that is, only a preset number of reference original audio signals are mixed).
Specifically, in order to make the video conference terminal know the priority level corresponding to each reference original audio signal, for each reference original audio signal, the first target reference video conference terminal may add an identifier for characterizing the priority level of the first target reference video conference terminal to the reference original audio signal before encoding the reference original audio signal, where the first target reference video conference terminal is the reference video conference terminal that sends the reference audio code stream corresponding to the reference original audio signal to the video conference terminal.
Through the mode, even if the speaking volume of the important participant (namely the video conference terminal with higher priority) is smaller, the important participant can participate in the mixed sound, so that the speaking of the important participant can be ensured to be listened by the other participants (namely the other video conference terminals).
In a possible implementation manner, for every two corresponding target reference original audio signals with different priorities, a weight occupied by a third target reference original audio signal when performing audio mixing is greater than a weight occupied by a fourth target reference original audio signal when performing audio mixing, where the third target reference original audio signal is a target reference original audio signal with a higher priority corresponding to the two corresponding target reference original audio signals with different priorities, and the fourth target reference original audio signal is a target reference original audio signal with a lower priority corresponding to the two corresponding target reference original audio signals with different priorities.
For each two corresponding target reference original audio signals with the same priority, the weights occupied by the two corresponding target reference original audio signals with the same priority when performing sound mixing may be the same.
Through the above manner, even if the speaking volume of the important participant (i.e. the video conference terminal with higher priority) is smaller, the important participant can participate in the mixing, and the speaking of the important participant (i.e. the video conference terminal with higher priority) can be more clearly heard by the other participants (i.e. the other video conference terminals).
Referring to fig. 2, which is a flowchart of another data processing method provided in an embodiment of the present application, in a possible implementation manner, the method further includes:
s201, the video conference terminal encodes the currently acquired original video signal to obtain a video code stream.
And S202, the video conference terminal sends the video code stream to each reference video conference terminal.
S203, if the video conference terminal receives at least one reference video code stream sent by the reference video conference terminal, for each reference video code stream, the video conference terminal decodes the reference video code stream to obtain a reference original video signal corresponding to the reference video code stream.
And S204, the video conference terminal allocates a playing area for each reference original video signal on a screen display area of the video conference terminal, wherein the playing area allocated to a target reference original video signal is larger than the playing area allocated to a candidate reference original video signal, the target reference original video signal is a reference original video signal with the highest corresponding priority, the candidate reference original video signal is a reference original video signal except the target reference original video signal in all the reference original video signals, and for each reference original video signal, the priority corresponding to the reference original video signal is the priority for sending a reference video code stream corresponding to the reference original video signal to the reference video conference terminal of the video conference terminal.
That is, the playback area assigned to the corresponding reference original video signal having the highest priority is the largest.
The size of the playback area allocated to each candidate reference original video signal may be the same or different.
Specifically, for each reference original video signal, the second target reference video conference terminal may add, to the reference original video signal, an identifier for representing the priority of the second target reference video conference terminal before encoding the reference original video signal, where the second target reference video conference terminal is a reference video conference terminal that sends the reference video code stream corresponding to the reference original video signal to the video conference terminal.
S205, for each of the reference original video signals, the video conference terminal plays the reference original video signal in the playing area allocated for the reference original video signal.
In addition, the playback area assigned to the target reference original video signal may also be highlighted.
The data processing method provided by the embodiment of the application can realize the development of the video conference without configuring the MCU, thereby reducing the front cost of the video conference.
Referring to fig. 3, a schematic structural diagram of a data processing apparatus provided in an embodiment of the present application is shown, where the apparatus includes:
the audio encoding module 301 is configured to encode, for each video conference terminal in a plurality of video conference terminals accessing the same video conference, an original audio signal currently acquired by the video conference terminal to obtain an audio code stream;
an audio sending module 302, configured to send the audio code stream to each reference video conference terminal in the video conference terminals except the video conference terminal;
an audio decoding module 303, configured to, when the video conference terminal receives at least one reference audio code stream sent by the reference video conference terminal, decode the reference audio code stream for each reference audio code stream to obtain a reference original audio signal corresponding to the reference audio code stream;
a sound mixing module 304, configured to perform sound mixing processing on the reference original audio signal to obtain a mixed audio signal if the reference original audio signal includes at least two reference original audio signals;
an audio playing module 305, configured to play the mixed audio signal.
In a possible embodiment, the apparatus further comprises:
a determining module, configured to determine whether a volume of an original audio signal reaches a preset threshold before the audio encoding module 301 encodes the original audio signal currently acquired by the video conference terminal to obtain an audio code stream;
the audio encoding module 301 is specifically configured to:
and if the volume of the original audio signal reaches the preset threshold value, encoding the original audio signal to obtain the audio code stream.
In a possible embodiment, each of the video conference terminals is provided with a priority; when the number of the reference original audio signals exceeds a preset number, where the preset number is greater than 2, the sound mixing module 304 is specifically configured to:
performing audio mixing processing on a target reference original audio signal to obtain the mixed audio signal, wherein the target reference original audio signal includes: the audio signal processing method comprises the steps of obtaining a first target reference original audio signal and a second target reference original audio signal, wherein the first target reference original audio signal is a reference original audio signal with higher volume and reduced by one by the preset number, the second target reference original audio signal is a reference original audio signal with the highest priority in all the reference original audio signals except the first target reference original audio signal, and for each reference original audio signal, the priority corresponding to the reference original audio signal is the priority of sending a reference audio code stream corresponding to the reference original audio signal to a reference video conference terminal of the video conference terminal.
In a possible implementation manner, for every two corresponding target reference original audio signals with different priorities, a weight occupied by a third target reference original audio signal when performing audio mixing is greater than a weight occupied by a fourth target reference original audio signal when performing audio mixing, where the third target reference original audio signal is a target reference original audio signal with a higher priority corresponding to the two corresponding target reference original audio signals with different priorities, and the fourth target reference original audio signal is a target reference original audio signal with a lower priority corresponding to the two corresponding target reference original audio signals with different priorities.
In one possible embodiment, the apparatus further comprises:
the video coding module is used for coding an original video signal currently acquired by the video conference terminal to obtain a video code stream;
the video sending module is used for sending the video code stream to each reference video conference terminal;
the video decoding module is used for decoding the reference video code stream for each reference video code stream to obtain a reference original video signal corresponding to the reference video code stream if the video conference terminal receives at least one reference video code stream sent by the reference video conference terminal;
the distribution module is used for distributing a playing area for each reference original video signal on a screen display area of the video conference terminal, wherein the playing area distributed to a target reference original video signal is larger than the playing area distributed to a candidate reference original video signal, the target reference original video signal is a reference original video signal with the highest corresponding priority, the candidate reference original video signal is a reference original video signal except the target reference original video signal in all the reference original video signals, and for each reference original video signal, the priority corresponding to the reference original video signal is the priority for sending a reference video code stream corresponding to the reference original video signal to the reference video conference terminal of the video conference terminal;
and the video playing module is used for playing the reference original video signal on a playing area allocated for the reference original video signal for each reference original video signal.
The data processing device provided by the embodiment of the application can realize development of a video conference without configuring an MCU (microprogrammed control Unit), thereby reducing the front cost of the video conference.
Referring to fig. 4, an electronic device 400 provided in an embodiment of the present application includes: a processor 401, a memory 402 and a bus, wherein the memory 402 stores machine-readable instructions executable by the processor 401, when the electronic device is running, the processor 401 and the memory 402 communicate with each other via the bus, and the processor 401 executes the machine-readable instructions to perform the steps of the method of data processing as described above.
Specifically, the memory 402 and the processor 401 can be general-purpose memory and processor, and are not limited to specific examples, and the processor 401 can execute the data processing method when executing the computer program stored in the memory 402.
Corresponding to the above data processing method, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to perform the steps of the above data processing method.
It can be clearly understood by those skilled in the art that, for convenience and simplicity of description, the specific working process of the system and the apparatus described above may refer to the corresponding process in the method embodiment, and is not described in detail in this application. In the several embodiments provided in this application, it should be understood that the disclosed system, and method may be implemented in other ways. The above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is merely a logical division, and there may be other divisions in actual implementation, and for example, a plurality of modules or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or modules through some communication interfaces, and may be in an electrical, mechanical or other form.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one position, or may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solutions of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a U disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. A method of data processing, the method comprising:
for each video conference terminal in a plurality of video conference terminals accessed into the same video conference, the video conference terminal encodes the original audio signal currently acquired by the video conference terminal to obtain an audio code stream;
the video conference terminal sends the audio code stream to each reference video conference terminal except the video conference terminal;
under the condition that the video conference terminal receives at least one reference audio code stream sent by the reference video conference terminal, for each reference audio code stream, the video conference terminal decodes the reference audio code stream to obtain a reference original audio signal corresponding to the reference audio code stream;
if the reference original audio signals comprise at least two reference original audio signals, the video conference terminal performs audio mixing processing on the reference original audio signals to obtain mixed audio signals;
the video conference terminal plays the mixed audio signal.
2. The data processing method of claim 1, wherein before the video conference terminal encodes the currently acquired original audio signal to obtain the audio code stream, the method further comprises:
the video conference terminal judges whether the volume of the original audio signal reaches a preset threshold value;
the video conference terminal encodes the original audio signal currently acquired by the video conference terminal to obtain an audio code stream, and the method comprises the following steps:
and if the volume of the original audio signal reaches the preset threshold, the video conference terminal encodes the original audio signal to obtain the audio code stream.
3. The data processing method according to claim 1, wherein each of the video conference terminals is provided with a priority; when the number of the reference original audio signals exceeds a preset number, the preset number is greater than 2, and the video conference terminal performs audio mixing processing on the reference original audio signals to obtain mixed audio signals, including:
the video conference terminal performs audio mixing processing on a target reference original audio signal to obtain the mixed audio signal, wherein the target reference original audio signal comprises: the reference original audio signals are all the reference original audio signals except the first target reference original audio signal, and for each reference original audio signal, the priority corresponding to the reference original audio signal is the priority of sending the reference audio code stream corresponding to the reference original audio signal to a reference video conference terminal of the video conference terminal.
4. The data processing method of claim 3, wherein for every two corresponding target reference original audio signals with different priorities, a weight occupied by a third target reference original audio signal during audio mixing is greater than a weight occupied by a fourth target reference original audio signal during audio mixing, wherein the third target reference original audio signal is a target reference original audio signal with a higher priority corresponding to the two corresponding target reference original audio signals with different priorities, and the fourth target reference original audio signal is a target reference original audio signal with a lower priority corresponding to the two corresponding target reference original audio signals with different priorities.
5. The data processing method of claim 3, wherein the method further comprises:
the video conference terminal encodes the currently acquired original video signal to obtain a video code stream;
the video conference terminal sends the video code stream to each reference video conference terminal;
if the video conference terminal receives at least one reference video code stream sent by the reference video conference terminal, the video conference terminal decodes the reference video code stream for each reference video code stream to obtain a reference original video signal corresponding to the reference video code stream;
the video conference terminal allocates a playing area for each reference original video signal on a screen display area of the video conference terminal, wherein the playing area allocated to a target reference original video signal is larger than the playing area allocated to a candidate reference original video signal, the target reference original video signal is a reference original video signal with the highest corresponding priority, the candidate reference original video signal is a reference original video signal except the target reference original video signal in all the reference original video signals, and for each reference original video signal, the priority corresponding to the reference original video signal is the priority of sending a reference video code stream corresponding to the reference original video signal to a reference video conference terminal of the video conference terminal;
for each reference original video signal, the video conference terminal plays the reference original video signal on a playing area allocated for the reference original video signal.
6. A data processing apparatus, characterized in that the apparatus comprises:
the audio coding module is used for coding an original audio signal currently acquired by each video conference terminal in a plurality of video conference terminals accessed to the same video conference to obtain an audio code stream;
the audio sending module is used for sending the audio code stream to each reference video conference terminal except the video conference terminal in the plurality of video conference terminals;
the audio decoding module is used for decoding each reference audio code stream to obtain a reference original audio signal corresponding to the reference audio code stream under the condition that the video conference terminal receives at least one reference audio code stream sent by the reference video conference terminal;
the audio mixing module is used for carrying out audio mixing processing on the reference original audio signals to obtain mixed audio signals if the reference original audio signals comprise at least two reference original audio signals;
and the audio playing module is used for playing the mixed audio signal.
7. The data processing apparatus of claim 6, wherein the apparatus further comprises:
the judging module is used for judging whether the volume of the original audio signal reaches a preset threshold value or not before the audio coding module codes the original audio signal currently collected by the video conference terminal to obtain an audio code stream;
the audio encoding module is specifically configured to:
and if the volume of the original audio signal reaches the preset threshold value, encoding the original audio signal to obtain the audio code stream.
8. The data processing apparatus according to claim 6, wherein each of the video conference terminals is provided with a priority; when the number of the reference original audio signals exceeds a preset number, where the preset number is greater than 2, the sound mixing module is specifically configured to:
performing audio mixing processing on a target reference original audio signal to obtain the mixed audio signal, wherein the target reference original audio signal includes: the reference original audio signals are all the reference original audio signals except the first target reference original audio signal, and for each reference original audio signal, the priority corresponding to the reference original audio signal is the priority of sending the reference audio code stream corresponding to the reference original audio signal to a reference video conference terminal of the video conference terminal.
9. An electronic device, comprising: a processor, a storage medium and a bus, the storage medium storing machine-readable instructions executable by the processor, the processor and the storage medium communicating via the bus when the electronic device is operating, the processor executing the machine-readable instructions to perform the steps of the data processing method according to any one of claims 1 to 5.
10. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when being executed by a processor, performs the steps of the data processing method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211147902.9A CN115242757B (en) | 2022-09-21 | 2022-09-21 | Data processing method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211147902.9A CN115242757B (en) | 2022-09-21 | 2022-09-21 | Data processing method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115242757A true CN115242757A (en) | 2022-10-25 |
| CN115242757B CN115242757B (en) | 2023-05-26 |
Family
ID=83680739
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211147902.9A Active CN115242757B (en) | 2022-09-21 | 2022-09-21 | Data processing method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115242757B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110279632A1 (en) * | 2010-05-13 | 2011-11-17 | Kulkarni Hrishikesh G | Multiway Telepresence without a Hardware MCU |
| CN102938833A (en) * | 2012-07-25 | 2013-02-20 | 苏州科达科技有限公司 | Method and device in video conference, multi-point control device and video conference system |
| WO2015180330A1 (en) * | 2014-05-30 | 2015-12-03 | 中兴通讯股份有限公司 | Volume adjustment method and device, and multipoint control unit |
| CN112272281A (en) * | 2020-10-09 | 2021-01-26 | 上海晨驭信息科技有限公司 | Regional distributed video conference system |
| CN113973103A (en) * | 2021-10-26 | 2022-01-25 | 北京达佳互联信息技术有限公司 | Audio processing method and device, electronic equipment and storage medium |
-
2022
- 2022-09-21 CN CN202211147902.9A patent/CN115242757B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110279632A1 (en) * | 2010-05-13 | 2011-11-17 | Kulkarni Hrishikesh G | Multiway Telepresence without a Hardware MCU |
| CN102938833A (en) * | 2012-07-25 | 2013-02-20 | 苏州科达科技有限公司 | Method and device in video conference, multi-point control device and video conference system |
| WO2015180330A1 (en) * | 2014-05-30 | 2015-12-03 | 中兴通讯股份有限公司 | Volume adjustment method and device, and multipoint control unit |
| CN112272281A (en) * | 2020-10-09 | 2021-01-26 | 上海晨驭信息科技有限公司 | Regional distributed video conference system |
| CN113973103A (en) * | 2021-10-26 | 2022-01-25 | 北京达佳互联信息技术有限公司 | Audio processing method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115242757B (en) | 2023-05-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2020238441A1 (en) | Multi-terminal screen projection method, computer device and storage medium | |
| CN102648584B (en) | Use the system of forward error correction inspection available bandwidth, method and medium | |
| US9560096B2 (en) | Local media rendering | |
| CN111432158B (en) | Load balancing method and device for media nodes, storage medium and electronic device | |
| CN111583942B (en) | Method and device for controlling coding rate of voice session and computer equipment | |
| CN102404543B (en) | Processing method and device of cascade conference place in cascade conference and cascade conference system | |
| CN110677696A (en) | Live broadcast interaction system and method, equipment and storage medium | |
| CN109217980B (en) | Encoding and decoding capacity configuration method, device and computer storage medium | |
| CN105611219A (en) | Method and device for processing video conference | |
| CN115065829A (en) | Multi-person wheat connecting method and related equipment | |
| CN112055166B (en) | Audio data processing method, device, conference system and storage medium | |
| CN104469032A (en) | Sound mixing processing method and system | |
| CN111866440B (en) | Method, device and equipment for pushing video data and storage medium | |
| CN110971862B (en) | Video conference broadcasting method and device | |
| CN110784731B (en) | Data stream transcoding method, device, equipment and medium | |
| CN115242757A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN111541905A (en) | Live broadcast method and device, computer equipment and storage medium | |
| CN104735390A (en) | Layout method and device for videos and audios in immersive conference | |
| JP4425887B2 (en) | VIDEO CONFERENCE SYSTEM, TERMINAL USED FOR VIDEO CONFERENCE SYSTEM, TERMINAL PROCESSING METHOD, AND ITS PROGRAM | |
| CN109120881B (en) | Multi-party conference control method, system and terminal | |
| CN114205547A (en) | Conference control method and system, multipoint control unit, configuration platform and medium | |
| CN112910827B (en) | Multi-party interaction system, method and device, electronic equipment and storage medium | |
| CN115865877B (en) | Audio and video processing method, equipment and computer readable storage medium | |
| KR20180105594A (en) | Multi-point connection control apparatus and method for video conference service | |
| US11798561B2 (en) | Method, apparatus, and non-transitory computer readable medium for processing audio of virtual meeting room |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |