CN117135150A - Optimization method and system based on audio and video fusion communication technology - Google Patents
Optimization method and system based on audio and video fusion communication technology Download PDFInfo
- Publication number
- CN117135150A CN117135150A CN202310442320.1A CN202310442320A CN117135150A CN 117135150 A CN117135150 A CN 117135150A CN 202310442320 A CN202310442320 A CN 202310442320A CN 117135150 A CN117135150 A CN 117135150A
- Authority
- CN
- China
- Prior art keywords
- audio
- video
- fusion
- equipment
- signal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004891 communication Methods 0.000 title claims abstract description 152
- 230000004927 fusion Effects 0.000 title claims abstract description 61
- 238000000034 method Methods 0.000 title claims abstract description 60
- 238000005457 optimization Methods 0.000 title claims abstract description 32
- 238000005516 engineering process Methods 0.000 title claims abstract description 31
- 230000005540 biological transmission Effects 0.000 claims abstract description 20
- 230000000977 initiatory effect Effects 0.000 claims abstract description 8
- 230000005236 sound signal Effects 0.000 claims description 56
- 238000001228 spectrum Methods 0.000 claims description 20
- 238000003860 storage Methods 0.000 claims description 16
- 238000012545 processing Methods 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 10
- 230000002452 interceptive effect Effects 0.000 claims description 10
- 238000004422 calculation algorithm Methods 0.000 claims description 8
- 238000007500 overflow downdraw method Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 8
- 238000013519 translation Methods 0.000 claims description 6
- 238000004140 cleaning Methods 0.000 claims description 4
- 230000003595 spectral effect Effects 0.000 claims description 4
- 238000007499 fusion processing Methods 0.000 claims description 3
- 238000003032 molecular docking Methods 0.000 claims description 3
- 230000001105 regulatory effect Effects 0.000 claims description 3
- 230000001276 controlling effect Effects 0.000 claims description 2
- 210000001503 joint Anatomy 0.000 claims description 2
- 230000000737 periodic effect Effects 0.000 claims description 2
- 230000008901 benefit Effects 0.000 abstract description 5
- 230000010354 integration Effects 0.000 abstract description 2
- 238000012544 monitoring process Methods 0.000 description 4
- 238000012795 verification Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000010295 mobile communication Methods 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical compound [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 229910021389 graphene Inorganic materials 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/80—Responding to QoS
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
- H04L51/04—Real-time or near real-time messaging, e.g. instant messaging [IM]
- H04L51/046—Interoperability with other network applications or services
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/40—Support for services or applications
- H04L65/403—Arrangements for multi-party communication, e.g. for conferences
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Telephonic Communication Services (AREA)
Abstract
The application discloses an optimization method and system based on an audio and video fusion communication technology, comprising the following steps: adopting a centralized deployment scheme, deploying a converged communication server platform by a central node; establishing a two-way transmission channel of local telephone and external telephone, and initiating an audio-video conference; synchronizing address book information to the converged communication server; and accessing the equipment of the headquarter to the converged communication server for accessing, and accessing the equipment of the subsection to the converged communication server for accessing the headquarter system. The application innovates interconnection and interworking among various types of communication, and has the advantages of open compatibility, interconnection and interworking of various types of systems, rich access modes and the like. The integration of a voice system, a video system, an IM system and the like is realized on a unified platform, and a user can realize all business communication requirements on one set of system; the method can be compatible with the original analog telephone of the user, and supports rich access modes such as video conference terminals, live broadcasting and the like; and providing single sign-on authentication to realize entrance unification.
Description
Technical Field
The application relates to the technical field of fusion communication, in particular to an optimization method and system based on an audio and video fusion communication technology.
Background
With the scale of internal communications and the ever-expanding business architecture, employee-to-employee collaboration, and team-to-team collaboration, has become increasingly important. Because the existing communication demands become more and more regional and mobile, the new office trends provide new challenges for the internal communication of enterprises, and the existing internal communication systems are more and more independent of each other, for example, a set of IM chat system, voice system, video system and business office system are needed for realizing instant messaging intercommunication, so that the systems are mutually split, independent islands are formed among the subsystems, and inconvenience is caused to users and operation and maintenance work. For example, in the existing use mode of communication inside enterprises, a telephone system is required for making a call, another conference system is provided when a conference is held, and document collaboration inside the enterprises is provided on other systems, which adds an obstacle to system communication in an intangible way, and the mutually-split communication system also limits the development of enterprise collaboration application.
Disclosure of Invention
This section is intended to outline some aspects of embodiments of the application and to briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section as well as in the description of the application and in the title of the application, which may not be used to limit the scope of the application.
The present application has been made in view of the above-described problems.
Therefore, the technical problems solved by the application are as follows: the existing enterprise internal communication system has the defects that the systems are mutually split, independent islands are formed among all subsystems, and inconvenience is caused to users and operation and maintenance work. Adding to the system communication and also limiting the development of enterprise collaboration applications.
In order to solve the technical problems, the application provides the following technical scheme: an optimization method based on an audio-video fusion communication technology comprises the following steps:
adopting a centralized deployment scheme, deploying a converged communication server platform by a central node;
establishing a two-way transmission channel of local telephone and external telephone, and initiating an audio-video conference;
synchronizing address book information to the converged communication server;
and accessing the equipment of the headquarter to the converged communication server for accessing, and accessing the equipment of the subsection to the converged communication server for accessing the headquarter system.
As a preferred scheme of the optimization method based on the audio-video fusion communication technology, the application comprises the following steps: the two-way transmission channel of the local telephone and the external telephone comprises:
docking a voice gateway through an SIP relay link;
the converged communication server transmits an external telephone for calling the local telephone to the voice gateway;
the voice gateway transmitting the outside line telephone to an operator;
and when the local call is made to the outside line telephone, the local call is returned through the original path of the operator.
As a preferred scheme of the optimization method based on the audio-video fusion communication technology, the application comprises the following steps: the process of synchronizing address book information to the converged communication server comprises the following steps:
the converged communication server is in butt joint with an LDAP server;
the converged communication server transmits authentication information to the LDAP server;
and synchronizing address book information to the converged communication server after the LDAP server authenticates.
As a preferred scheme of the optimization method based on the audio-video fusion communication technology, the application comprises the following steps: the audio and video fusion process of the fusion communication server comprises the following steps:
judging the type of the connected communication equipment: when the types of the connected communication equipment are the same, dividing the communication equipment into similar equipment sets, directly outputting signals without translation coding and decoding, and directly distributing media streams to all the communication equipment through the fusion communication server; when the types of the connected communication equipment are different, the communication equipment is divided into different equipment sets, and translation is carried out according to audio/video coding and decoding of different communication equipment;
establishing communication channels in different types of equipment sets, and judging whether the connected communication equipment supports video capability or not: when the connected communication equipment does not support the video capability, the communication equipment which does not support the video capability is divided into a call equipment set only, and the communication equipment which supports the audio and video capability is divided into a call and video equipment set;
aiming at a call equipment only set, eliminating video streams in the media streams, decoding audio and video signals by using different equipment specific audio and video codecs, and distributing the media streams to connected communication equipment through the converged communication server;
aiming at a call and video equipment set, carrying out weighted fusion on audio and video signals in a media stream by using a weighted average method, decoding the audio and video signals by using specific audio and video codecs of different equipment, and distributing the media stream to the connected communication equipment through the fusion communication server;
judging the state of a network signal: when the network signal is good, dividing the network signal into a network fluency state, and outputting an audio/video signal by adopting the maximum output power; when the network signals are general, the network signals are divided into a network good state, and the weights occupied by the audio signals and the video signals are regulated in an equalizing way; when the network signals are bad, the network delay state is divided, the network self-adaptive mode is started, the weight occupied by the audio signals and the video signals is adjusted, the weight occupied by the audio signals is increased, and the audio call is preferentially ensured.
As a preferred scheme of the optimization method based on the audio-video fusion communication technology, the application comprises the following steps: the fusion method of the audio and video signals based on the time domain comprises the following steps:
wherein t represents time, G (t) represents the fused audio signal and video signal, X i (t) and Y i (t) represents an i-th audio signal and an i-th video signal, respectively, w i ，v i Is the weight of the ith signal in the fusion.
The fusion method of the audio and video signals based on the frequency domain comprises the following steps:
wherein f represents frequency, X (f) and Y (f) represent power spectrums of the audio signal and the video signal after fusion, xi (f) and Yi (f) represent power spectrums of the i-th audio signal and the i-th video signal, respectively, and w i ，v i Is the weight of the ith signal in the fusion.
As a preferred scheme of the optimization method based on the audio-video fusion communication technology, the application comprises the following steps: the signal fusion method for establishing the two-way transmission channel of the local telephone and the external telephone comprises the following steps:
the fusion algorithm formula of the audio signal based on the weighted average method comprises the following steps:
where n is expressed as a position on the time axis, y (n) is the output audio signal, x i (n) represents the i-th input audio signal, w i (n) is a weight value of the ith audio signal at the nth time. In the implementation process of the weighted average method, the corresponding weight coefficient is usually calculated according to the signal-to-noise ratio and the power of different paths of audio signals.
The audio signal is based on a fusion algorithm formula of spectral subtraction:
where k is the position on the frequency axis, Y (k) is the power spectrum of the output audio signal, X i (k) Representing the power spectrum of the ith audio signal, E i (k) And (5) representing the noise power spectrum estimated by the ith audio signal, wherein N is the number of audio channels.
As a preferred scheme of the optimization method based on the audio-video fusion communication technology, the application comprises the following steps: the signal fusion process for establishing the two-way transmission channel of the local telephone and the external telephone comprises the following steps:
the fusion algorithm formula of the audio signal based on the weighted average method comprises the following steps:
where n is expressed as a position on the time axis, y (n) is the output audio signal, x i (n) represents the i-th input audio signal, w i (n) is a weight value of the ith audio signal at the nth time. In the implementation process of the weighted average method, the corresponding weight coefficient is usually calculated according to the signal-to-noise ratio and the power of different paths of audio signals.
The fusion algorithm formula of the audio signal based on the spectral subtraction comprises the following steps:
where k is the position on the frequency axis, Y (k) is the power spectrum of the output audio signal, X i (k) Representing the power spectrum of the ith audio signal, E i (k) And (5) representing the noise power spectrum estimated by the ith audio signal, wherein N is the number of audio channels.
As a preferable scheme of the optimization system based on the audio and video fusion communication technology, the optimization system comprises the following components:
the conference service module is used for applying media resources and controlling conferences and providing a privately-arranged solution;
the call service module is used for acquiring the SIP extension number, registering the extension number acquired by the terminal, applying related resources, realizing the incoming and outgoing of PSTN by interfacing the voice gateway, and broadcasting an intelligent voice menu;
the interactive cooperation service module is used for supporting the whiteboard cooperation and whiteboard annotation service;
the IM state service module is used for processing the display and update of the online state of the user;
the IM conversation buffer service module is used for storing conversation in a buffer, notifying information, synchronizing information in multiple ends and automatically pushing history information to new members of a group;
the IM file service module is used for setting local storage or distributed storage as required, ensuring data security, supporting second transmission and breakpoint continuous transmission, supporting periodic automatic cleaning setting of files and group chat file management;
and the IM service processing service module is used for notifying pushing by the mobile terminal, allowing the mobile terminal to be connected with a third party for pushing, and supporting electronic voting, electronic bulletin authority management and viewing.
As a preferable scheme of the optimization system based on the audio-video fusion communication technology, the application comprises the following steps: the conference module includes: a session control service, a conference scheduling service, an SFU media forwarding service, an interactive live broadcast service, a live broadcast push stream service and a recorded broadcast service.
The call service module comprises: user center, voice business service, voice media service, voice load service.
The IM file service module is set to be local storage or distributed storage according to the requirement, so that the data security is ensured; and (3) transmitting files and pictures, and periodically and automatically cleaning and setting the files and managing the group chat files through second transmission and breakpoint continuous transmission.
A computer device, comprising: a memory and a processor; the memory stores a computer program characterized in that: the processor, when executing the computer program, implements the steps of the method of any of the present application.
A computer-readable storage medium having stored thereon a computer program, characterized by: which when executed by a processor, carries out the steps of the method described in the application.
The application has the beneficial effects that: the optimization method and the system based on the audio and video fusion communication technology provided by the application are innovative in interconnection among various types of communication, and have the advantages of open compatibility, interconnection among various types of systems, rich access modes and the like. The integration of a voice system, a video system, an IM system and the like is realized on a unified platform, and a user can realize all business communication requirements on one set of system; the method can be compatible with original analog phones, IP phones and traditional SIP/H.323 video conference terminals of users; the method supports rich access modes such as a computer client, a mobile phone APP, an H5 client, an analog phone, an IP phone, monitoring equipment, a video conference terminal, live broadcast and the like; and the single sign-on authentication is provided, the H5 mode is integrated with a third party system, and H5 of each system can be quickly accessed, so that the unification of the entrance is realized.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. Wherein:
fig. 1 is an overall flowchart of an optimization method based on an audio/video fusion communication technology according to a first embodiment of the present application;
fig. 2 is a schematic diagram of a centralized deployment scenario of an optimization method based on an audio/video fusion communication technology according to a first embodiment of the present application;
fig. 3 is a schematic diagram of centralized deployment scenario optimization of an optimization method based on an audio/video fusion communication technology according to a third embodiment of the present application.
Detailed Description
So that the manner in which the above recited objects, features and advantages of the present application can be understood in detail, a more particular description of the application, briefly summarized above, may be had by reference to the embodiments, some of which are illustrated in the appended drawings. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments of the present application without making any inventive effort, shall fall within the scope of the present application.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application, but the present application may be practiced in other ways other than those described herein, and persons skilled in the art will readily appreciate that the present application is not limited to the specific embodiments disclosed below.
Further, reference herein to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic can be included in at least one implementation of the application. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments.
While the embodiments of the present application have been illustrated and described in detail in the drawings, the cross-sectional view of the device structure is not to scale in the general sense for ease of illustration, and the drawings are merely exemplary and should not be construed as limiting the scope of the application. In addition, the three-dimensional dimensions of length, width and depth should be included in actual fabrication.
Also in the description of the present application, it should be noted that the orientation or positional relationship indicated by the terms "upper, lower, inner and outer", etc. are based on the orientation or positional relationship shown in the drawings, are merely for convenience of describing the present application and simplifying the description, and do not indicate or imply that the apparatus or elements referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as limiting the present application. Furthermore, the terms "first, second, or third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
The terms "mounted, connected, and coupled" should be construed broadly in this disclosure unless otherwise specifically indicated and defined, such as: can be fixed connection, detachable connection or integral connection; it may also be a mechanical connection, an electrical connection, or a direct connection, or may be indirectly connected through an intermediate medium, or may be a communication between two elements. The specific meaning of the above terms in the present application will be understood in specific cases by those of ordinary skill in the art.
Example 1
Referring to fig. 1-2, for a first embodiment of the present application, an optimization method based on an audio/video fusion communication technology is provided, including:
s1: adopting a centralized deployment scheme, deploying a converged communication server platform by a central node;
further, each branch office and mobile user access the headquarter platform via the internet.
Further, the devices of each branch office may optionally employ various modes such as IP (Internet Protocol internet protocol) phones, video conference terminals, software clients, etc.
It should be noted that in the conventional technical architecture, the IP phones can only communicate in the same type of device, but cannot be combined with the video conference, and cannot communicate with each other due to inconsistent codec between the devices.
Further, judging the type of the connected communication equipment:
when the types of the connected communication equipment are the same, dividing the communication equipment into similar equipment sets, directly outputting signals without translation coding and decoding, and directly distributing media streams to all the communication equipment through the fusion communication server;
when the types of the connected communication equipment are different, the communication equipment is divided into different equipment sets, and translation is carried out according to audio/video coding and decoding of different communication equipment;
further, a communication channel is established in the different types of equipment sets, and whether the connected communication equipment supports video capability is judged:
when the connected communication equipment does not support the video capability, the communication equipment which does not support the video capability is divided into a call equipment set only, and the communication equipment which supports the audio and video capability is divided into a call and video equipment set;
furthermore, aiming at the call equipment set, eliminating the video stream in the media stream, decoding the audio and video signals by using different equipment specific audio and video codecs, and distributing the media stream to the connected communication equipment through the converged communication server;
furthermore, for the call and video equipment set, the weighted average method is used for carrying out weighted fusion on the audio and video signals in the media stream, then the audio and video signals are decoded by using specific audio and video codecs of different equipment, and the media stream is distributed to the connected communication equipment through the fusion communication server;
further, judging the state of the network signal: when the network signal is good, dividing the network signal into a network fluency state, and outputting an audio/video signal by adopting the maximum output power; when the network signals are general, the network signals are divided into a network good state, and the weights occupied by the audio signals and the video signals are regulated in an equalizing way; when the network signals are bad, the network delay state is divided, the network self-adaptive mode is started, the weight occupied by the audio signals and the video signals is adjusted, the weight occupied by the audio signals is increased, and the audio call is preferentially ensured.
Furthermore, the fusion method of the audio and video signals based on the time domain comprises the following steps:
wherein t represents time, G (t) represents the fused audio signal and video signal, xi (t) and Yi (t) represent the ith audio signal and the ith video signal, respectively, and w i ，v i Is the weight of the ith signal in the fusion.
Furthermore, the fusion method of the audio and video signals based on the frequency domain comprises the following steps:
wherein f represents frequency, X (f) and Y (f) represent power spectrums of the audio signal and the video signal after fusion, xi (f) and Yi (f) represent power spectrums of the i-th audio signal and the i-th video signal, respectively, and w i ，v i Is the weight of the ith signal in the fusion.
For example, the video conference terminal uses h.264hp (a video codec technical standard named by the international telecommunication union telecommunication standard office under the name of h.26x series) and g.711 (a speech compression standard customized by the international telecommunication union telecommunication standard office), and the IP phone uses OPUS (a format of sound coding standardized by the internet engineering task force) for audio codec, which does not support video codec, and the codecs between the two devices are inconsistent and cannot communicate with each other. Through the optimization of the audio and video fusion communication technology, media nodes of the fusion communication server are respectively and independently connected with two devices in a corresponding encoding and decoding format, the fusion communication server is used for completing the forwarding of corresponding audio and video, the audio stream of the IP telephone is transcoded into a G.711 format and sent to the video conference terminal, the video stream of the video conference terminal is removed, and the audio stream is transcoded into an OPUS and sent to the IP telephone, so that the mutual communication between the video conference terminal and the IP telephone is realized. Similarly, in a conference, different similar devices can be supported to communicate with each other, when one device needs to receive a media stream, the fusion communication server can integrate and transmit audio and video streams of other devices, and if the device does not support video capability, the video stream in the media stream is removed.
The present embodiment also provides a computing device comprising, a memory and a processor; the memory is used for storing computer executable instructions, and the processor is used for executing the computer executable instructions to realize the conventional substation monitoring information automatic verification method according to the embodiment.
The present embodiment also provides a storage medium having stored thereon a computer program which, when executed by a processor, implements a conventional substation monitoring information automatic verification method as set forth in the above embodiments.
The storage medium proposed in this embodiment belongs to the same inventive concept as a conventional substation monitoring information automatic verification method proposed in the above embodiment, and technical details not described in detail in this embodiment can be seen in the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile memory may include read only memory, magnetic tape, floppy disk, flash memory, optical memory, high density embedded nonvolatile memory, resistive memory, magnetic memory, ferroelectric memory, phase change memory, graphene memory, and the like. Volatile memory can include random access memory, external cache memory, or the like. By way of illustration, and not limitation, RAM can take many forms, such as static random access memory or dynamic random access memory. The databases referred to in the embodiments provided herein may include at least one of a relational database and a non-relational database. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processor referred to in the embodiments provided in the present application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, or the like, but is not limited thereto.
S2: establishing a two-way transmission channel of local telephone and external telephone, and initiating an audio-video conference;
further, the voice gateway is connected through the SIP relay link, and the converged communication server transmits the call of the local call to the voice gateway and further to an operator;
note that SIP is collectively referred to as Session initialization Protocol session initiation protocol.
Still further, the voice gateway supports E1 (a telecommunications standard) line access.
Furthermore, when the local telephone calls the E1 line number, the operator also transmits the call to the voice gateway and forwards the call to the converged communication server, so that the call function of the local telephone access and PSTN (Public Switched Telephone Network public switched telephone network) is realized, and the demand for the local telephone is met;
furthermore, in the conference, the conference control module can invite the local telephone to enter the conference and form an audio call, and the user can directly dial an external telephone to carry out the conference through the mobile phone.
After the further local telephone enters the conference, the audio stream is transmitted to enter the conference, and the pictures of other equipment (video equipment, telephone equipment and the like) in the conference are combined to form a uniform picture layout, so that a complete one-way audio and video media stream is formed and transmitted to equipment to be received; meanwhile, the converged communication server can integrate and remove the video stream part of the media streams of other devices and transmit the video stream part to the local telephone.
Furthermore, the fusion algorithm formula of the audio signal based on the weighted average method comprises the following steps:
where n is the position on the time axis, y (n) is the output audio signal, xi (n) is the i-th input audio signal, and wi (n) is the weight value of the i-th audio signal at the n-th time. In the implementation process of the weighted average method, the corresponding weight coefficient is usually calculated according to the signal-to-noise ratio and the power of different paths of audio signals.
Further, the fusion algorithm formula based on the spectral subtraction is as follows:
where k is the position on the frequency axis, Y (k) is the power spectrum of the output audio signal, X i (k) Representing the power spectrum of the ith audio signal, E i (k) And (5) representing the noise power spectrum estimated by the ith audio signal, wherein N is the number of audio channels. If the value of Y (k) is greater than 0, the output power spectrum output value is Y (k), and if the value of Y (k) is less than 0, the output power spectrum output value is 0.
It should be noted that, in the conventional video conference, access of a standard local telephone cannot be supported.
S3: synchronizing address book information to a converged communication server;
further, through the authentication of the LDAP server, the LDAP server synchronizes address book information to the converged communication server after authentication;
it should be noted that, the converged communication server supports the docking of an LDAP (Lightweight Directory Access Protocol light directory access protocol) server, synchronizes address book information to the converged communication server after authentication by the LDAP server, but does not store information such as passwords and the like, only stores a user login name, and when a user needs to log in on the converged communication server, the converged communication server transmits authentication information to the LDAP server and waits for an authentication result of the LDAP server. The automatic synchronization of the enterprise organization architecture and the contact information is realized, and the requirement that only one enterprise address book is managed among enterprise multisystems is met;
furthermore, the scheme can meet the convenience requirement of enterprise management staff account numbers, an administrator can rapidly add, delete and edit user account numbers, and the user only needs to record one account number information.
It should be noted that, the account systems of the conventional communication system are independent, different accounts and passwords are required to be recorded for different systems, and an administrator also needs to manage a plurality of account systems respectively.
S4: and accessing the equipment of the headquarter to the converged communication server for accessing, and accessing the equipment of the subsection to the converged communication server for accessing the headquarter system.
Furthermore, the desktop telephone of the headquarter, the video terminal of the conference room and the desktop computer end of the user access to the converged communication server through the internal network access of the headquarter enterprise, and the identity authentication can be completed through the account number.
Furthermore, users of the branches access the headquarter system through the enterprise private network/Internet to the converged communication server, and the interconnection between the headquarter and the branches can be realized through the account number;
furthermore, the external business users access to the converged communication system through 3G (third generation mobile communication technology)/4G (fourth generation mobile communication technology)/5G (fifth generation mobile communication technology), the Internet, wireless local area network and the like on the computer end/mobile end/webpage end, and the identity authentication can be completed through the account number, so that the mobile office scene is realized.
Example 2
The second embodiment of the present application provides an optimization system based on an audio/video fusion communication technology, comprising: the system comprises a conference service module, a call service module, an interactive collaboration service module, an IM (Instant Messenger instant messaging) state service module, an IM session cache service module, an IM file service module and an IM business processing service module.
Still further, the conference service module further comprises: a conference control service, a conference scheduling service, a SFU (Selective Forwarding Unit) media forwarding service, an interactive live broadcast service, a live broadcast push stream service and a recorded broadcast service.
After the extension registration is completed, a video conference can be initiated, and the application of media resources and the control of the conference are performed through a conference control service and a conference scheduling service. The privately-distributed solution is provided, the highest 3+N master-slave servers are supported, any node fails and automatically migrates within 3 minutes, the conference is not perceived and continuously recovered, and the influence on service continuity is reduced. The dual-standard protocol of SIP (Session initialization Protocol session initiation protocol)/H.323 (part of ITU multimedia communication series standard H.32x) is supported, and the dual-standard protocol is widely compatible with mainstream video conference brand terminals.
It should be noted that the communication and collaboration platform has the advantages of large concurrency, cluster disaster recovery, smooth capacity expansion, flexible deployment and the like, and meets the converged communication demands of various enterprises, especially large-scale government enterprises, multi-level organizations, thereby reducing the cost and enhancing the efficiency.
Still further, the talk service module IP-PBX (Internet Protocol-Private Branch eXchange internet protocol-private branch exchange) also includes user centers, voice traffic services (registration, gateway mediation, interactive voice response services), voice media services, voice load services, and the like.
Furthermore, through the user center, the converged communication terminal can acquire the SIP (Session initialization Protocol session initiation protocol) extension number after logging in; the extension number acquired by the terminal can be registered through the voice service. When an extension number performs P2P (Point-to-Point) call or initiates a video conference, related resources can be applied to voice media service through voice service; when the call of the internal and external networks is involved, intelligent wall riding can be performed, and when the call of the extension telephone outside fixed line and the mobile phone number are involved, the voice gateway can be docked through voice service (gateway mediation service), so that PSTN (Public Switched Telephone Network public switched telephone network) incoming and outgoing call can be realized; when the outside line number calls into the voice gateway, intelligent voice menu broadcasting is performed through voice service IVR (Interactive Voice Response interactive voice response) service, and the voice gateway can be switched to an internal extension according to voice prompt.
Furthermore, the interactive collaboration service module provides collaboration related services for supporting whiteboard collaboration and whiteboard annotation services. The method can support receiving and transmitting auxiliary stream white boards, and enables multiple people to write endorsements, endorsement erasing and the like at the same time, so that the multiple people can cooperate at the same time in the conference process conveniently.
It should be noted that the IM status service module is configured to process the display and update of the online status of the user. The module can determine the real-time online state of the user by detecting various information such as the login state of the user, the network connection condition, whether the application program runs in the foreground or not and the like, and displays the real-time online state on interfaces such as a user list, a chat window and the like.
It should be noted that, the IM session cache service module supports point-to-point and group messages, stores the session in the cache, and the chat message can support ten thousand concurrent numbers, withdraw the message within one minute, notify the message, synchronize the message multiple ends, and automatically push the history message to the new member of the group.
It should be noted that the IM file service module supports setting to local storage or distributed storage as required to ensure data security. And the transmission of files and pictures supports second transmission and breakpoint continuous transmission, and the files are automatically cleaned and set at regular intervals and are subjected to group chat file management.
It should be noted that, the IM service processing service module may support mobile notification pushing by processing the IM related service, allow access to third party pushing, and support electronic voting, electronic bulletin rights management, and viewing.
Example 3
Referring to fig. 3, a third embodiment of the present application is scientifically demonstrated through benefit calculation and simulation experiments in order to verify the beneficial effects of the present application.
First, for the method of the above embodiment, the method is applied to compare the deployment mode (before optimization) of the fusion communication system, which is isolated from the conventional audio, video and IM (Instant Messenger instant messaging) systems.
Operating environment: network: intranet and internet, operating system: centrOS7.5, number of CPUs (Central Processing Unit Central processing units): 2, total kernel number of CPU: 16, bus runlengths: 32, memory: 64GB, disk space: 500GB
The testing method comprises the following steps: the converged communication system optimized by the patent is deployed on a single server, is communicated with an E1 (a telecommunication standard) gateway through a Session Initiation Protocol (SIP) relay mode and is connected to an operator telephone line, is connected with an analog gateway through the SIP relay mode and is used for connecting an analog telephone, is directly connected with a video conference terminal, an IP telephone and a computer/mobile phone through an Internet protocol (Internet Protocol) network, is configured with the mapping of the Internet for external network connection, and is used for testing the mutual calling of different types of equipment such as the IP telephone, the computer/mobile phone client and the video conference terminal, and verifying the specific interconnection and interworking effect.
The experiment mainly verifies the communication state between the devices, and can realize audio or video call without actual data presentation.
Experimental results: through the patent optimization, the converged communication system can realize the audio/video call between the analog telephone, the video conference terminal, the IP telephone and the computer/mobile phone client, and the software client supports the IM (Instant Messenger instant messaging) communication function.
Therefore, the verification user can freely select IP phones, computer/mobile phone clients, video conference terminals and other devices to communicate, the defect that the traditional communication mode needs to specify communication devices is avoided, and the communication efficiency can be improved by more than 50%.
Compared with the traditional method for mutually isolating the audio system, the video system and the IM system, the method can promote the type of the audio-video call supporting access equipment of the system, promote the situation that the original audio system, the video system and the IM system are mutually isolated and can not be connected to each other to support interconnection and intercommunication, and a user can arbitrarily select equipment and modes to carry out business communication according to requirements, so that the office communication efficiency is improved.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile memory may include read only memory, magnetic tape, floppy disk, flash memory, optical memory, high density embedded nonvolatile memory, resistive memory, magnetic memory, ferroelectric memory, phase change memory, graphene memory, and the like. Volatile memory can include random access memory, external cache memory, or the like. By way of illustration, and not limitation, RAM can take many forms, such as static random access memory or dynamic random access memory. The databases referred to in the embodiments provided herein may include at least one of a relational database and a non-relational database. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processor referred to in the embodiments provided in the present application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, or the like, but is not limited thereto.
It should be noted that the above embodiments are only for illustrating the technical solution of the present application and not for limiting the same, and although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present application may be modified or substituted without departing from the spirit and scope of the technical solution of the present application, which is intended to be covered in the scope of the claims of the present application.
Claims (10)
1. An optimization method based on an audio-video fusion communication technology is characterized by comprising the following steps:
adopting a centralized deployment scheme, deploying a converged communication server platform by a central node;
establishing a two-way transmission channel of local telephone and external telephone, and initiating an audio-video conference;
synchronizing address book information to the converged communication server;
and accessing the equipment of the headquarter to the converged communication server for accessing, and accessing the equipment of the subsection to the converged communication server for accessing the headquarter system.
2. The optimization method based on the audio-video fusion communication technology as set forth in claim 1, wherein: the two-way transmission channel of the local telephone and the external telephone comprises:
docking a voice gateway through an SIP relay link;
the converged communication server transmits an external telephone for calling the local telephone to the voice gateway;
the voice gateway transmitting the outside line telephone to an operator;
and when the local call is made to the outside line telephone, the local call is returned through the original path of the operator.
3. The optimization method based on the audio-video fusion communication technology as set forth in claim 1, wherein: the process of synchronizing address book information to the converged communication server comprises the following steps:
the converged communication server is in butt joint with an LDAP server;
the converged communication server transmits authentication information to the LDAP server;
and synchronizing address book information to the converged communication server after the LDAP server authenticates.
4. The optimization method based on the audio-video fusion communication technology as set forth in claim 1, wherein: the audio and video fusion process of the fusion communication server comprises the following steps:
judging the type of the connected communication equipment: when the types of the connected communication equipment are the same, dividing the communication equipment into similar equipment sets, directly outputting signals without translation coding and decoding, and directly distributing media streams to all the communication equipment through the fusion communication server; when the types of the connected communication equipment are different, the communication equipment is divided into different equipment sets, and translation is carried out according to audio/video coding and decoding of different communication equipment;
establishing communication channels in different types of equipment sets, and judging whether the connected communication equipment supports video capability or not: when the connected communication equipment does not support the video capability, the communication equipment which does not support the video capability is divided into a call equipment set only, and the communication equipment which supports the audio and video capability is divided into a call and video equipment set;
aiming at a call equipment only set, eliminating video streams in the media streams, decoding audio and video signals by using different equipment specific audio and video codecs, and distributing the media streams to connected communication equipment through the converged communication server;
aiming at a call and video equipment set, carrying out weighted fusion on audio and video signals in a media stream by using a weighted average method, decoding the audio and video signals by using specific audio and video codecs of different equipment, and distributing the media stream to the connected communication equipment through the fusion communication server;
judging the state of a network signal: when the network signal is good, dividing the network signal into a network fluency state, and outputting an audio/video signal by adopting the maximum output power; when the network signals are general, the network signals are divided into a network good state, and the weights occupied by the audio signals and the video signals are regulated in an equalizing way; when the network signals are bad, the network delay state is divided, the network self-adaptive mode is started, the weight occupied by the audio signals and the video signals is adjusted, the weight occupied by the audio signals is increased, and the audio call is preferentially ensured.
5. The optimization method based on the audio-video fusion communication technology as set forth in claim 4, wherein: the fusion method of the audio and video signals based on the time domain comprises the following steps:
wherein t represents time, G (t) represents the fused audio signal and video signal, xi (t) and Yi (t) represent the ith audio signal and the ith video signal, respectively, and w i ，v i The weight of the ith signal in fusion;
the fusion method of the audio and video signals based on the frequency domain comprises the following steps:
wherein f represents frequency, X (f) and Y (f) represent power spectrums of the audio signal and the video signal after fusion, xi (f) and Yi (f) represent power spectrums of the i-th audio signal and the i-th video signal, respectively, and w i ，v i Is the weight of the ith signal in the fusion.
6. The optimization method based on the audio-video fusion communication technology as set forth in claim 1, wherein: the signal fusion method for establishing the two-way transmission channel of the local telephone and the external telephone comprises the following steps:
the fusion algorithm formula of the audio signal based on the weighted average method comprises the following steps:
where n is expressed as a position on the time axis, y (n) is the output audio signal, x i (n) represents the i-th input audio signal, w i (n) is the weight value of the ith audio signal at the nth time; in the implementation process of the weighted average method, corresponding weight coefficients are calculated according to the signal-to-noise ratios and the power of different paths of audio signals;
the audio signal is based on a fusion algorithm formula of spectral subtraction:
where k is the position on the frequency axis, Y (k) is the power spectrum of the output audio signal, X i (k) Representing the power spectrum of the ith audio signal, E i (k) And (5) representing the noise power spectrum estimated by the ith audio signal, wherein N is the number of audio channels.
7. An optimization system based on an audio-video fusion communication technology is characterized by comprising:
the conference service module is used for applying media resources and controlling conferences and providing a privately-arranged solution;
the call service module is used for acquiring the SIP extension number, registering the extension number acquired by the terminal, applying related resources, realizing the incoming and outgoing of PSTN by interfacing the voice gateway, and broadcasting an intelligent voice menu;
the interactive cooperation service module is used for supporting the whiteboard cooperation and whiteboard annotation service;
the IM state service module is used for processing the display and update of the online state of the user;
the IM conversation buffer service module is used for storing conversation in a buffer, notifying information, synchronizing information in multiple ends and automatically pushing history information to new members of a group;
the IM file service module is used for setting local storage or distributed storage as required, ensuring data security, supporting second transmission and breakpoint continuous transmission, supporting periodic automatic cleaning setting of files and group chat file management;
and the IM service processing service module is used for notifying pushing by the mobile terminal, allowing the mobile terminal to be connected with a third party for pushing, and supporting electronic voting, electronic bulletin authority management and viewing.
8. The optimization system based on the audio-video fusion communication technology as set forth in claim 7, wherein: the conference module includes: a meeting control service, a meeting scheduling service, an SFU media forwarding service, an interactive live broadcast service, a live broadcast push stream service and a recorded broadcast service;
the call service module comprises: user center, voice business service, voice media service, voice load service;
the IM file service module is set to be local storage or distributed storage according to the requirement, so that the data security is ensured; and (3) transmitting files and pictures, and periodically and automatically cleaning and setting the files and managing the group chat files through second transmission and breakpoint continuous transmission.
9. A computer device, comprising: a memory and a processor; the memory stores a computer program characterized in that: the processor, when executing the computer program, implements the steps of the method of any one of claims 1 to 8.
10. A computer-readable storage medium having stored thereon a computer program, characterized by: the computer program implementing the steps of the method of any one of claims 1 to 8 when executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310442320.1A CN117135150A (en) | 2023-04-23 | 2023-04-23 | Optimization method and system based on audio and video fusion communication technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310442320.1A CN117135150A (en) | 2023-04-23 | 2023-04-23 | Optimization method and system based on audio and video fusion communication technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117135150A true CN117135150A (en) | 2023-11-28 |
Family
ID=88857088
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310442320.1A Pending CN117135150A (en) | 2023-04-23 | 2023-04-23 | Optimization method and system based on audio and video fusion communication technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117135150A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117440440A (en) * | 2023-12-21 | 2024-01-23 | 艾康恩(深圳)电子科技有限公司 | Bluetooth headset low-delay transmission method |
-
2023
- 2023-04-23 CN CN202310442320.1A patent/CN117135150A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117440440A (en) * | 2023-12-21 | 2024-01-23 | 艾康恩(深圳)电子科技有限公司 | Bluetooth headset low-delay transmission method |
| CN117440440B (en) * | 2023-12-21 | 2024-03-15 | 艾康恩(深圳)电子科技有限公司 | Bluetooth headset low-delay transmission method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10715672B2 (en) | Social networking-based teleconferencing system and method | |
| US10171521B2 (en) | Seamlessly conferencing a previously-connected telephone call | |
| US7764632B2 (en) | Software bridge for multi-point multi-media teleconferencing and telecollaboration | |
| US11855795B1 (en) | Communication bridging in a remote office environment | |
| CN101090475B (en) | Conference layout controls and control protocol | |
| CN101536511B (en) | System and method for single action initiation of a video conference | |
| CN1633652B (en) | Videoconference system playback controlling system, server and display method | |
| CN107995456A (en) | Wisdom garden video conferencing system | |
| US8245043B2 (en) | Audio start service for Ad-hoc meetings | |
| CN101090328A (en) | Associating independent multimedia sources into a conference call | |
| US20110150200A1 (en) | Web guided collaborative audio | |
| CN101828375B (en) | Method and system for real-time media synchronisation across a network | |
| CN101090329A (en) | Intelligent audio limit method, system and node | |
| US10129304B2 (en) | Systems and methods for making common services available across network endpoints | |
| CN111050000A (en) | Method, device and system for realizing soft phone of call center | |
| CN103888712A (en) | Multilingual synchronous audio and video conference system | |
| JP2015165645A (en) | Transmission system, transmission management device and program | |
| CN117135150A (en) | Optimization method and system based on audio and video fusion communication technology | |
| US20070180029A1 (en) | Method and apparatus for restricting visual material for conference services | |
| JP5877470B2 (en) | Commercial communication system and method | |
| Rosas et al. | Videoconference system based on WebRTC with access to the PSTN | |
| WO2022203891A1 (en) | Method and system for integrating video content in a video conference session | |
| CN109167885A (en) | It is attended a banquet the method for between Double tabletop seamless login based on T-Lib agreement | |
| RU2631255C1 (en) | System and method for telephone communication on internet | |
| US20240146560A1 (en) | Participant Audio Stream Modification Within A Conference |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |