CN117650932A - Multilayer safe real-time audio and video method and device - Google Patents
Multilayer safe real-time audio and video method and device Download PDFInfo
- Publication number
- CN117650932A CN117650932A CN202311648152.8A CN202311648152A CN117650932A CN 117650932 A CN117650932 A CN 117650932A CN 202311648152 A CN202311648152 A CN 202311648152A CN 117650932 A CN117650932 A CN 117650932A
- Authority
- CN
- China
- Prior art keywords
- video
- audio
- real
- time
- client
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 230000006854 communication Effects 0.000 claims abstract description 38
- 238000004891 communication Methods 0.000 claims abstract description 35
- 230000007246 mechanism Effects 0.000 claims abstract description 34
- 238000012795 verification Methods 0.000 claims abstract description 20
- 238000012544 monitoring process Methods 0.000 claims abstract description 10
- 206010000117 Abnormal behaviour Diseases 0.000 claims abstract description 4
- 230000005540 biological transmission Effects 0.000 claims description 21
- 238000004590 computer program Methods 0.000 claims description 5
- 230000002159 abnormal effect Effects 0.000 claims description 3
- 230000006399 behavior Effects 0.000 claims description 3
- 230000002708 enhancing effect Effects 0.000 abstract description 2
- 238000005516 engineering process Methods 0.000 description 32
- 230000006870 function Effects 0.000 description 31
- 238000012545 processing Methods 0.000 description 10
- 238000012937 correction Methods 0.000 description 7
- 239000008186 active pharmaceutical agent Substances 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000013475 authorization Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 4
- 230000006835 compression Effects 0.000 description 4
- 238000007906 compression Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 230000009467 reduction Effects 0.000 description 4
- 230000001953 sensory effect Effects 0.000 description 3
- 230000003044 adaptive effect Effects 0.000 description 2
- 230000033228 biological regulation Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 238000012856 packing Methods 0.000 description 2
- 230000000750 progressive effect Effects 0.000 description 2
- 238000011084 recovery Methods 0.000 description 2
- 238000000926 separation method Methods 0.000 description 2
- 230000011664 signaling Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000006735 deficit Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 210000005069 ears Anatomy 0.000 description 1
- 210000003128 head Anatomy 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 238000003908 quality control method Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/08—Network architectures or network communication protocols for network security for authentication of entities
- H04L63/0884—Network architectures or network communication protocols for network security for authentication of entities by delegation of authentication, e.g. a proxy authenticates an entity to be authenticated on behalf of this entity vis-à-vis an authentication entity
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/08—Network architectures or network communication protocols for network security for authentication of entities
- H04L63/0807—Network architectures or network communication protocols for network security for authentication of entities using tickets, e.g. Kerberos
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/08—Network architectures or network communication protocols for network security for authentication of entities
- H04L63/0876—Network architectures or network communication protocols for network security for authentication of entities based on the identity of the terminal or configuration, e.g. MAC address, hardware or software configuration or device fingerprint
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/10—Network architectures or network communication protocols for network security for controlling access to devices or network resources
- H04L63/105—Multiple levels of security
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/1066—Session management
- H04L65/1083—In-session procedures
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/40—Support for services or applications
- H04L65/403—Arrangements for multi-party communication, e.g. for conferences
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
- H04L9/321—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials involving a third party or a trusted authority
- H04L9/3213—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials involving a third party or a trusted authority using tickets or tokens, e.g. Kerberos
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
- H04L9/3247—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials involving digital signatures
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/40—Network security protocols
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Business, Economics & Management (AREA)
- General Business, Economics & Management (AREA)
- Power Engineering (AREA)
- Telephonic Communication Services (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention provides a multilayer safe real-time audio and video method and a device, wherein the method comprises the following steps: s1, an identity verification mechanism is established, and the identity verification mechanism is deployed in a cloud server; s2, establishing audio and video connection between a cloud server and a client based on the RTC; s3, generating an access token of the client according to the identity authentication mechanism, and determining the authority of the client to join the video conference according to the information of the access token; and S4, tracking communication abnormal behaviors of the video conference based on a real-time monitoring algorithm and giving an alarm in real time. The invention is used for enhancing the accuracy and the safety of user identity verification, providing multi-level protection in the whole communication process, and solving the safety problem of real-time audio and video communication in the prior art.
Description
Technical Field
The invention belongs to the technical field of computers, and particularly relates to a multilayer safe real-time audio/video method and device.
Background
In the field of modern communications, audio and video technology on the cloud has been widely used in a variety of industries including education, medical treatment, enterprise management, and the like. The audio and video communication method can provide low-delay and high-efficiency real-time audio and video communication experience for users, and greatly facilitates remote communication and cooperation of people. However, despite these significant advantages, existing real-time audio-video communication methods have certain limitations in terms of security.
The real-time audio and video communication method in the prior art is not strict enough for the identity verification of the participants, is easy to cause unauthorized user access, increases the risks of communication content leakage and abuse, ensures insufficient safety, limits the practicability of the communication in specific application scenes, and is particularly prominent in the fields requiring high confidentiality and safety guarantee.
In view of this, the present invention has been made.
Disclosure of Invention
In view of the above, the invention discloses a multi-layer safe real-time audio/video method and device, which are used for enhancing the accuracy and safety of user identity verification, providing multi-layer protection in the whole communication process and solving the safety problem of real-time audio/video communication in the prior art.
Specifically, the invention is realized by the following technical scheme:
in a first aspect, the invention discloses a multilayer safe real-time audio/video method, which comprises the following steps:
s1, an identity verification mechanism is established, and the identity verification mechanism is deployed in a cloud server;
s2, establishing audio and video connection between a cloud server and a client based on the RTC;
s3, generating an access token of the client according to the identity authentication mechanism, and determining the authority of the client to join the video conference according to the information of the access token;
and S4, tracking communication abnormal behaviors of the video conference based on a real-time monitoring algorithm and giving an alarm in real time.
Further, in the step S1, the authentication mechanism includes:
acquiring an App ID and generating an App Key corresponding to the App ID;
generating unique identification signatures, expire Time and Nonce values based on a Signature generation algorithm by combining the App ID, the App ID and the User ID;
calling a Login interface of the client SDK, and carrying out identity information authentication according to the User ID, the identification signature, the Expire Time and the Nonce value.
Further, in the step S3, the access token includes identity information and conference access rights.
Further, in the step S2, the method for establishing the audio/video connection between the cloud server and the client based on the RTC includes:
the transmitting end packages the data into a data packet according to a certain rule and transmits the data packet to the receiving end;
the receiving end unpacks the data packet into original data according to the head information of the data packet;
and transmitting the original data according to a specific sequence and time interval.
Further, the transmitting end and the receiving end establish connection according to a handshake mechanism.
Further, in the data transmission process, the sequence and low-delay characteristics of the original data are maintained based on the TCP protocol and the UDP protocol.
In a second aspect, the present invention discloses a multi-layer secure real-time audio-video device, comprising:
and an identity authentication mechanism establishment and deployment module: establishing an identity verification mechanism, and deploying the identity verification mechanism in a cloud server;
and the audio and video connection module is used for: establishing audio and video connection between a cloud server and a client based on the RTC;
and the permission generation and judgment module is used for: generating an access token of the client according to the identity authentication mechanism, and determining the authority of the client to join the video conference according to the information of the access token;
and the monitoring alarm module: and tracking the abnormal communication behavior of the video conference based on a real-time monitoring algorithm and giving an alarm in real time.
In a third aspect, the present invention discloses a computer readable storage medium having stored thereon a computer program which when executed by a processor implements the steps of the multi-layered secure real-time audiovisual method of the first aspect.
In a fourth aspect, the present invention discloses a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, said processor implementing the steps of the multi-layer secure real-time audiovisual method according to the first aspect when said program is executed.
Compared with the prior art, the invention has the beneficial effects that:
according to the multilayer safe real-time audio and video method and device, through setting the user identity verification and authorization mechanism, only legal users can access and use the audio and video communication function, meanwhile, the application integration difficulty is reduced, and the safety of an open interface is improved; real-time audio and video transmission between the cloud server and the client is realized by using the RTC technology, and data transmission and receiving can be performed according to a certain sequence and time interval in a data packing and unpacking mode.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
FIG. 1 is a flowchart of a multi-layer secure real-time audio/video method according to an embodiment of the present invention;
FIG. 2 is a flowchart of a method for authentication and authorization according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating the operation of conference management according to an embodiment of the present invention;
fig. 4 is a schematic diagram of an audio/video communication structure according to an embodiment of the present invention;
FIG. 5 is a diagram of an internal RTC standardized architecture according to an embodiment of the present invention;
fig. 6 is a flowchart of an audio/video communication process according to an embodiment of the present invention;
fig. 7 is a schematic diagram of a multi-layer secure real-time audio/video device according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of a computer device according to an embodiment of the present invention.
Detailed Description
The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings and detailed description, but it will be understood by those skilled in the art that the examples described below are some, but not all, examples of the present invention, and are intended to be illustrative of the present invention only and should not be construed as limiting the scope of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In order to more clearly illustrate the technical scheme of the invention, the following description is given by way of specific examples.
Examples
Referring to fig. 1, the invention discloses a multilayer safe real-time audio and video method, which comprises the following steps:
s1, an identity verification mechanism is established, and the identity verification mechanism is deployed in a cloud server;
s2, establishing audio and video connection between a cloud server and a client based on the RTC;
s3, generating an access token of the client according to the identity authentication mechanism, and determining the authority of the client to join the video conference according to the information of the access token;
and S4, tracking communication abnormal behaviors of the video conference based on a real-time monitoring algorithm and giving an alarm in real time.
The actual operation of the invention takes the audio and video connection of the court command center and the intelligent execution APP as an example according to the following flow, and is specifically expressed as follows:
building a cloud server: establishing a cloud server with high performance and expandability, and carrying audio and video communication services;
in the intelligent execution APP of the judge, a conference service port is called for authentication, so that a user identity verification and authorization mechanism is realized, and only legal users can access and use an audio and video communication function; in the application process, the application integration difficulty is reduced, the safety of an open interface is improved, and the service opening capability summary supports an authentication mode of an App ID. The App ID is used as an application identifier, and the same App ID can be used on a desktop terminal, a mobile terminal and a Web application of a third party at the same time.
Referring to fig. 2, an authentication mechanism is established and deployed in a cloud server; before identity authentication, applying for an App ID on a conference console, and acquiring the App ID and an App Key; integrating a Signature generation algorithm at a server, initializing an SDK and transmitting an App ID;
in the authentication process, a third party client sends a User ID to a third party server; the third party server generates Signature materials Signature for authentication according to the App ID, the User ID, the App Key and the like;
returning a Signature, an ExpirieTime and a Nonce value at a third party server;
the third party client calls the Login interface of the client SDK, and parameters are User ID, signature, expireTime and Nonce.
The SDK of the client initiates identity verification and authentication to the conference server, and the SDK acquires an access Token after the authentication is passed. The Token is invisible to third party clients and is responsible for maintenance and refresh by the SDK.
In this embodiment, a conference management function is implemented on a cloud server, including conference creation, joining, and exiting operations. Referring to fig. 3, the functions of the pre-appointment protocol and the instant conference can be realized through conference management, and the specific implementation method comprises the following steps: the conference service opens RESTful APIs to the outside, and integrates the audio and video conference functions into a service system through the APIs; the third party application reserves the conference by inputting related parameters such as an API gateway access address, a conference user account number and the like, and can obtain a conference link after reservation is successful, click a link conference, and the system automatically calls a client to enter the conference so as to realize conference.
And establishing audio and video connection between the cloud server and the client based on the RTC. Real-time audio and video transmission between judges or between judges and command centers is realized by using RTC technology. The audio and video data are transmitted to the equipment of the opposite side through audio and video coding and transmission protocols, so that real-time voice and image communication is realized.
The specific audio/video transmission implementation architecture is as follows, see fig. 4:
the audio and video acquisition module calls a system API, reads equipment from a system microphone and a camera and acquires audio and video data; specifically, the audio-video data includes audio data and video data, the audio data is PCM data, and the video data is YUV data;
the audio and video coding module uses different coding modes according to different types of data to compress and code the original PCM and YUV data;
the network transmission module encapsulates the compressed and encoded data into RTP packets, and transmits the RTP packets to the opposite terminal through the network, and the opposite terminal receives the RTP data;
the audio/video decoding module restores the received compression encoded data into original PCM, YUV, RGB data;
after the audio and video rendering module obtains the original data, the audio data is output to a loudspeaker, and the video data is output to a display.
Based on the audio and video transmission realization architecture, an audio and video communication system is realized, the architecture is realized by the internal standardization of the RTC, and the expansion is carried out on the basis, so that only relevant APIs are exposed to the outside; the specific implementation mode is as follows:
referring to fig. 5, the system is divided into four layers, including: an interface layer, a Session layer, an engine layer and a device layer;
the interface layer is exposed on the service side, and the service side can develop audio and video to realize communication by using a native API interface or a WebAPI;
the Session layer is used for controlling service logic such as media negotiation and collection of Candidate and the like;
the engine layer comprises an audio engine, a video engine and network transmission;
the equipment layer is mainly interacted with hardware and is responsible for audio acquisition and playing, video acquisition, physical network and the like;
specifically, referring to fig. 6, the implementation process of the audio/video communication is as follows:
the two communication parties are named as a caller and a callee respectively, the internal logic of the two sides is similar, and the caller end is taken as an example, and the specific flow is as follows:
invoking an audio/video detection module to detect whether the terminal has available audio/video equipment;
calling an audio and video acquisition module to acquire audio and video data of a user;
judging whether to start recording or authorization according to the selection of a user;
exchanging SDP through signaling module;
creating a core object RTCPeeerconnection of webRTC, and then adding acquired audio and video data;
RTCPeerConnection sends a request to a STUN (SessionTraversal UtilitiesforNAT)/TURN (Trav ersal Using RelaysaroundNAT) server, and returns the external network ip address and port number of the caller;
the external network ip address and port of the other party are mutually transmitted by the caller and the caller through the signaling server (media negotiation);
and finally, the P2P link is established, so that audio and video data can be continuously sent to the opposite terminal.
The RTC transmits real-time data flow through a network, so that the real-time performance and stability of the data are ensured. In the embodiment, a reliable data transmission mechanism is provided by utilizing a TCP protocol, so that the integrity and the sequence of data are ensured; and the UDP protocol provides low-delay data transmission characteristics, so that the method is suitable for scenes with high real-time requirements.
In a specific embodiment, the method for performing data packing and unpacking includes: the transmitting end packages the transmitted data into data packets according to a certain rule and then transmits the data packets to the receiving end through a network. And the receiving end unpacks the data packet into original data according to the header information of the data packet. This arrangement enables data to be transmitted and received in a certain order and time interval.
The method for realizing data transmission and control comprises the following steps: the sending end and the receiving end establish connection through a handshake mechanism, so that mutual identification and communication of the two parties can be ensured. The transmitting end transmits the data packet to the receiving end, and the receiving end decodes and displays the data by receiving and processing the data packet. Meanwhile, the RTC realizes the control of data transmission through control information, and specifically comprises the operations of opening and closing data streams, packet loss retransmission and the like.
In a specific embodiment, the real-time audio and video method of the invention provides high-quality audio and video communication experience by selecting a proper coding and decoding algorithm and optimizing network transmission through communication quality control.
In terms of video, a full high-definition video is provided, a 1080P double-flow high-definition scheme is supported, and the following high-definition standards are supported: 720P (1280×720) @25/30fps, 720P (1280×720) @50/60fps, 1080P (1920×1080) @25/30fps, 4K (3840×2160) (P: progressive, i.e., progressive scan). The method supports the enhancement VME (Video Motion Enhancement) of the active video, and the images are sharper and clearer under the same illumination and lens conditions through the performances of video front-back processing, noise reduction, contrast enhancement, edge enhancement and the like; the image compression ratio is improved by matching with the latest hardware processing chip and the optimized H.264HP encoding and decoding algorithm, and more vivid, clearer and smoother pictures can be provided for users under the same bandwidth.
In terms of audio, the standard definition video conference system can reach the highest audio coding and decoding of 14KHz, the high definition video conference system can reach the audio coding and decoding of more than 20KHz, all sound ranges which can be heard by human ears are covered, the sound experience effect of a theater level is achieved, the synchronous transmission technology of left and right sound channels is supported, and simultaneous interpretation in the video conference system can be realized. The voice quality is greatly improved by adopting the AAC-LD/Opus broadband voice technology, and the CD tone quality effect can be achieved. Supporting all-round speech processing, clear full duplex digital audio can be provided to users using fast Adaptive Echo Cancellation (AEC), automatic Gain Control (AGC), and Automatic Noise Suppression (ANS) techniques. By adopting the independent processing technology of the input audio channel, the patent technologies such as audioenhancement and voiceclean embrittlement make the sound full and rich. Forward error correction FEC (Forward Error Correction), backward error correction (Backward Error Correction), anti-packet PLC (Packet Loss Concealment), automatic transmission enhanced NetATE (Net Automatic-Transfer-Enhancement) and adaptive audio jitter buffer AJB (Audio JitterBuffer) are supported, ensuring an optimal hearing experience under network impairments.
The method selects a High-definition low-bandwidth technology, adopts an active video enhancement technology VME (Video Motion Enhancement) +H.264HP (High Profile), brings ultra-low-bandwidth High-definition experience to clients, and is characterized in the following aspects: the H.264HP coding algorithm is utilized, so that the 50% code rate can be greatly reduced under the condition that the subjective quality difference of the video is not great; by utilizing the VME technology, after decoding, a clearer video sensory effect which is more in line with the comfort level of human eyes can be obtained. The VME+H.264HP 'dual-core' processing technology effectively enhances the video quality end to end, and obtains clearer sensory effects which are more in line with the comfort level of human eyes. Meanwhile, at least 25% -50% of bandwidth is saved, and 1080P effect of the minimum 384K bandwidth (per screen) is achieved. Compared with the H264 BP technology, the VME+H.264HP dual-core technology can really save at least half of the bandwidth and effectively enhance the video quality. The method and the system have the advantages that a great guarantee of video communication is remarkably improved, partial upgrade or no upgrade of the network can be selected when an enterprise deploys the video system, the video sensory effect which is clearer and more suitable for human eye comfort can be obtained, and the communication efficiency is improved.
By adopting the network self-adaption technology, the system has original technology, has super-strong network adaptability, builds a stable high-definition system, and can ensure normal holding of a meeting in all directions.
The ultra-strong error correction SEC (Super Error Concealment) ultra-strong packet loss prevention processing technology can still ensure smooth video under the environment that the network packet loss rate reaches 20%; an IRC intelligent speed regulation technology is adopted to dynamically adjust the bandwidth of the video conference code stream, so that the optimal audio and video experience is ensured; the super-strong network jitter resistance can reach 1000ms at maximum, and the smooth proceeding of the conference is ensured. The unique QoS guarantee technology supports the ultra-strong error correction technology, and the system can automatically adjust different error correction strategies of network transmission and video and audio processing according to rich QOS modes of an IP network, so that the optimal effect of images and audios under various severe network conditions is ensured. The method has great improvement on indexes such as network jitter, delay, packet error and the like.
Specifying corresponding solutions for different situations includes:
packet loss retransmission technique: and under the conditions of small packet loss and sporadic packet loss, a packet loss retransmission technology is used.
Automatic speed reduction technology: for packet loss caused by network congestion, the automatic speed reduction technology can greatly reduce the packet loss rate on the network.
Packet loss resistant technology: can resist 20% packet loss.
An integrated intelligent speed regulation (IRC) technology is adopted in the system, the current video and audio packet loss condition is counted in real time in the video conference process, and when continuous packet loss exists in the network, the automatic start of intelligent speed reduction is supported; after the network packet loss is improved, the intelligent speed increase is supported by automatic start so as to realize the best audio and video experience. IRC (Intelligent Rate Control), automatically detecting the bandwidth change condition occupied by network service, intelligently selecting the optimal image resolution suitable for various network bandwidths, and keeping good conference quality.
And integrating the disconnection recovery (RoD), wherein the system adopts a disconnection recovery technology, the situation that the IP network is abnormally interrupted can be encountered in an actual conference, and the intelligent cooperative terminal of the system can immediately resume (RoD-Reconnect on Disconnect) the conference function as long as the network is recovered to be normal within 60 seconds, so that the conference can continue to be normally carried out.
Packet loss retransmission (ARQ), the system employs a packet loss retransmission technique (ARQ, automatic Repeat reQuest) which enables reliable information transmission on the basis of unreliable service by using both acknowledgement and timeout mechanisms. If the sender does not receive an acknowledgement frame within a period of time after the transmission, it will typically resend, ensuring a high smoothness of the videoconference.
Integrated SVC multi-stream technology SVC (Scalable Video Coding) is an extension of the h.264 standard, based on the h.264AVC (Advanced Video Coding) video codec standard, utilizing various efficient algorithmic tools of AVC codecs to produce decoded video at different frame rates, resolutions, or quality levels. The system has a multi-stream forwarding technology based on an H.264SVC coding technology, and on the basis of ensuring high-efficiency video compression performance, a video broadcasting end can generate video compression code streams with different frame rates and resolutions through one-time coding so as to adapt to application requirements of different network bandwidths, different display screens and terminal decoding capacities. The multi-stream forwarding only completes video forwarding at the network side, does not need video coding and decoding conversion, and completes video coding and decoding and multi-picture layout at the terminal side. The system not only supports the access of the AVC terminal, but also supports the access conference of the SVC conference terminal, which is called as a multi-stream mixed conference, and the multi-stream forwarding conference is a special case of the multi-stream mixed conference, and is hereinafter called as the multi-stream mixed conference. In a multi-stream hybrid conference, an SVC terminal may receive other conference site code streams of different resolutions.
In a specific embodiment, the real-time audio and video method of the invention combines the GIS system of the command center and the APP of the intelligent executive court in depth, and combines the court business data according to the actual case handling requirement of the court executive court, thereby realizing the reservation conference function, the instant wire function and the one-to-one group function.
The above functions are designed according to different service scenes, and the implementation process of calling the audio and video functions in the actual functions is as follows:
the first step: a video call is created. Participants may be created through software or web pages.
And a second step of: the participants are invited. After a call is created, the participants need to be invited to join the call.
And a third step of: the participant joins the call. The participant joins the call via a link or ID number.
Fourth step: and starting the video call. After all participants join, the video call may be opened.
Fifth step: and carrying out video call. The participants can conduct real-time audio and video communication, including voice, video, text chat and other functions.
Sixth step: ending the video call. After the video call is ended, the participant may exit.
For the reserved meeting function: the invention realizes the reservation conference function based on the cloud through the RTC technology. The court command center can reserve conferences through the function, invite other court command centers and execute court references. The reservation conference function provides input and management of information such as conference time, place, participants, etc. to ensure smooth holding of the conference.
For the instant meeting function: the invention realizes the instant conference function based on the cloud by the RTC technology. The court command center can initiate an instant conference through the function and invite other court command centers and perform court references. The instant conference function provides audio and video call, screen sharing, file transmission and other functions so as to facilitate real-time communication and collaboration among meeting participants.
For the instant wire function: the invention realizes the instant connection function based on the cloud by the RTC technology. The executive officer can connect the command center of the court to which the function belongs through one key to obtain necessary guidance and support. The instant connection function provides modes such as audio and video conversation, text chat and the like so as to realize real-time communication between the judge and the command center.
For a group of functions: the cloud-based cluster management system realizes cloud-based cluster management functions through the RTC technology. The executive judges can contact the principal online through the function, conduct about talking, making a writing and the like with the executive judges. A group of functions provide audio and video call, text chat, file transfer and other functions so that the judges and the parties can communicate and cooperate in real time.
The invention also provides a multilayer safe real-time audio-video device, as shown in fig. 7, which specifically comprises:
and an identity authentication mechanism establishment and deployment module: establishing an identity verification mechanism, and deploying the identity verification mechanism in a cloud server;
and the audio and video connection module is used for: establishing audio and video connection between a cloud server and a client based on the RTC;
and the permission generation and judgment module is used for: generating an access token of the client according to the identity authentication mechanism, and determining the authority of the client to join the video conference according to the information of the access token;
and the monitoring alarm module: and tracking the abnormal communication behavior of the video conference based on a real-time monitoring algorithm and giving an alarm in real time.
The device mainly comprises the four modules, and the aim of parallel operation can be realized by well constructing the system and simultaneously mounting the same file system.
In the implementation, each module may be implemented as an independent entity, or may be combined arbitrarily, and implemented as the same entity or several entities, and the implementation of each unit may be referred to the foregoing method embodiment, which is not described herein again.
Fig. 8 is a schematic structural diagram of a computer device according to the present disclosure. Referring to FIG. 8, the computer device 400 includes at least a memory 402 and a processor 401; the memory 402 is connected to the processor through the communication bus 403, and is configured to store computer instructions executable by the processor 401, where the processor 401 is configured to read the computer instructions from the memory 402 to implement the steps of the real-time audio/video method according to any one of the foregoing embodiments.
For the above-described device embodiments, reference is made to the description of the method embodiments for the relevant points, since they essentially correspond to the method embodiments. The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the objectives of the disclosed solution. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices including, for example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal magnetic disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.
Finally, it should be noted that: while this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features of specific embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. On the other hand, the various features described in the individual embodiments may also be implemented separately in the various embodiments or in any suitable subcombination. Furthermore, although features may be acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.
Similarly, although operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous. Moreover, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.
Thus, particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. Furthermore, the processes depicted in the accompanying drawings are not necessarily required to be in the particular order shown, or sequential order, to achieve desirable results. In some implementations, multitasking and parallel processing may be advantageous.
The foregoing description of the preferred embodiments of the present disclosure is not intended to limit the disclosure, but rather to cover all modifications, equivalents, improvements and alternatives falling within the spirit and principles of the present disclosure.
Claims (9)
1. The multilayer safe real-time audio and video method is characterized by comprising the following steps of:
s1, an identity verification mechanism is established, and the identity verification mechanism is deployed in a cloud server;
s2, establishing audio and video connection between a cloud server and a client based on the RTC;
s3, generating an access token of the client according to the identity authentication mechanism, and determining the authority of the client to join the video conference according to the information of the access token;
and S4, tracking communication abnormal behaviors of the video conference based on a real-time monitoring algorithm and giving an alarm in real time.
2. The real-time audio-visual method according to claim 1, wherein in the step S1, the authentication mechanism comprises:
acquiring an App ID and generating an App Key corresponding to the App ID;
generating unique identification signatures, expire Time and Nonce values based on a Signature generation algorithm by combining the App ID, the App ID and the User ID;
calling a Login interface of the client SDK, and carrying out identity information authentication according to the User ID, the identification signature, the Expire Time and the Nonce value.
3. The real-time audio-visual method according to claim 1, wherein in step S3, the access token comprises identity information and conference access rights.
4. The method according to claim 1, wherein in the step S2, the method for establishing an audio/video connection between the cloud server and the client based on the RTC includes:
the transmitting end packages the data into a data packet according to a certain rule and transmits the data packet to the receiving end;
the receiving end unpacks the data packet into original data according to the head information of the data packet;
and transmitting the original data according to a specific sequence and time interval.
5. The method according to claim 4, wherein the transmitting end and the receiving end establish a connection according to a handshake mechanism.
6. The method according to claim 4, wherein the sequence and low-latency characteristics of the original data are maintained based on a TCP protocol and a UDP protocol during the data transmission.
7. A multi-layer secure real-time audio-visual apparatus employing the method of any one of claims 1-6, comprising:
and an identity authentication mechanism establishment and deployment module: establishing an identity verification mechanism, and deploying the identity verification mechanism in a cloud server;
and the audio and video connection module is used for: establishing audio and video connection between a cloud server and a client based on the RTC;
and the permission generation and judgment module is used for: generating an access token of the client according to the identity authentication mechanism, and determining the authority of the client to join the video conference according to the information of the access token;
and the monitoring alarm module: and tracking the abnormal communication behavior of the video conference based on a real-time monitoring algorithm and giving an alarm in real time.
8. A computer readable storage medium having stored thereon a computer program, characterized in that the program when executed performs the steps of the real-time audiovisual method of any one of claims 1-6.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the real-time audiovisual method according to any one of claims 1-6 when the program is executed by the processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311648152.8A CN117650932A (en) | 2023-12-04 | 2023-12-04 | Multilayer safe real-time audio and video method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311648152.8A CN117650932A (en) | 2023-12-04 | 2023-12-04 | Multilayer safe real-time audio and video method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117650932A true CN117650932A (en) | 2024-03-05 |
Family
ID=90046070
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311648152.8A Pending CN117650932A (en) | 2023-12-04 | 2023-12-04 | Multilayer safe real-time audio and video method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117650932A (en) |
-
2023
- 2023-12-04 CN CN202311648152.8A patent/CN117650932A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7773581B2 (en) | Method and apparatus for conferencing with bandwidth control | |
| US20190073993A1 (en) | Artificially generated speech for a communication session | |
| US20050237931A1 (en) | Method and apparatus for conferencing with stream selectivity | |
| US6466248B1 (en) | Videoconference recording | |
| US8456510B2 (en) | Virtual distributed multipoint control unit | |
| AU2011258272B2 (en) | Systems and methods for scalable video communication using multiple cameras and multiple monitors | |
| EP2176987B1 (en) | Multi-point to multi-point intercom system | |
| CA2591732C (en) | Intelligent audio limit method, system and node | |
| KR100906687B1 (en) | Multi-point conference system and multi-point conference device | |
| EP1868348B1 (en) | Conference layout control and control protocol | |
| US20070294263A1 (en) | Associating independent multimedia sources into a conference call | |
| US20120086769A1 (en) | Conference layout control and control protocol | |
| EP1868347A2 (en) | Associating independent multimedia sources into a conference call | |
| US20070291667A1 (en) | Intelligent audio limit method, system and node | |
| US6646674B1 (en) | TV telephone system | |
| WO2007035109A1 (en) | Method for gatekeeper streaming | |
| JP2012151555A (en) | Television conference system, television conference relay device, television conference relay method and relay program | |
| CN117650932A (en) | Multilayer safe real-time audio and video method and device | |
| CN112653868A (en) | Cloud-based multi-person remote scene secure video conference communication system | |
| Johanson | Multimedia communication, collaboration and conferencing using Alkit Confero | |
| CN112689118B (en) | Data transmission method and device for multi-screen network terminal | |
| Sorokin et al. | IP Video Conferencing: A Tutorial | |
| CN106331567A (en) | Video communication method and device | |
| da Silva | Video-conference system based on open source software | |
| MX2007006914A (en) | Intelligent audio limit method, system and node. |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |