CN116582699A - Audio and video processing method, device, equipment and medium - Google Patents

Audio and video processing method, device, equipment and medium Download PDF

Info

Publication number
CN116582699A
CN116582699A CN202310642722.6A CN202310642722A CN116582699A CN 116582699 A CN116582699 A CN 116582699A CN 202310642722 A CN202310642722 A CN 202310642722A CN 116582699 A CN116582699 A CN 116582699A
Authority
CN
China
Prior art keywords
audio
video stream
video
output
interface
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202310642722.6A
Other languages
Chinese (zh)
Other versions
CN116582699B (en
Inventor
孙忠恒
孙佩
黄敏成
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Rongsheng Information Technology Co ltd
Original Assignee
Guangzhou Rongsheng Information Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Rongsheng Information Technology Co ltd filed Critical Guangzhou Rongsheng Information Technology Co ltd
Priority to CN202310642722.6A priority Critical patent/CN116582699B/en
Publication of CN116582699A publication Critical patent/CN116582699A/en
Application granted granted Critical
Publication of CN116582699B publication Critical patent/CN116582699B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/233—Processing of audio elementary streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/439—Processing of audio elementary streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

The invention discloses an audio and video processing method, an audio and video processing device, audio and video processing equipment and a medium. The system board receives a plurality of network audio and video streams through a plurality of first input interfaces and distributes the network audio and video streams to each media board respectively; each media board receives a local audio/video stream through a second input interface of the media board; and obtaining the synthesis configuration information of each output interface of the device; and aiming at each output interface and each input interface identifier, selecting an audio-video stream corresponding to the input interface identifier from a plurality of received network audio-video streams and local audio-video streams, synthesizing the audio-video streams to be output according to a synthesis rule, generating a synthesized video stream and outputting the synthesized video stream to external equipment. The problem that the interface processing cannot be accurately carried out on the multi-path audio and video streams is solved, the processing efficiency and accuracy of different audio and video streams are improved, so that a worker can carry out more accurate line switching, the data isolation of the multi-path audio and video streams is realized, and the situation that the multi-path audio and video streams cannot overflow to a local network is guaranteed.

Description

Audio and video processing method, device, equipment and medium
Technical Field
The invention relates to the technical field of data processing, in particular to an audio and video processing method, an audio and video processing device, audio and video processing equipment and a medium.
Background
In daily life, a large number of audio and video streams are required to be acquired at any moment, different audio and video streams correspond to different interface formats, and various audio and video stream acquisition ways are available, so that the audio and video streams can be acquired locally or the audio and video streams can be acquired online.
The inventors have found that the following drawbacks exist in the prior art in the process of implementing the present invention: at present, multiple paths of audio and video streams are collected, but no method is available for determining the type of the audio and video streams, so that the identification processing of an audio and video stream interface is difficult, the data processing is difficult, and the identification burden of a staff interface is increased.
Disclosure of Invention
The invention provides an audio and video processing method, an audio and video processing device, audio and video processing equipment and media, so as to improve the efficiency and the accuracy of processing different audio and video streams.
According to an aspect of the present invention, there is provided an audio/video processing method, where the method is applied to an audio/video processing device, where the audio/video processing device includes a system board and a plurality of media boards, and the system board includes a plurality of first input interfaces, where the first input interfaces are used to receive a network audio/video stream; each media board comprises at least one second input interface and a plurality of output interfaces, wherein the second input interfaces are used for receiving local audio and video streams, and the output interfaces are used for outputting the audio and video streams to connected external equipment;
The method comprises the following steps: the system board receives a plurality of network audio and video streams through a plurality of first input interfaces and distributes the network audio and video streams to the media boards respectively;
receiving local audio and video streams in each media board through a second input interface of the media board;
the media board acquires the synthesis configuration information of each own output interface, wherein the synthesis configuration information comprises at least one input interface identifier and a corresponding synthesis rule;
the media board selects an audio-video stream corresponding to the input interface identifier from a plurality of received network audio-video streams and local audio-video streams as an audio-video stream to be output according to at least one input interface identifier in the composite configuration information of the media board aiming at each output interface;
the media board synthesizes the audio and video streams to be output according to the synthesis rule to generate a synthesized video stream;
and the media board outputs the synthesized video stream to corresponding external equipment through a current output interface.
According to another aspect of the present invention, there is provided an audio/video processing apparatus, wherein the apparatus is applied to an audio/video processing device, the audio/video processing device including a system board and a plurality of media boards, the system board including a plurality of first input interfaces, the first input interfaces being configured to receive a network audio/video stream; each media board comprises at least one second input interface and a plurality of output interfaces, wherein the second input interfaces are used for receiving local audio and video streams, and the output interfaces are used for outputting the audio and video streams to connected external equipment;
The device comprises:
the network audio/video stream distribution module is used for receiving a plurality of network audio/video streams through a plurality of first input interfaces by the system board and respectively distributing each network audio/video stream to each media board;
the local audio/video stream receiving module is used for receiving local audio/video streams in each media board through a second input interface of the local audio/video stream receiving module;
the synthesis configuration information acquisition module is used for acquiring synthesis configuration information of each output interface of the media board through the media board, wherein the synthesis configuration information comprises at least one input interface identifier and a corresponding synthesis rule;
the audio/video stream to be output determining module is used for selecting an audio/video stream corresponding to the input interface identifier from a plurality of received network audio/video streams and a local audio/video stream as an audio/video stream to be output according to at least one input interface identifier in the synthesis configuration information of the audio/video stream to be output by the media board for each output interface;
the synthesized video stream generating module is used for synthesizing the audio and video streams to be output through the media board according to the synthesis rule to generate a synthesized video stream;
and the synthesized video stream output module is used for outputting the synthesized video stream to corresponding external equipment through the current output interface through the media board.
According to another aspect of the present invention, there is provided an audio/video processing apparatus, including a system board and a plurality of media boards, the system board including a plurality of first input interfaces for receiving network audio/video streams; each media board comprises at least one second input interface and a plurality of output interfaces, wherein the second input interfaces are used for receiving local audio and video streams, and the output interfaces are used for outputting the audio and video streams to connected external equipment;
the system board is used for:
receiving a plurality of network audio and video streams through a plurality of first input interfaces, and respectively distributing each network audio and video stream to each media board;
the media board is used for:
receiving a local audio/video stream through a second input interface of the local audio/video stream;
acquiring synthesis configuration information of each output interface of the device, wherein the synthesis configuration information comprises at least one input interface identifier and a corresponding synthesis rule;
for each output interface, selecting an audio-video stream corresponding to the input interface identifier from a plurality of received network audio-video streams and local audio-video streams according to at least one input interface identifier in the synthesized configuration information of the output interface as an audio-video stream to be output;
Synthesizing the audio and video streams to be output according to the synthesis rule to generate a synthesized video stream;
and outputting the synthesized video stream to corresponding external equipment through a current output interface.
According to another aspect of the present invention, there is provided a computer readable storage medium storing computer instructions for causing a processor to execute an audio/video processing method according to any one of the embodiments of the present invention.
According to the technical scheme, the system board receives a plurality of network audio and video streams through a plurality of first input interfaces and distributes the network audio and video streams to each media board respectively; each media board receives a local audio/video stream through a second input interface of the media board; and obtaining the synthesis configuration information of each output interface of the device; and aiming at each output interface and each input interface identifier, selecting an audio-video stream corresponding to the input interface identifier from a plurality of received network audio-video streams and local audio-video streams, synthesizing the audio-video streams to be output according to a synthesis rule, generating a synthesized video stream and outputting the synthesized video stream to external equipment. The problem that the interface processing cannot be accurately carried out on the multi-path audio and video streams is solved, the processing efficiency and accuracy of different audio and video streams are improved, so that a worker can carry out more accurate line switching, the data isolation of the multi-path audio and video streams is realized, and the situation that the multi-path audio and video streams cannot overflow to a local network is guaranteed.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention or to delineate the scope of the invention. Other features of the present invention will become apparent from the description that follows.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1a is a flowchart of an audio/video processing method according to a first embodiment of the present invention;
fig. 1b is a schematic structural diagram of a specific connection relationship of an audio/video processing device in a method according to a first embodiment of the present invention;
fig. 1c is a schematic structural diagram of a specific application scenario of an audio/video processing device in a method according to a first embodiment of the present invention;
fig. 2 is a schematic structural diagram of an audio/video processing device according to a second embodiment of the present invention;
fig. 3 is a schematic structural diagram of an audio/video processing device according to a third embodiment of the present invention.
Detailed Description
In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
It should be noted that the terms "target," "current," and the like in the description and claims of the present invention and the above-described drawings are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example 1
Fig. 1a is a flowchart of an audio/video processing method according to an embodiment of the present invention, where the embodiment is applicable to a case of performing interface recognition on multiple video streams. The method may be performed by an audio-visual processing device, which may be implemented in software and/or hardware.
The method is applied to audio and video processing equipment, wherein the audio and video processing equipment comprises a system board and a plurality of media boards, the system board comprises a plurality of first input interfaces, and the first input interfaces are used for receiving network audio and video streams; each media board comprises at least one second input interface and a plurality of output interfaces, wherein the second input interfaces are used for receiving local audio and video streams, and the output interfaces are used for outputting the audio and video streams to connected external equipment.
Accordingly, as shown in fig. 1a, the method comprises:
s110, the system board receives a plurality of network audio and video streams through a plurality of first input interfaces, and distributes the network audio and video streams to the media boards respectively.
The first input interface may be an interface for receiving a network audio/video stream, where the first input interface corresponds to a network port on the system board. The system board can store multiple network audio and video streams and output the network audio and video streams to multiple media boards.
In this embodiment, the audio and video processing device may be deployed in a back-end command center.
Exemplary, as shown in fig. 1b, a schematic structural diagram of a specific connection relationship of an audio/video processing device is shown. In fig. 1b, a system board (31) and an internal switching unit connected to the system board are included, as well as 5 media boards. The system board includes a first input interface, that is, an interface corresponding to the Lan2 port in fig. 1b, and receives the network audio and video stream through the Lan2 port.
In addition, each media board includes at least one second input interface, i.e. Hdmi-in (High Definition Multimedia Interface, high-definition multimedia interface) interface, and a plurality of output interfaces, i.e. Hdmi-out interface. Specifically, the Hdmi-in interface is configured to receive a local audio/video stream, and the Hdmi-out interface is configured to output the audio/video stream to a connected external device, that is, a plurality of display screens in fig. 1 b.
Specifically, the first input interface may input 20 paths of network audio/video streams; the second input interface may input 5 local audio and video streams.
Specifically, the input device corresponding to the input interface may include: high definition cameras, PCs (Personal Computer, personal computers), IPCs (IP cameras), DVD (Digital Videodisc ) players, digital television set-top boxes, and the like. The external device may include: plasma televisions, liquid crystal televisions, television walls, monitors, and the like.
Optionally, the system board communicates with each media board through an internal switch; the system board also comprises a first network port and a second network port; the system board receives a plurality of network audio and video streams through a plurality of first input interfaces and distributes the network audio and video streams to the media boards respectively, and the system board comprises: after the system board is accessed into different network audio and video sources through a plurality of first input interfaces, network audio and video streams input by the different network audio and video sources are received through the first network interfaces; the system board transmits the network audio and video stream to a second network port through the first network port, and distributes the network audio and video stream to each media board through an internal switch through the second network port.
The first network port may be a network port for receiving a network audio/video stream, and the second network port may be a network port for outputting the network audio/video stream.
In the previous example, as shown in fig. 1b, the network audio/video stream is received through the Lan2 port of the system board, the received network audio/video stream is output from the Lan1 port of the system board, and the network audio/video stream is distributed to each media board through the Lan1 port through the internal switch (i.e. the internal switching unit in fig. 1 b).
The advantages of this arrangement are that: the data isolation of the multipath audio and video streams can be realized, so that the situation that the multipath audio and video streams cannot overflow to a local network is ensured, the conflict of the data is reduced, and the safety of the data is improved.
S120, receiving the local audio and video stream through the second input interface of each media board.
The second input interface may be an interface for receiving a local audio/video stream.
In the previous examples, each media board comprises at least one second input interface, in fig. 1b, the second input interface is also referred to as the Hdmi-in interface. Specifically, the Hdmi-in interface is used for receiving the local audio/video stream.
Optionally, after each media board receives the local audio/video stream through its own second input interface, the method further includes: and sending the local audio and video stream to other media boards through the internal switch.
In this embodiment, it is assumed that after the local audio/video streams are received through the second input interfaces corresponding to the media boards, the exchange processing operation between the local audio/video streams between the media boards may also be performed through the internal switch. Therefore, the flexibility of processing the local audio and video streams can be improved, and the interactive processing of the local audio and video streams among a plurality of media boards is realized.
S130, the media board acquires the synthesis configuration information of each output interface of the media board, wherein the synthesis configuration information comprises at least one input interface identifier and a corresponding synthesis rule.
Wherein the composition configuration information may be information describing the interface parameters and composition rules.
Optionally, the audio and video processing device further includes a display screen; the method further comprises the steps of: displaying a configuration page through the display screen, wherein the configuration page comprises an output position configuration item, a split screen mode configuration item and an input interface configuration item, and each output position in the output position configuration item is provided with a corresponding output interface identifier; receiving configuration information input by a user from the output position configuration item, the split screen mode configuration item and the input interface configuration item respectively; after a user selects a split screen mode from the split screen mode configuration items, entering a split screen configuration page so as to facilitate the user to set the input interface identification of each split screen in the split screen configuration page; generating a synthesis rule according to the configuration information of the split screen mode configuration item, and generating synthesis configuration information according to the synthesis rule, the configuration information of the input interface configuration item and the output interface identifier corresponding to the output position configuration item.
In this embodiment, the audio-video processing apparatus further includes a display screen. In the display screen, a configuration page may be displayed.
Specifically, the configuration page includes an output position configuration item, a split screen mode configuration item, and an input interface configuration item. Wherein each output position in the output position configuration item has a corresponding output interface identifier; and receiving configuration information input by a user from the output position configuration item, the split screen mode configuration item and the input interface configuration item respectively. For example, if 8 split screens are selected, a jump may be made to an 8 split screen configuration page, and clicking on each window in the 8 split screen configuration page may configure the input interface of the corresponding window.
In addition, the split screen modes may include 1 split screen, 2 split screen, 4 split screen, 8 split screen modes.
Accordingly, when the user selects the split mode from the split mode configuration items, it is assumed that the split mode of 8 split is selected. Then, a split screen configuration page is entered so that a user sets an input interface identifier of each split screen in the split screen configuration page, that is, the first split screen is used for receiving an output audio/video stream of a corresponding certain media board, and the second split screen is used for receiving an audio/video stream of the media board 1 on the assumption that the first split screen is used for receiving an audio/video stream of the media board 1. Further, a synthesis rule is generated according to the configuration information of the split screen mode configuration item, and synthesis configuration information is generated according to the synthesis rule, the configuration information of the input interface configuration item and the output interface identifier corresponding to the output position configuration item.
And S140, selecting an audio-video stream corresponding to the input interface identifier from the received multiple network audio-video streams and the local audio-video stream as an audio-video stream to be output according to at least one input interface identifier in the synthesized configuration information of the output interfaces by the media board aiming at each output interface.
In this embodiment, the corresponding output processing of the audio/video streams respectively matched with the plurality of network audio/video streams and the local audio/video stream is implemented according to the input interface identifier in the synthesis configuration information.
For example, assuming that the selected split screen mode is 4 split screens, and the corresponding input interface identifiers are the network interfaces 1-3 and the local interfaces 2 respectively, the audio/video stream to be output is the audio/video stream corresponding to the network interfaces 1-3 and the local interfaces 2.
And S150, the media board synthesizes the audio and video streams to be output according to the synthesis rule to generate a synthesized video stream.
And S160, the media board outputs the synthesized video stream to corresponding external equipment through a current output interface.
In this embodiment, the network audio and video stream and the local audio and video stream may be received in a mixed manner, and then the network audio and video stream and the local audio and video stream are merged and output.
Specifically, the network audio and video stream is received through the system board, and the local audio and video stream is received through the media board. Each media board can provide a local output port and a network output port, wherein the local output port is used for outputting the confluence to local equipment; the network output port is used for compressing the confluence codes and outputting the compressed confluence codes to the network equipment through the system board.
Exemplary, as shown in fig. 1c, a schematic structural diagram of a specific application scenario of the audio/video processing device is shown. Out1 port of the media board 1 corresponds to conference room 1, and out2 port corresponds to conference room 2; the out3 port of the media board 2 corresponds to the hall, the out4 port corresponds to the conference room 3, and so on. Out1 can display 1-14 paths of streams, out2 ports can display 15-17 paths of streams, and audio and video streams displayed by each Out port are configurable. It will be appreciated that the media boards 1 and 2 are located in an audio video processing device, comprising N conference rooms.
Optionally, the output interface includes a local output interface and a network output interface; the media board outputs the composite video stream to corresponding external equipment through a current output interface, and the method comprises the following steps: if the current output interface is a local output interface, outputting the synthesized video stream to corresponding display equipment for display through the local output interface; and if the current output interface is a network output interface, the synthesized video stream is encoded and then sent to the system board through the network output interface, and the system board uploads the synthesized video stream to a corresponding server.
In this embodiment, the output interface of each media board may include a local output interface and a network output interface. Firstly judging whether the current output interface is a local output interface or a network output interface, and outputting the synthesized video stream to corresponding display equipment for display through the local output interface if the current output interface is the local output interface.
Further, if the current output interface is a network output interface, the synthesized video stream is encoded and then sent to the system board through the network output interface, and the system board uploads the synthesized video stream to the corresponding server.
The advantage of this arrangement is that when the output of the composite video stream is performed from different types of output interfaces, different encoding operations need to be performed on the composite audio/video stream and output to the server or the display screen.
Optionally, the audio/video processing device is connected to the control terminal, and the method further includes: receiving a picture switching request sent by the control terminal through the system board, wherein the picture switching request comprises a target output interface identifier and updated synthesis configuration information corresponding to the target output interface identifier; determining a media board corresponding to the target output interface identifier as a target media board; and sending the updated synthesis configuration information to the target media board, so that the target media board updates the audio/video stream to be output in real time according to the updated synthesis configuration information.
In this embodiment, the system board may also receive a screen switching request sent by the control terminal. Specifically, the frame switching request may further include a target output interface identifier, and updated composite configuration information corresponding to the target output interface identifier.
Further, the target media boards can be identified through the target output interface identifiers, specifically, the target output interface identifiers are used for representing the interface identifiers of each media board, and the output interface identifiers of each media board are unique.
Correspondingly, after the target media board is determined, the corresponding updated composite configuration information is required to be sent to the target media board so as to complete the real-time updating processing of the audio/video stream to be output.
Optionally, the interface types of the second input interfaces of the media boards are the same or different, and the interface types include at least one of HDMI interface, VGA (Video Graphics Array ) interface, SDI (Serial Digital Interface, digital component serial interface) interface; the communication protocol supported by each of the first input interfaces is the same or different.
In particular, the communication protocol may include RTSP (Real-time Streaming Protocol ), ONVIF, and GB28181, among others.
In this embodiment, the second input interface type of each media board may be the same or different; and the communication protocols supported by the first input interface may be the same or different, so that the processing of audio and video streams with multiple interface types can be realized. And this is a design adopted for more adapting to different interfaces, such as HDMI interface input currently, and when the input needs to be changed to VGA or SDI interface input, only the media card needs to be changed.
In addition, the external network camera is connected with the system board through the exchanger, and the system board pulls the audio and video of the external network camera to flow in through protocols such as RTSP, GB28181, ONVIF and the like; then the received audio and video stream is multicast and sent to an internal exchanger according to each port; and each media board receives the audio and video stream at the respective network port, decodes, outputs the audio and video stream to the HDMI for display, can encode the audio and video data, broadcasts and transmits the compressed audio and video data to the system board, and the system board receives the audio and video stream and transmits the audio and video stream to the outside through RTSPSERVER service.
Further, the local audio and video stream is connected with the media board through an HDMI cable; after the media board collects the HDMI audio and video stream, the media board performs screen combination and outputs the HDMI audio and video stream to an HDMI display screen; simultaneously, the media board compresses the audio and video stream and then multicasts the compressed audio and video stream to the internal exchanger through each port; and then other media boards can also receive the audio and video stream, then decode and output the audio and video stream to the HDMI for display, and meanwhile, the system board receives the audio and video stream and then sends the audio and video stream to the outside through RTSPSERVER service.
Optionally, acquiring multiple paths of video streams in real time through a system unit module; sequentially carrying out identification processing on each video stream through a system unit module, and determining video stream categories corresponding to each video stream respectively; wherein the video stream categories include: a local video stream category or a network video stream category; determining a video stream interface corresponding to each video stream according to the video stream category through a system unit module, and outputting the video stream from the matched video stream interface to a media unit module; and carrying out interface identification processing on the received video stream through the media unit module to obtain an interface identification result corresponding to the received video stream.
Optionally, the determining, by the system unit module, a video stream interface corresponding to each video stream according to the video stream category, and outputting the video stream from the matched video stream interface to the media unit module, includes: if the video stream category is a local video stream category, determining that a video stream interface corresponding to the local video stream category is a local video stream interface, and outputting the video stream from the local video stream interface to a media unit module; if the video stream category is a network video stream category, determining that a video stream interface corresponding to the network video stream category is a network video stream interface, and outputting the video stream from the network video stream interface to a media unit module.
Optionally, the performing, by the media unit module, interface identification processing on the received video stream to obtain an interface identification result corresponding to the received video stream, where the interface identification result includes: performing interface identification processing on the received local video stream through a media unit module to obtain a first interface identification result corresponding to the received local video stream; performing interface identification processing on the received network video stream through a media unit module to obtain a second interface identification result corresponding to the received network video stream; and determining the interface identification result according to the first interface identification result and the second interface identification result.
Optionally, after the interface identification process is performed on the received video stream by the media unit module to obtain an interface identification result corresponding to the received video stream, the method further includes: and feeding back the interface identification result to a user through an interface identification result output unit module so as to realize the identification of the video stream interface.
Optionally, the identifying, by the system unit module, each video stream sequentially, and determining a video stream category corresponding to each video stream respectively includes: acquiring each video storage path corresponding to each video stream through a system unit module; and sequentially acquiring a target video storage path through the system unit module, identifying the target video storage path, and determining the video stream type corresponding to the target video stream.
Optionally, the obtaining, by the system unit module, each video storage path corresponding to each video stream includes: performing standard format standard conversion on each video stream through the system unit module to determine a standard format video stream; and acquiring each video storage path corresponding to each standard format video stream through the system unit module.
Optionally, after sequentially acquiring a target video storage path through the system unit module and identifying the target video storage path, determining a video stream category corresponding to the target video stream, the method further includes: judging whether the associated video stream associated with the target video stream exists in each video stream through the system unit module, and if so, determining the video stream category corresponding to the target video stream as the video stream category of the associated video stream; if the video stream category does not exist, returning to execute the operation of sequentially acquiring a target video storage path through the system unit module, identifying the target video storage path, and determining the video stream category corresponding to the target video stream until each video storage path is traversed.
According to the technical scheme, the system board receives a plurality of network audio and video streams through a plurality of first input interfaces and distributes the network audio and video streams to each media board respectively; each media board receives a local audio/video stream through a second input interface of the media board; and obtaining the synthesis configuration information of each output interface of the device; and aiming at each output interface and each input interface identifier, selecting an audio-video stream corresponding to the input interface identifier from a plurality of received network audio-video streams and local audio-video streams, synthesizing the audio-video streams to be output according to a synthesis rule, generating a synthesized video stream and outputting the synthesized video stream to external equipment. The problem that the interface processing cannot be accurately carried out on the multi-path audio and video streams is solved, the processing efficiency and accuracy of different audio and video streams are improved, so that a worker can carry out more accurate line switching, the data isolation of the multi-path audio and video streams is realized, and the situation that the multi-path audio and video streams cannot overflow to a local network is guaranteed.
Example two
Fig. 2 is a schematic structural diagram of an audio/video processing apparatus according to a second embodiment of the present invention, where the audio/video processing apparatus according to the present embodiment may be implemented by software and/or hardware, and may be configured in a terminal device or a server to implement an audio/video processing method according to the embodiment of the present invention. As shown in fig. 2, the apparatus includes: the system comprises a network audio-video stream distribution module 210, a local audio-video stream receiving module 220, a synthesis configuration information acquisition module 230, an audio-video stream to be output determination module 240, a synthesis video stream generation module 250 and a synthesis video stream output module 260.
The device is applied to audio and video processing equipment, wherein the audio and video processing equipment comprises a system board and a plurality of media boards, the system board comprises a plurality of first input interfaces, and the first input interfaces are used for receiving network audio and video streams; each media board comprises at least one second input interface and a plurality of output interfaces, wherein the second input interfaces are used for receiving local audio and video streams, and the output interfaces are used for outputting the audio and video streams to connected external equipment;
the device comprises:
the network audio/video stream distribution module 210 is configured to receive a plurality of network audio/video streams through a plurality of first input interfaces by the system board, and distribute each network audio/video stream to each media board respectively;
a local audio/video stream receiving module 220, configured to receive, in each media board, a local audio/video stream through its own second input interface;
a composite configuration information obtaining module 230, configured to obtain, through the media board, composite configuration information of each output interface of the media board, where the composite configuration information includes at least one input interface identifier and a corresponding composite rule;
the audio/video stream to be output determining module 240 is configured to select, for each output interface through the media board, according to at least one input interface identifier in the composite configuration information of the output interface, an audio/video stream corresponding to the input interface identifier from the received multiple network audio/video streams and the local audio/video stream, as an audio/video stream to be output;
The composite video stream generating module 250 is configured to synthesize the audio and video streams to be output according to the synthesis rule through the media board, so as to generate a composite video stream;
and the composite video stream output module 260 is configured to output, through the media board, the composite video stream to a corresponding external device through a current output interface.
According to the technical scheme, the system board receives a plurality of network audio and video streams through a plurality of first input interfaces and distributes the network audio and video streams to each media board respectively; each media board receives a local audio/video stream through a second input interface of the media board; and obtaining the synthesis configuration information of each output interface of the device; and aiming at each output interface and each input interface identifier, selecting an audio-video stream corresponding to the input interface identifier from a plurality of received network audio-video streams and local audio-video streams, synthesizing the audio-video streams to be output according to a synthesis rule, generating a synthesized video stream and outputting the synthesized video stream to external equipment. The problem that the interface processing cannot be accurately carried out on the multi-path audio and video streams is solved, the processing efficiency and accuracy of different audio and video streams are improved, so that a worker can carry out more accurate line switching, the data isolation of the multi-path audio and video streams is realized, and the situation that the multi-path audio and video streams cannot overflow to a local network is guaranteed.
Optionally, the system board communicates with each media board through an internal switch; the system board also comprises a first network port and a second network port;
the network audio/video stream distribution module 210 may be specifically configured to: after the system board is accessed into different network audio and video sources through a plurality of first input interfaces, network audio and video streams input by the different network audio and video sources are received through the first network interfaces; the system board transmits the network audio and video stream to a second network port through the first network port, and distributes the network audio and video stream to each media board through an internal switch through the second network port.
Optionally, the local audio/video stream receiving module 220 may be further specifically configured to: after each media board receives the local audio/video stream through the second input interface, the local audio/video stream is sent to other media boards through the internal switch.
Optionally, the output interface includes a local output interface and a network output interface; the composite video stream output module 260 may be specifically configured to: if the current output interface is a local output interface, outputting the synthesized video stream to corresponding display equipment for display through the local output interface; and if the current output interface is a network output interface, the synthesized video stream is encoded and then sent to the system board through the network output interface, and the system board uploads the synthesized video stream to a corresponding server.
Optionally, the audio/video processing device is connected to a control terminal, and the method may further specifically be used for: receiving a picture switching request sent by the control terminal through the system board, wherein the picture switching request comprises a target output interface identifier and updated synthesis configuration information corresponding to the target output interface identifier; determining a media board corresponding to the target output interface identifier as a target media board; and sending the updated synthesis configuration information to the target media board, so that the target media board updates the audio/video stream to be output in real time according to the updated synthesis configuration information.
Optionally, the audio and video processing device further includes a display screen; the method may also be specifically used for: displaying a configuration page through the display screen, wherein the configuration page comprises an output position configuration item, a split screen mode configuration item and an input interface configuration item, and each output position in the output position configuration item is provided with a corresponding output interface identifier; receiving configuration information input by a user from the output position configuration item, the split screen mode configuration item and the input interface configuration item respectively; after a user selects a split screen mode from the split screen mode configuration items, entering a split screen configuration page so as to facilitate the user to set the input interface identification of each split screen in the split screen configuration page; generating a synthesis rule according to the configuration information of the split screen mode configuration item, and generating synthesis configuration information according to the synthesis rule, the configuration information of the input interface configuration item and the output interface identifier corresponding to the output position configuration item.
Optionally, the interface types of the second input interfaces of the media boards are the same or different, and the interface types include at least one of an HDMI interface, a VGA interface and an SDI interface; the communication protocol supported by each of the first input interfaces is the same or different.
The audio and video processing device provided by the embodiment of the invention can execute the audio and video processing method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
Example III
Fig. 3 is a schematic structural diagram of an audio/video processing device according to a third embodiment of the present invention, where the audio/video processing device 310 includes a system board 320 and a plurality of media boards 330, and the system board 320 includes a plurality of first input interfaces 321, where the first input interfaces 321 are configured to receive network audio/video streams; each media board 330 includes at least one second input interface 331 and a plurality of output interfaces 332, where the second input interface 331 is configured to receive a local audio/video stream, and the output interfaces are configured to output the audio/video stream to a connected external device.
Wherein the system board 320 is configured to:
receiving a plurality of network audio and video streams through a plurality of first input interfaces 321, and respectively distributing each network audio and video stream to each media board 330;
The media board 330 is configured to:
receiving a local audio-video stream through its own second input interface 331;
acquiring synthesis configuration information of each output interface of the device, wherein the synthesis configuration information comprises at least one input interface identifier and a corresponding synthesis rule;
for each output interface, selecting an audio-video stream corresponding to the input interface identifier from a plurality of received network audio-video streams and local audio-video streams according to at least one input interface identifier in the synthesized configuration information of the output interface as an audio-video stream to be output;
synthesizing the audio and video streams to be output according to the synthesis rule to generate a synthesized video stream;
and outputting the synthesized video stream to corresponding external equipment through a current output interface.
Example IV
A fourth embodiment of the present invention also provides a computer-readable storage medium containing computer-readable instructions, which when executed by a computer processor, are configured to perform an audio-video processing method, the method comprising: the system board receives a plurality of network audio and video streams through a plurality of first input interfaces and distributes the network audio and video streams to the media boards respectively; receiving local audio and video streams in each media board through a second input interface of the media board; the media board acquires the synthesis configuration information of each own output interface, wherein the synthesis configuration information comprises at least one input interface identifier and a corresponding synthesis rule; the media board selects an audio-video stream corresponding to the input interface identifier from a plurality of received network audio-video streams and local audio-video streams as an audio-video stream to be output according to at least one input interface identifier in the composite configuration information of the media board aiming at each output interface; the media board synthesizes the audio and video streams to be output according to the synthesis rule to generate a synthesized video stream; and the media board outputs the synthesized video stream to corresponding external equipment through a current output interface.
Of course, the computer-readable storage medium according to the embodiments of the present invention may include computer-executable instructions not only for performing the method operations described above, but also for performing the related operations in the audio/video processing method according to any of the embodiments of the present invention.
From the above description of embodiments, it will be clear to a person skilled in the art that the present invention may be implemented by means of software and necessary general purpose hardware, but of course also by means of hardware, although in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, etc., and include several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments of the present invention.
It should be noted that, in the embodiment of the audio/video processing apparatus, each unit and module included are only divided according to the functional logic, but not limited to the above-mentioned division, so long as the corresponding functions can be implemented; in addition, the specific names of the functional units are also only for distinguishing from each other, and are not used to limit the protection scope of the present invention.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.

Claims (10)

1. The audio and video processing method is characterized by being applied to audio and video processing equipment, wherein the audio and video processing equipment comprises a system board and a plurality of media boards, the system board comprises a plurality of first input interfaces, and the first input interfaces are used for receiving network audio and video streams; each media board comprises at least one second input interface and a plurality of output interfaces, wherein the second input interfaces are used for receiving local audio and video streams, and the output interfaces are used for outputting the audio and video streams to connected external equipment; the method comprises the following steps:
the system board receives a plurality of network audio and video streams through a plurality of first input interfaces and distributes the network audio and video streams to the media boards respectively;
receiving local audio and video streams in each media board through a second input interface of the media board;
The media board acquires the synthesis configuration information of each own output interface, wherein the synthesis configuration information comprises at least one input interface identifier and a corresponding synthesis rule;
the media board selects an audio-video stream corresponding to the input interface identifier from a plurality of received network audio-video streams and local audio-video streams as an audio-video stream to be output according to at least one input interface identifier in the composite configuration information of the media board aiming at each output interface;
the media board synthesizes the audio and video streams to be output according to the synthesis rule to generate a synthesized video stream;
and the media board outputs the synthesized video stream to corresponding external equipment through a current output interface.
2. The method of claim 1, wherein the system board communicates with each media board through an internal switch; the system board also comprises a first network port and a second network port;
the system board receives a plurality of network audio and video streams through a plurality of first input interfaces and distributes the network audio and video streams to the media boards respectively, and the system board comprises:
after the system board is accessed into different network audio and video sources through a plurality of first input interfaces, network audio and video streams input by the different network audio and video sources are received through the first network interfaces;
The system board transmits the network audio and video stream to a second network port through the first network port, and distributes the network audio and video stream to each media board through an internal switch through the second network port.
3. The method of claim 2, wherein after each media board receives the local audio-video stream through its own second input interface, further comprising:
and sending the local audio and video stream to other media boards through the internal switch.
4. A method according to claim 1, 2 or 3, wherein the output interfaces comprise a local output interface and a network output interface; the media board outputs the composite video stream to corresponding external equipment through a current output interface, and the method comprises the following steps:
if the current output interface is a local output interface, outputting the synthesized video stream to corresponding display equipment for display through the local output interface;
and if the current output interface is a network output interface, the synthesized video stream is encoded and then sent to the system board through the network output interface, and the system board uploads the synthesized video stream to a corresponding server.
5. The method of claim 1, wherein the audio-video processing device is connected to a control terminal, the method further comprising:
receiving a picture switching request sent by the control terminal through the system board, wherein the picture switching request comprises a target output interface identifier and updated synthesis configuration information corresponding to the target output interface identifier;
determining a media board corresponding to the target output interface identifier as a target media board;
and sending the updated synthesis configuration information to the target media board, so that the target media board updates the audio/video stream to be output in real time according to the updated synthesis configuration information.
6. The method of claim 1, wherein the audio-video processing device further comprises a display screen;
the method further comprises the steps of:
displaying a configuration page through the display screen, wherein the configuration page comprises an output position configuration item, a split screen mode configuration item and an input interface configuration item, and each output position in the output position configuration item is provided with a corresponding output interface identifier;
receiving configuration information input by a user from the output position configuration item, the split screen mode configuration item and the input interface configuration item respectively; after a user selects a split screen mode from the split screen mode configuration items, entering a split screen configuration page so as to facilitate the user to set the input interface identification of each split screen in the split screen configuration page;
Generating a synthesis rule according to the configuration information of the split screen mode configuration item, and generating synthesis configuration information according to the synthesis rule, the configuration information of the input interface configuration item and the output interface identifier corresponding to the output position configuration item.
7. The method of claim 1 or 2 or 3 or 5 or 6, wherein the interface type of the second input interface of each media board is the same or different, the interface type including at least one of a high definition multimedia interface HDMI interface, a video graphics array VGA interface, a digital component serial interface SDI interface; the communication protocol supported by each of the first input interfaces is the same or different.
8. The audio and video processing device is characterized by being applied to audio and video processing equipment, wherein the audio and video processing equipment comprises a system board and a plurality of media boards, the system board comprises a plurality of first input interfaces, and the first input interfaces are used for receiving network audio and video streams; each media board comprises at least one second input interface and a plurality of output interfaces, wherein the second input interfaces are used for receiving local audio and video streams, and the output interfaces are used for outputting the audio and video streams to connected external equipment;
The device comprises:
the network audio/video stream distribution module is used for receiving a plurality of network audio/video streams through a plurality of first input interfaces by the system board and respectively distributing each network audio/video stream to each media board;
the local audio/video stream receiving module is used for receiving local audio/video streams in each media board through a second input interface of the local audio/video stream receiving module;
the synthesis configuration information acquisition module is used for acquiring synthesis configuration information of each output interface of the media board through the media board, wherein the synthesis configuration information comprises at least one input interface identifier and a corresponding synthesis rule;
the audio/video stream to be output determining module is used for selecting an audio/video stream corresponding to the input interface identifier from a plurality of received network audio/video streams and a local audio/video stream as an audio/video stream to be output according to at least one input interface identifier in the synthesis configuration information of the audio/video stream to be output by the media board for each output interface;
the synthesized video stream generating module is used for synthesizing the audio and video streams to be output through the media board according to the synthesis rule to generate a synthesized video stream;
and the synthesized video stream output module is used for outputting the synthesized video stream to corresponding external equipment through the current output interface through the media board.
9. An audio and video processing device comprises a system board and a plurality of media boards, wherein the system board comprises a plurality of first input interfaces, and the first input interfaces are used for receiving network audio and video streams; each media board comprises at least one second input interface and a plurality of output interfaces, wherein the second input interfaces are used for receiving local audio and video streams, and the output interfaces are used for outputting the audio and video streams to connected external equipment;
the system board is used for:
receiving a plurality of network audio and video streams through a plurality of first input interfaces, and respectively distributing each network audio and video stream to each media board;
the media board is used for:
receiving a local audio/video stream through a second input interface of the local audio/video stream;
acquiring synthesis configuration information of each output interface of the device, wherein the synthesis configuration information comprises at least one input interface identifier and a corresponding synthesis rule;
for each output interface, selecting an audio-video stream corresponding to the input interface identifier from a plurality of received network audio-video streams and local audio-video streams according to at least one input interface identifier in the synthesized configuration information of the output interface as an audio-video stream to be output;
synthesizing the audio and video streams to be output according to the synthesis rule to generate a synthesized video stream;
And outputting the synthesized video stream to corresponding external equipment through a current output interface.
10. A computer readable storage medium storing computer instructions for causing a processor to perform the audio-video processing method of any one of claims 1-7.
CN202310642722.6A 2023-05-31 2023-05-31 Audio and video processing method, device, equipment and medium Active CN116582699B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310642722.6A CN116582699B (en) 2023-05-31 2023-05-31 Audio and video processing method, device, equipment and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310642722.6A CN116582699B (en) 2023-05-31 2023-05-31 Audio and video processing method, device, equipment and medium

Publications (2)

Publication Number Publication Date
CN116582699A true CN116582699A (en) 2023-08-11
CN116582699B CN116582699B (en) 2023-10-13

Family

ID=87535757

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310642722.6A Active CN116582699B (en) 2023-05-31 2023-05-31 Audio and video processing method, device, equipment and medium

Country Status (1)

Country Link
CN (1) CN116582699B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140137169A1 (en) * 2012-11-13 2014-05-15 Askey Computer Corp. Signal expansion selection device
CN108124158A (en) * 2016-11-29 2018-06-05 北京视联动力国际信息技术有限公司 Multimedia terminal and the data processing method of multimedia terminal
CN115802007A (en) * 2022-11-16 2023-03-14 深圳开鸿数字产业发展有限公司 Monitoring system control method and device based on RTSP (real time streaming protocol) and readable storage medium
CN115988269A (en) * 2022-12-21 2023-04-18 北京飞讯数码科技有限公司 Video playing method, device, system, electronic equipment and storage medium
CN116170629A (en) * 2021-11-24 2023-05-26 华为技术有限公司 Method for transmitting code stream, electronic equipment and computer readable storage medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140137169A1 (en) * 2012-11-13 2014-05-15 Askey Computer Corp. Signal expansion selection device
CN108124158A (en) * 2016-11-29 2018-06-05 北京视联动力国际信息技术有限公司 Multimedia terminal and the data processing method of multimedia terminal
CN116170629A (en) * 2021-11-24 2023-05-26 华为技术有限公司 Method for transmitting code stream, electronic equipment and computer readable storage medium
CN115802007A (en) * 2022-11-16 2023-03-14 深圳开鸿数字产业发展有限公司 Monitoring system control method and device based on RTSP (real time streaming protocol) and readable storage medium
CN115988269A (en) * 2022-12-21 2023-04-18 北京飞讯数码科技有限公司 Video playing method, device, system, electronic equipment and storage medium

Also Published As

Publication number Publication date
CN116582699B (en) 2023-10-13

Similar Documents

Publication Publication Date Title
JP5508450B2 (en) Automatic video layout for multi-stream and multi-site telepresence conferencing system
CN101820524A (en) Video display method for video conference
CN101635854B (en) Method and device for realizing transcoding merging
CN112929580A (en) Multi-view video playing method, device, system, server and client device
JPH09219851A (en) Method and equipment for controlling multi-spot video conference
CN106713942B (en) Video processing method and device
US20110090970A1 (en) Terminal and relay node for video conference system
CN102082944A (en) Conference control method, device and system containing tele-presence conference hall
JP2006191641A (en) Multimedia signal matching system and method for performing picture-in-picture function
CN101472138A (en) System and method for implementing share of digital set-top box program stream
CN104601863A (en) IP matrix system for recording and playing
MX2012005455A (en) Mosaic application for generating output utilizing content from multiple television receivers.
CN111385515B (en) Video conference data transmission method and video conference data transmission system
CN108243318B (en) Method and device for realizing live broadcast of multiple image acquisition devices through single interface
US8892888B2 (en) Multiple stream decrypting and decoding systems and related methods thereof
CN100542234C (en) Display device and control method thereof
JP2006211668A (en) Transport stream receiving apparatus for providing multi-screen and control method thereof
CN101212677B (en) Serial multimedia stream handling device and serial stream transmission method
CN102695036A (en) Video conferencing system and application method thereof
CN116582699B (en) Audio and video processing method, device, equipment and medium
CN110784731B (en) Data stream transcoding method, device, equipment and medium
EP1967008A2 (en) Video encoding and decoding
JP2010283683A (en) Device and method for decoding stream
JP2019041328A (en) Medium processing unit, program and method
JP5579657B2 (en) Video frame synchronization system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant