CN107770564B - Method and device for remotely acquiring audio and video data - Google Patents
Method and device for remotely acquiring audio and video data Download PDFInfo
- Publication number
- CN107770564B CN107770564B CN201610685752.5A CN201610685752A CN107770564B CN 107770564 B CN107770564 B CN 107770564B CN 201610685752 A CN201610685752 A CN 201610685752A CN 107770564 B CN107770564 B CN 107770564B
- Authority
- CN
- China
- Prior art keywords
- audio
- data
- network
- video
- video acquisition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
- H04N21/4227—Providing Remote input by a user located remotely from the client device, e.g. at work
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The embodiment of the invention discloses a method for remotely acquiring audio and video data, which comprises the following steps: establishing network connection with remote audio and video acquisition equipment through a network port; registering a virtual local audio/video capture device in an operating system and using the network connection as a data source for the virtual local audio/video capture device; receiving network data transmitted by the remote audio and video acquisition equipment through the network port, wherein the network data is transmitted by the network connection; and transmitting the network data to the virtual local audio/video acquisition equipment, and outputting the network data in a streaming media format through an output port of the virtual local audio/video acquisition equipment. In addition, the embodiment of the invention also correspondingly discloses a device for remotely acquiring the audio and video data. By adopting the method and the device, the operation convenience for remotely acquiring the audio and video data can be improved.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a method and a device for remotely acquiring audio and video data.
Background
With the development of streaming media technology and internet media, live broadcast websites and live broadcast software supporting individual user release live broadcast have come to the fore, after the individual user registers an account, the individual user only needs to install a camera (usually using a USB interface) and a microphone (usually using an audio interface) on a personal computer, or uses a smart phone (usually having a camera and a microphone), and then visits the live broadcast website or logs in the live broadcast software on the personal computer or the smart phone, so that the live broadcast can be conveniently performed, and the individual user can be shown as a main broadcast.
However, when a user uses a live website or live software in the conventional technology, the live website or live software can usually only collect video and audio data through a local camera and a microphone, and collect the recorded live video and audio data by calling a hardware driver of the local camera and a hardware driver of a sound card. The local camera used on the live broadcast computer is usually connected by a USB interface, and the extension line of the USB interface is usually short, so that a user using a live broadcast website and live broadcast software must keep a short distance from the live broadcast computer, the live broadcast space range is limited, and the operation convenience is not enough.
Disclosure of Invention
Based on this, in order to solve the technical problems that the terminal for live broadcasting in the traditional technology can only use the locally installed camera, so that the live broadcasting scene is limited and the operation convenience is insufficient, a method for remotely acquiring audio and video data is especially provided.
A method for remotely acquiring audio and video data comprises the following steps:
establishing network connection with remote audio and video acquisition equipment through a network port;
registering a virtual local audio/video capture device in an operating system and using the network connection as a data source for the virtual local audio/video capture device;
receiving network data transmitted by the remote audio and video acquisition equipment through the network port, wherein the network data is transmitted by the network connection;
and transmitting the network data to the virtual local audio/video acquisition equipment, and outputting the network data in a streaming media format through an output port of the virtual local audio/video acquisition equipment.
Optionally, in one embodiment, the virtual local audio/video capture device includes a virtual camera;
the registering of the virtual local audio/video capture device in the operating system includes:
the virtual camera device is registered in the operating system.
Optionally, in one embodiment, the format of the received network data, which is transmitted by the remote audio/video capture device through the network connection, is a network video coding format;
the outputting in a streaming media format through the output port of the virtual local audio/video capture device comprises:
and decoding the network data in the network video coding format into a streaming media format of the original video acquisition data of the remote audio and video acquisition equipment through the virtual camera equipment.
Optionally, in one embodiment, the network video streaming media format is MPEG, h.264, AVS or h.265; the streaming media format of the original video acquisition data is YUV or RGB format.
Optionally, in one embodiment, the virtual local audio/video capture device includes a virtual sound card and a virtual microphone;
the registering of the virtual local audio/video capture device in the operating system includes:
a virtual sound card and a virtual microphone are registered in the operating system.
Optionally, in one embodiment, the outputting in a streaming media format through the output port of the virtual local audio/video capture device includes:
decoding the network data into audio data in the PCM format through the virtual sound card;
and transmitting the audio data in the PCM format obtained by decoding to the virtual microphone, and outputting the audio data through an output port of the virtual microphone.
Optionally, in one embodiment, the number of the network connections is two or more;
said communicating said network data to said virtual local audio/video capture device further comprises:
and displaying the created network connection, acquiring the selected network connection, and transmitting the network data transmitted by the selected network connection to the virtual local audio/video acquisition equipment.
Optionally, in one embodiment, the remote audio/video acquisition device is an unmanned aerial vehicle including a camera, and the network connection is a wifi connection.
In addition, for thereby solving the live broadcast scene that is used for live broadcast terminal in the conventional art to lead to only using the camera of local installation and being limited, the not enough technical problem of operation convenience, a remote acquisition audio and video data's device has especially been provided.
An apparatus for remotely capturing audio and video data, comprising:
the network connection establishing module is used for establishing network connection with the remote audio and video acquisition equipment through a network port;
the virtual equipment registration module is used for registering virtual local audio/video acquisition equipment in an operating system and using the network connection as a data source of the virtual local audio/video acquisition equipment;
the network data receiving module is used for receiving network data transmitted by the remote audio and video acquisition equipment through the network port;
and the audio/video data output module is used for transmitting the network data to the virtual local audio/video acquisition equipment and outputting the network data in a streaming media format through an output port of the virtual local audio/video acquisition equipment.
Optionally, in one embodiment, the virtual local audio/video capture device includes a virtual camera;
the virtual device registration module is further configured to register the virtual camera device in the operating system.
Optionally, in one embodiment, the format of the received network data, which is transmitted by the remote audio/video capture device through the network connection, is a network video coding format;
the audio and video data output module is also used for decoding the network data in the network video coding format into a streaming media format of the original video acquisition data of the remote audio and video acquisition equipment through the virtual camera equipment.
Optionally, in one embodiment, the network video streaming media format is MPEG, h.264, AVS or h.265; the streaming media format of the original video acquisition data is YUV or RGB format.
Optionally, in one embodiment, the virtual local audio/video capture device includes a virtual sound card and a virtual microphone;
the virtual device registration module is further configured to register a virtual sound card and a virtual microphone in an operating system.
Optionally, in one embodiment, the audio/video data output module is further configured to decode the network data into audio data in the PCM format through the virtual sound card; and transmitting the audio data in the PCM format obtained by decoding to the virtual microphone, and outputting the audio data through an output port of the virtual microphone.
Optionally, in one embodiment, the number of the network connections is two or more;
the audio/video data output module is also used for displaying the created network connection, acquiring the selected network connection and transmitting the network data transmitted by the selected network connection to the virtual local audio/video acquisition equipment.
Optionally, in one embodiment, the remote audio/video acquisition device is an unmanned aerial vehicle including a camera, and the network connection is a wifi connection.
The embodiment of the invention has the following beneficial effects:
after the method and the device for remotely acquiring audio and video data are adopted, the live broadcast terminal does not need to be locally connected with camera equipment, but can be connected with a remote unmanned aerial vehicle, a monitoring camera or a smart phone through a wireless Wi-Fi connection or the Internet to establish network connection, then the live broadcast picture shot by the remote unmanned aerial vehicle, the monitoring camera or the smart phone is acquired through the network connection, and the acquired audio and video data are transmitted into live broadcast software through corresponding output ports simulating actual camera equipment or microphone equipment by registering a virtual camera, a virtual sound card and a virtual microphone, so that the live broadcast terminal cannot be limited in a narrow space by a traditional camera with a USB interface when acquiring the live broadcast picture, but can acquire richer live broadcast materials by the unmanned aerial vehicle, the monitoring camera or the smart phone, thereby improving the convenience of operation. In addition, for live broadcast software, data is read through ports of the virtual camera, the virtual sound card and the virtual microphone, so that the method can realize adaptation without modifying the live broadcast software, and has better adaptability.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Wherein:
fig. 1 is a networking diagram of a system for remotely acquiring audio and video data in one embodiment;
FIG. 2 is a flow diagram of a method for remotely capturing audio and video data, in accordance with one embodiment;
FIG. 3 is an interface diagram to view a virtual camera device in a device manager, under an embodiment;
FIG. 4 is an interface diagram of the selection of a virtual microphone as a data source in live software in one embodiment;
FIG. 5 is a diagram illustrating a video capture process of live software, in one embodiment;
FIG. 6 is a diagram illustrating an audio capture process of live software, in one embodiment;
FIG. 7 is a diagram illustrating a process of data collection and playback for live software, in accordance with an embodiment;
fig. 8 is a schematic structural diagram of an apparatus for remotely acquiring audio/video data according to an embodiment;
fig. 9 is a schematic structural diagram of a computer device operating the foregoing method for remotely acquiring audio-video data in one embodiment.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In order to solve the technical problems that a terminal for live broadcasting in the prior art can only use a locally-installed camera, so that the live broadcasting scene is limited and the operation convenience is insufficient, a method for remotely acquiring audio and video data is particularly provided. The method may be implemented in dependence on a computer program, which may be a virtual device manager or a driver manager of live software. The computer program can run on a computer system based on von neumann architecture, which can be a terminal device such as a personal computer, a notebook computer, a tablet computer, and a smartphone, having a network adapter.
As shown in fig. 1, in the application scenario shown in fig. 1, a computer a and a remote unmanned aerial vehicle B maintain Wi-Fi connection, wherein a camera and a microphone are installed on the unmanned aerial vehicle B, and live broadcast software is installed on the computer a or a live broadcast website is logged in. In the application scene, the computer A provided with the live broadcast APP executes the method for remotely acquiring the audio and video data, receives the audio and video data acquired by the microphone and the camera of the unmanned aerial vehicle through Wi-Fi connection, and outputs the audio and video data through the ports of the virtual microphone and the virtual camera, so that the audio and video data can be transmitted to live broadcast software. To live broadcast software, can directly use the data of this port, and need not carry out the adaptation, consequently, need not modify the third party's any program code of live broadcast software can automatic adaptation, and unmanned aerial vehicle B is owing to through Wi-Fi and computer A wireless connection, therefore can gather real-time audio and video data in the position far away from computer A, this is when making the user live broadcast software on using computer A, the spatial dimension of gathering audio and video can not receive the influence of the connecting wire length of the camera of USB interface among the traditional art, and can utilize long-range audio and video collection equipment such as unmanned aerial vehicle to gather, thereby make when using live broadcast software, the operation convenience is better.
Specifically, as shown in fig. 2, the method for remotely acquiring audio/video data includes:
step S102: and establishing network connection with the remote audio and video acquisition equipment through the network port.
In this embodiment, the remote audio/video acquisition device may be a remote terminal device equipped with an audio/video acquisition device such as a microphone or a camera and a network adapter, such as an unmanned aerial vehicle, a surveillance camera, a smart phone, and the like. Referring to fig. 1, a network adapter is installed on a live broadcast terminal executing the method, and the live broadcast terminal establishes network connection with a remote audio and video acquisition device through the network adapter. The network connection can be a Wi-Fi wireless connection directly established between the live broadcast terminal and the remote audio/video acquisition equipment, or the connection between the live broadcast terminal and the remote audio/video acquisition equipment (for example, a monitoring camera network based on TCP/IP protocol transmission) can be established through the internet or a private network by connecting the live broadcast terminal with a router through a network cable.
The live broadcast terminal can establish TCP connection or UDP connection with remote audio and video acquisition equipment through network connection based on a TCP/IP protocol, and is used for transmitting data. The live broadcast terminal can establish TCP connection or UDP connection with remote audio and video acquisition equipment by using a socket, and the operating system can allocate ports for corresponding connection.
Step S104: registering a virtual local audio/video capture device in an operating system and using the network connection as a data source for the virtual local audio/video capture device.
The virtual local audio/video capture device includes a virtual local video capture device and a virtual local audio capture device. The virtual local video capture device may be a virtual camera device. In the operating system, when a new entity device is inserted into an interface of a terminal, the new entity device is registered in the operating system according to a predefined protocol stack, including device identification and device information, and if a driver of the device exists in the operating system, reading and writing of the device can be realized. In this embodiment, a virtual device may be registered in the operating system by program-simulating the protocol type of the camera device, and a network connection is defined as a data source of the virtual device instead of a specific USB line. Referring to fig. 3, if the registered virtual local video capture device is a virtual camera and the registered device name is "interpolated Camara," it can be observed in the device manager of the windows operating system that a new device "interpolated Camara" will appear in the registered image device classification.
Optionally, the live broadcast terminal may register the virtual camera device in the operating system through the DirectShow component. The DirectShow is a COM-based streaming media processing development package promoted by Microsoft and is released together with a DirectX development package, and the DirectShow is used for developing a virtual camera without being written by a driver, so that the DirectShow is small in size. Corresponding interface functions in a development kit of the assembly can be called to register the virtual camera in an operating system, and then socket sockets established with remote video acquisition equipment such as an unmanned aerial vehicle and a monitoring camera can be used as data sources of the virtual camera.
Optionally, the live broadcast terminal may register a virtual local audio capture device in the operating system through the avstream driver component, where the virtual local audio capture device includes a virtual sound card and a virtual microphone. The virtual microphone implementation principle is that microsoft's avstream driver framework is used, and actually, a virtual microphone and a virtual sound card are implemented, and virtual sound card data directly lead to the virtual microphone. And a virtual microphone which is driven to output the sound received by the virtual sound card to the inside. Therefore, the user layer only needs to decode the received coded audio into a PCM format, and then uses a standard system audio playing API to play the audio to the virtual sound card, so that the external sound can be output to the virtual microphone.
After registering the virtual camera, the virtual sound card and the virtual microphone, the user opens live broadcast software on the live broadcast terminal, as shown in fig. 4, corresponding virtual camera equipment or virtual microphone equipment can be configured on a hardware configuration interface to serve as the camera and the microphone of the live broadcast software, and for the live broadcast software, the user does not know whether the camera or the microphone is actually inserted entity hardware equipment or virtual equipment, and only needs to acquire data through a corresponding port. Therefore, the method can be directly adapted to the live broadcast software without code modification of the live broadcast software, thereby improving the expansibility and the convenience of operation.
Step S106: and receiving network data transmitted by the remote audio and video acquisition equipment through the network port by the network connection.
The video and audio of a far-end scene are collected by the camera equipment on the unmanned aerial vehicle, the monitoring camera or the smart phone which are positioned at the far end of the live broadcast terminal, and then the video and audio are transmitted to live broadcast software through network connection.
Optionally, the remote audio/video acquisition device transmits network data in a network video coding format, that is, a format of the network data received by the live broadcast terminal and transmitted by the remote audio/video acquisition device through the network connection is a network video coding format. The remote audio/video acquisition device generally needs to encode the acquired audio/video data in order to facilitate transmission of a TCP/IP network, and the adopted network video encoding format may be one of MPEG, h.264, AVS or h.265, and the adopted network audio encoding format may be an ACC format.
Step S108: and transmitting the network data to the virtual local audio/video acquisition equipment, and outputting the network data in a streaming media format through an output port of the virtual local audio/video acquisition equipment.
As described above, after the virtual camera, the virtual sound card, and the virtual microphone are registered, the live broadcast terminal may separate the video portion and the audio portion from the received network data, transmit the video portion to the virtual camera, transmit the audio portion to the virtual sound card device, process the network data by the virtual camera and the virtual sound card, and output the network data in a streaming media format through the output port.
In this embodiment, when registering a virtual camera, a format conversion method of a data source of the virtual camera, that is, a method of converting network data into streaming media format data output by an output port by the virtual camera, may be defined. Since the received network data is network data in a network video coding format, corresponding codes may be defined, when the network data in the network video coding format is received, as shown in fig. 5, the network data in the network video coding format may be decoded into a streaming media format of the original video capture data of the remote audio/video capture device by the virtual camera device, where the streaming media format of the original video capture data is YUV (e.g., an original format that is usually captured by a camera device such as YUV420 format) or RGB format. This makes the video output by the output port of the virtual camera be the streaming media data in YUV format.
As described above, since the user has set the camera in the live broadcast software as the virtual camera, the streaming media data in the YUV format can be read from the output port corresponding to the virtual camera, so that the live broadcast software can process the streaming media data. For live broadcast software, the stream media data in the YUV format is data acquired by one camera device, and it is not necessary to care whether the data is acquired from a far end or acquired by a camera accessed by a local USB interface of a live broadcast terminal, so that the adaptability is improved.
For the network data of the audio, a corresponding program code may be added in advance when the virtual sound card is registered, and after the network data is transferred to the virtual sound card by the live broadcast terminal, the corresponding program code is executed, as shown in fig. 6, the network data is decoded into the audio data in the PCM format (the original data collected by the microphone is usually PCM-format encoded data, ACC encoding is performed in network transmission, and the original audio data in the PCM format can be obtained by decoding the encoded data in the virtual sound card), and then the audio data in the PCM format obtained by decoding is transferred to the virtual microphone and output through an output port of the virtual microphone.
As described above, since the user has set the microphone in the live software as the virtual microphone, the streaming media data in the PCM format can be read from the output port corresponding to the virtual microphone, so that the streaming media data can be processed by the live software. For live broadcast software, the streaming media data in the PCM format is data collected by a microphone device accessing a sound card, and it is not necessary to care whether the data is collected from a remote end or a microphone accessed by a local sound card interface of a live broadcast terminal, so that the adaptability is improved.
After the live broadcast terminal reads the audio and video data of remote acquisition through the virtual camera and the virtual microphone, as shown in fig. 7, the server of the live broadcast platform can be transmitted to through the internet after being coded, the server of the live broadcast platform distributes the audio and video data to a plurality of live broadcast clients, and the live broadcast clients can observe the pictures shot by the unmanned aerial vehicle or the monitoring camera.
Furthermore, for the same live broadcast terminal, network connection can be established with a plurality of remote audio and video acquisition devices, the plurality of remote audio and video acquisition devices can acquire audio and video data simultaneously and transmit the audio and video data to the live broadcast terminal through network connection, and the live broadcast terminal can select to access live broadcast software in the audio and video data acquired by the plurality of remote audio and video acquisition devices.
In one embodiment, for each network connection, a corresponding virtual local audio/video capture device may be registered, that is, each network connection registers a virtual camera, a virtual sound card and a virtual microphone, as shown in fig. 4, the user may select the corresponding virtual camera as the captured data source in the video and audio settings of the live broadcast software.
In another embodiment, a management interface of the virtual device may be provided, and the created network connection is displayed in the management interface of the virtual device, so that the user can select a corresponding network connection, and when the user selects a network connection, the selected network connection is obtained, and network data transmitted by the selected network connection is transmitted to the virtual local audio/video acquisition device. Namely, the selected network connection is used as a data source of the registered virtual camera, virtual sound card and virtual microphone. Therefore, when a plurality of remote audio and video acquisition devices exist, switching of the acquired pictures can be realized only by registering one virtual camera, one virtual sound card and one virtual microphone, and computer resources are saved.
It should be noted that, there may be a variety of application scenarios executed by the above method for remotely acquiring audio/video data.
For example, in an application scenario, the remote audio/video acquisition device may be an unmanned aerial vehicle, the live broadcast terminal may be a smart phone or a portable notebook computer, and a user may run live broadcast software on the smart phone, and keep connecting with a server of a live broadcast platform through a data network, while using Wi-Fi connection with the unmanned aerial vehicle. The user can fly unmanned aerial vehicle and then gather the live picture and connect through Wi-Fi and transmit for the smart mobile phone, then the smart mobile phone is transmitting the server for live platform through data network, and the server of direct platform of broadcasting distributes again, and then other users can be real-time the picture of watching unmanned aerial vehicle and shooing, and this also can utilize unmanned aerial vehicle and smart mobile phone to broadcast live when just making the user open air.
In another application scenario, the remote audio/video acquisition device may be a monitoring camera, and the live broadcast terminal may be a personal computer. For example, zoo managers can erect surveillance cameras in a certain garden of a zoo to continuously shoot the situation of the garden, then establish a local area network with the surveillance cameras and an indoor personal computer through laying network cables, the ecological videos of animals shot by the surveillance cameras are transmitted to the personal computer through the local area network, users run live broadcast software on the personal computer, and select virtual cameras and the like corresponding to the network connection as data sources of the live broadcast software, and after the live broadcast software uploads live broadcast platforms in real time, other users can also watch live broadcast pictures of the zoo.
In another application scenario, the remote audio/video acquisition device may be a smart phone, and the live broadcast terminal may be an indoor personal computer, for example, a plurality of users may hold the smart phone to take charge of outdoor scene shooting, and then transmit the data taken by the outdoor scene to the indoor personal computer through a data network, and operate the main broadcast of the personal computer indoors, so that a plurality of pictures taken charge of the outdoor scene shooting may be switched, and selectively uploaded to the live broadcast platform, and when watching the live broadcast of the main broadcast, other users may switch to browse among the live broadcast outdoor scene pictures taken by the plurality of users using the mobile phone.
In addition, in order to solve the technical problems that a terminal for live broadcasting in the conventional technology can only use a locally installed camera, so that a live broadcasting scene is limited and operation convenience is insufficient, in an embodiment, a device for remotely acquiring audio and video data is further provided, as shown in fig. 8, the device for remotely acquiring audio and video data includes a network connection establishing module 102, a virtual device registration module 104, a network data receiving module 106, and an audio and video data output module 108, wherein:
the network connection establishing module 102 is used for establishing network connection with a remote audio and video acquisition device through a network port;
a virtual device registration module 104, configured to register a virtual local audio/video capture device in an operating system, and use the network connection as a data source of the virtual local audio/video capture device;
a network data receiving module 106, configured to receive, through the network port, network data that is transmitted by the remote audio/video acquisition device through the network connection;
and the audio/video data output module 108 is configured to transmit the network data to the virtual local audio/video capture device, and output the network data in a streaming media format through an output port of the virtual local audio/video capture device.
Optionally, in an embodiment, the virtual local audio/video capture device includes a virtual camera; the virtual device registration module 104 is further configured to register the virtual camera device in the operating system.
Optionally, in an embodiment, the format of the received network data, which is transmitted by the remote audio/video capture device through the network connection, is a network video coding format; the audio/video data output module 108 is further configured to decode, by the virtual camera device, the network data in the network video coding format into a streaming media format of the original video acquisition data of the remote audio/video acquisition device.
Optionally, in an embodiment, the network video streaming media format is MPEG, h.264, AVS or h.265; the streaming media format of the original video acquisition data is YUV or RGB format.
Optionally, in an embodiment, the virtual local audio/video capture device includes a virtual sound card and a virtual microphone; the virtual device registration module 104 is also configured to register a virtual sound card and a virtual microphone in the operating system.
Optionally, in an embodiment, the audio/video data output module 108 is further configured to decode the network data into audio data in the PCM format through the virtual sound card; and transmitting the audio data in the PCM format obtained by decoding to the virtual microphone, and outputting the audio data through an output port of the virtual microphone.
Optionally, in an embodiment, the number of the network connections is two or more; the audio/video data output module 108 is further configured to display the created network connection, obtain a selected network connection, and transmit network data transmitted by the selected network connection to the virtual local audio/video capture device.
Optionally, in an embodiment, the remote audio/video acquisition device is an unmanned aerial vehicle including a camera, and the network connection is a wifi connection.
After the method and the device for remotely acquiring audio and video data are adopted, the live broadcast terminal does not need to be locally connected with camera equipment, but can be connected with a remote unmanned aerial vehicle, a monitoring camera or a smart phone through a wireless Wi-Fi connection or the Internet to establish network connection, then the live broadcast picture shot by the remote unmanned aerial vehicle, the monitoring camera or the smart phone is acquired through the network connection, and the acquired audio and video data are transmitted into live broadcast software through corresponding output ports simulating actual camera equipment or microphone equipment by registering a virtual camera, a virtual sound card and a virtual microphone, so that the live broadcast terminal cannot be limited in a narrow space by a traditional camera with a USB interface when acquiring the live broadcast picture, but can acquire richer live broadcast materials by the unmanned aerial vehicle, the monitoring camera or the smart phone, thereby improving the convenience of operation. In addition, for live broadcast software, data is read through ports of the virtual camera, the virtual sound card and the virtual microphone, so that the method can realize adaptation without modifying the live broadcast software, and has better adaptability.
In one embodiment, as shown in fig. 9, fig. 9 illustrates a terminal 10 of a computer system based on von neumann architecture, which runs the above-described method of remotely collecting audio-video data. The computer system can be terminal equipment such as a smart phone, a tablet computer, a palm computer, a notebook computer or a personal computer. Specifically, an external input interface 1001, a processor 1002, a memory 1003, and an output interface 1004 connected through a system bus may be included. The external input interface 1001 may optionally include at least a network interface 10012. Memory 1003 can include external memory 10032 (e.g., a hard disk, optical or floppy disk, etc.) and internal memory 10034. The output interface 1004 may include at least a display 10042 or the like.
In this embodiment, the method is executed based on a computer program, program files of which are stored in the external memory 10032 of the computer system 10 based on the von neumann architecture, loaded into the internal memory 10034 at the time of execution, and then compiled into machine code and then transferred to the processor 1002 to be executed, so that the computer system 10 based on the von neumann architecture forms the logical network connection establishing module 102, the virtual device registering module 104, the network data receiving module 106, and the audio/video data output module 108. In the execution process of the method for remotely acquiring audio and video data, the input parameters are received through the external input interface 1001, transferred to the memory 1003 for caching, and then input into the processor 1002 for processing, and the processed result data is cached in the memory 1003 for subsequent processing or transferred to the output interface 1004 for outputting.
Specifically, the processor 1002 is further configured to execute the following steps:
establishing network connection with remote audio and video acquisition equipment through a network port;
registering a virtual local audio/video capture device in an operating system and using the network connection as a data source for the virtual local audio/video capture device;
receiving network data transmitted by the remote audio and video acquisition equipment through the network port, wherein the network data is transmitted by the network connection;
and transmitting the network data to the virtual local audio/video acquisition equipment, and outputting the network data in a streaming media format through an output port of the virtual local audio/video acquisition equipment.
Optionally, in an embodiment, the processor 1002 is further configured to perform: the virtual camera device is registered in the operating system.
Optionally, in an embodiment, the processor 1002 is further configured to perform: and decoding the network data in the network video coding format into a streaming media format of the original video acquisition data of the remote audio and video acquisition equipment through the virtual camera equipment.
Optionally, in an embodiment, the processor 1002 is further configured to perform: a virtual sound card and a virtual microphone are registered in the operating system.
Optionally, in an embodiment, the processor 1002 is further configured to perform: decoding the network data into audio data in the PCM format through the virtual sound card; and transmitting the audio data in the PCM format obtained by decoding to the virtual microphone, and outputting the audio data through an output port of the virtual microphone.
Optionally, in an embodiment, the processor 1002 is further configured to perform: and displaying the created network connection, acquiring the selected network connection, and transmitting the network data transmitted by the selected network connection to the virtual local audio/video acquisition equipment.
The above disclosure is only for the purpose of illustrating the preferred embodiments of the present invention, and it is therefore to be understood that the invention is not limited by the scope of the appended claims.
Claims (9)
1. A method for remotely acquiring audio and video data is characterized in that the method is applied to a live broadcast terminal, live broadcast software is installed in the live broadcast terminal, and the method comprises the following steps:
establishing network connection with two or more remote audio and video acquisition devices through a network port;
for each network connection, simulating a protocol type of audio and video acquisition equipment through a program, registering virtual camera equipment in an operating system based on a DirectShow component, registering virtual local audio acquisition equipment in the operating system based on an avstream driving component, and taking the network connection as a data source of the virtual local audio/video acquisition equipment corresponding to the network connection, wherein the virtual local audio and video acquisition equipment comprises the virtual camera equipment and the virtual local audio acquisition equipment, and the virtual local audio acquisition equipment comprises a virtual sound card and a virtual microphone;
acquiring a selected target virtual local audio and video acquisition device, wherein the target virtual local audio and video acquisition device is selected through a hardware configuration interface of the live broadcast software;
receiving network data which is transmitted by a remote audio and video acquisition device corresponding to the selected target virtual local audio and video acquisition device through a network port and is transmitted by a network connection corresponding to the selected target virtual local audio and video acquisition device, wherein the network data comprises video data in a network video coding format and audio data in a network audio coding format;
separating the video data and the audio data from the network data, transmitting the video data to a target virtual camera device corresponding to the target virtual local audio and video acquisition device, and transmitting the audio data to a target virtual sound card device corresponding to the target virtual local audio and video acquisition device;
decoding the video data in the network video coding format into a streaming media format of original video acquisition data of the remote audio and video acquisition equipment through the target virtual camera equipment, decoding the audio data in the network audio coding format into a PCM format through the target virtual sound card, transmitting the audio data in the PCM format obtained by decoding to target virtual microphone equipment corresponding to the target virtual local audio and video acquisition equipment, and outputting the decoded video data through an output port of the target virtual camera equipment and outputting the decoded audio data through the target virtual microphone equipment;
and the live broadcast software encodes the decoded video data and the decoded audio data to obtain encoded network data, and outputs the encoded network data to a server of a live broadcast platform so that the server distributes the encoded network data to a plurality of live broadcast clients.
2. A method for remotely capturing audiovisual data according to claim 1, wherein said network video encoding format is MPEG, h.264, AVS or h.265; the streaming media format of the original video acquisition data is YUV or RGB format.
3. A method for remotely acquiring audio-visual data according to claim 1 or 2, characterised in that the number of said network connections is two or more;
the acquiring of the selected target virtual local audio and video acquisition device comprises:
displaying a hardware configuration interface of the live broadcast software, wherein the created network connection is displayed in the hardware configuration interface of the live broadcast software;
acquiring a network connection selected in a hardware configuration interface of the live broadcast software;
and taking the virtual local audio and video acquisition equipment corresponding to the selected network connection as the target virtual local audio and video acquisition equipment.
4. The method according to claim 1 or 2, wherein the remote audio/video acquisition device is an unmanned aerial vehicle comprising a camera, and the network connection is a wifi connection.
5. The utility model provides a device of long-range collection audio frequency and video data, its characterized in that, the device is applied to in the live terminal, live terminal installs live broadcast software, the device includes:
the network connection establishing module is used for establishing network connection with two or more remote audio and video acquisition devices through a network port;
the virtual device registration module is used for simulating the protocol type of audio/video acquisition equipment through a program for each network connection, registering virtual camera equipment in an operating system based on a DirectShow component, registering virtual local audio acquisition equipment in the operating system based on an avstream driving component, and using the network connection as a data source of virtual local audio/video acquisition equipment corresponding to the network connection, wherein the virtual local audio/video acquisition equipment comprises the virtual camera equipment and the virtual local audio acquisition equipment, and the virtual local audio acquisition equipment comprises a virtual sound card and a virtual microphone;
the network data receiving module is used for acquiring a selected target virtual local audio and video acquisition device, wherein the target virtual local audio and video acquisition device is selected through a hardware configuration interface of the live broadcast software; receiving network data which is transmitted by a remote audio and video acquisition device corresponding to the selected target virtual local audio and video acquisition device through a network port and is transmitted by a network connection corresponding to the selected target virtual local audio and video acquisition device, wherein the network data comprises video data in a network video coding format and audio data in a network audio coding format;
the audio and video data output module is used for separating the network data into the video data and the audio data, transmitting the video data to target virtual camera equipment corresponding to the target virtual local audio and video acquisition equipment and transmitting the audio data to target virtual sound card equipment corresponding to the target virtual local audio and video acquisition equipment; decoding the video data in the network video coding format into a streaming media format of original video acquisition data of the remote audio and video acquisition equipment through the target virtual camera equipment, decoding the audio data in the network audio coding format into a PCM format through the target virtual sound card, transmitting the audio data in the PCM format obtained by decoding to target virtual microphone equipment corresponding to the target virtual local audio and video acquisition equipment, and outputting the decoded video data through an output port of the target virtual camera equipment and outputting the decoded audio data through the target virtual microphone equipment; and the live broadcast software encodes the decoded video data and the decoded audio data to obtain encoded network data, and outputs the encoded network data to a server of a live broadcast platform so that the server distributes the encoded network data to a plurality of live broadcast clients.
6. The apparatus for remotely capturing audiovisual data according to claim 5, wherein said network video encoding format is MPEG, h.264, AVS or h.265; the streaming media format of the original video acquisition data is YUV or RGB format.
7. A device for remotely acquiring audio-visual data according to claim 5 or 6, characterised in that the number of said network connections is two or more;
when the network data receiving module acquires the selected target virtual local audio/video acquisition device, the network data receiving module is used for:
displaying a hardware configuration interface of the live broadcast software, wherein the created network connection is displayed in the hardware configuration interface of the live broadcast software;
acquiring a network connection selected in a hardware configuration interface of the live broadcast software;
and taking the virtual local audio and video acquisition equipment corresponding to the selected network connection as the target virtual local audio and video acquisition equipment.
8. The device of claim 5 or 6, wherein the remote audio/video acquisition device is an unmanned aerial vehicle comprising a camera, and the network connection is a wifi connection.
9. A terminal, characterized in that the terminal comprises a processor and a memory;
the memory for storing program code;
the processor configured to execute the code in the memory to cause the terminal to perform the method of any one of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610685752.5A CN107770564B (en) | 2016-08-18 | 2016-08-18 | Method and device for remotely acquiring audio and video data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610685752.5A CN107770564B (en) | 2016-08-18 | 2016-08-18 | Method and device for remotely acquiring audio and video data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107770564A CN107770564A (en) | 2018-03-06 |
| CN107770564B true CN107770564B (en) | 2021-07-27 |
Family
ID=61262712
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610685752.5A Active CN107770564B (en) | 2016-08-18 | 2016-08-18 | Method and device for remotely acquiring audio and video data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107770564B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113113029A (en) * | 2018-08-29 | 2021-07-13 | 胡开良 | Unmanned aerial vehicle voiceprint news tracking method |
| CN111277588A (en) * | 2020-01-19 | 2020-06-12 | 深圳市朗强科技有限公司 | Data sending and receiving method, device and system |
| CN112910899A (en) * | 2021-02-02 | 2021-06-04 | 深圳市瑞驰信息技术有限公司 | Audio and video transparent transmission method under android system |
| CN112988249B (en) * | 2021-03-23 | 2024-05-28 | 上海摩勤智能技术有限公司 | Method and device for creating sound card, electronic equipment and storage medium |
| CN113271425A (en) * | 2021-04-19 | 2021-08-17 | 瑞芯微电子股份有限公司 | Interaction system and method based on virtual equipment |
| CN116614650A (en) * | 2023-06-16 | 2023-08-18 | 上海随幻智能科技有限公司 | Voice and picture synchronous private domain live broadcast method, system, equipment, chip and medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101951515A (en) * | 2010-09-21 | 2011-01-19 | 深圳市同洲电子股份有限公司 | Method, system and set-top box for sharing interface of mobile terminal to television |
| CN103702175A (en) * | 2013-12-28 | 2014-04-02 | 珠海全志科技股份有限公司 | Multi-screen interactive method and multi-screen interactive device |
| CN104683863A (en) * | 2013-11-28 | 2015-06-03 | 中国移动通信集团公司 | Method and equipment for multimedia data transmission |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7996878B1 (en) * | 1999-08-31 | 2011-08-09 | At&T Intellectual Property Ii, L.P. | System and method for generating coded video sequences from still media |
| CN101833520B (en) * | 2009-03-11 | 2013-01-16 | 凹凸电子（武汉）有限公司 | Method for simultaneously utilizing one camera by a plurality of application programs |
| CN102143144B (en) * | 2010-10-15 | 2015-05-27 | 华为终端有限公司 | Communication module, remote monitoring system and remote monitoring terminal |
| CN102869123B (en) * | 2012-08-02 | 2016-05-25 | Tcl集团股份有限公司 | Be applied to virtual peripheral collecting method and the system of intelligent display device |
| US20140320592A1 (en) * | 2013-04-30 | 2014-10-30 | Microsoft Corporation | Virtual Video Camera |
| CN103327417A (en) * | 2013-07-11 | 2013-09-25 | 亿览在线网络技术（北京）有限公司 | Method and device for directly broadcasting real-time long-distance audio and video frequency |
| CN103686328A (en) * | 2013-12-23 | 2014-03-26 | 乐视致新电子科技（天津）有限公司 | Method and device for adding camera to smart television |
-
2016
- 2016-08-18 CN CN201610685752.5A patent/CN107770564B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101951515A (en) * | 2010-09-21 | 2011-01-19 | 深圳市同洲电子股份有限公司 | Method, system and set-top box for sharing interface of mobile terminal to television |
| CN104683863A (en) * | 2013-11-28 | 2015-06-03 | 中国移动通信集团公司 | Method and equipment for multimedia data transmission |
| CN103702175A (en) * | 2013-12-28 | 2014-04-02 | 珠海全志科技股份有限公司 | Multi-screen interactive method and multi-screen interactive device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107770564A (en) | 2018-03-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107770564B (en) | Method and device for remotely acquiring audio and video data | |
| US8659638B2 (en) | Method applied to endpoint of video conference system and associated endpoint | |
| CN106162221A (en) | The synthetic method of live video, Apparatus and system | |
| CN104660891A (en) | Method and apparatus in a motion video capturing system | |
| US10165229B2 (en) | Network communication system and method thereof | |
| US10044782B2 (en) | Terminal device, data transmission method, and computer-readable recording medium | |
| CN111092898B (en) | Message transmission method and related equipment | |
| US20180014063A1 (en) | Method and Apparatus for Accessing a Terminal Device Camera to a Target Device | |
| CN108055595B (en) | Video image redirection method and computer-readable storage medium | |
| CN103248685A (en) | Smart phone-based P2P monitoring method and system | |
| CN105933738A (en) | Live video streaming method, device and system | |
| CN114040232B (en) | Screen projection system, screen projection method, electronic equipment and storage medium | |
| CN110267093A (en) | Live video method for pushing, device, storage medium, terminal and live streaming mirror | |
| US10644928B2 (en) | Device, system, and method to perform real-time communication | |
| CN113438442A (en) | Conference data sharing method and device | |
| CN104053045A (en) | Display device and share control method thereof | |
| CN114630051A (en) | Video processing method and system | |
| CN112261421A (en) | Virtual reality display method and device, electronic equipment and storage medium | |
| CN105451004A (en) | Intelligent set top box, and media processing system and method supporting video communication monitoring | |
| US9648276B2 (en) | Transmission management apparatus, transmission system, transmission management method and recording medium | |
| CN112822435A (en) | Security method, device and system allowing user to easily access | |
| CN104639501A (en) | Data stream transmission method, equipment and system | |
| CN103581607A (en) | Method for transmitting video stream to local endpoint host using remote camera device | |
| CN106664432B (en) | Multimedia information playing method and system, acquisition equipment and standardized server | |
| CN104038741A (en) | Projection method for video data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |