CN107835433B - Event wide-view live broadcasting system, associated equipment and live broadcasting method - Google Patents
Event wide-view live broadcasting system, associated equipment and live broadcasting method Download PDFInfo
- Publication number
- CN107835433B CN107835433B CN201710432123.6A CN201710432123A CN107835433B CN 107835433 B CN107835433 B CN 107835433B CN 201710432123 A CN201710432123 A CN 201710432123A CN 107835433 B CN107835433 B CN 107835433B
- Authority
- CN
- China
- Prior art keywords
- wide
- view
- video stream
- streaming media
- media server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/242—Synchronization processes, e.g. processing of PCR [Program Clock References]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/643—Communication protocols
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention relates to a race wide-view live broadcasting system which comprises wide-view live broadcasting equipment, a streaming media server and playing equipment, wherein the wide-view live broadcasting equipment acquires a plurality of wide-view image frames; obtaining an internal parameter matrix and a distortion coefficient matrix of the wide-view-angle camera unit; sending the internal parameter matrix and the distortion coefficient matrix to a streaming media server; encoding the plurality of wide view image frames into a generated video stream; and sending the video stream to a streaming media server; the streaming media server schedules and caches the video stream, and the playing device decodes, expands and maps the video stream.
Description
Technical Field
The present application relates to live devices and methods, and more particularly, to live event devices, systems, and methods.
Background
Live events generally refer to the process of transmitting images and sounds of the event scene to, for example, a television, a radio, a computer, a mobile terminal, etc. through a cable television network, a broadcast network, or the internet, etc. in real time. With the development of internet technology, live broadcast technology based on internet has been developed, for example, streaming media technology appears, which can compress media data in sections and then send the data through a network, and transmit video and audio on the network in real time for a viewer to use specific playing software to play and view, and the viewer can use the playing software to enjoy live broadcast pictures of a certain shooting angle of a video capture device in real time.
When the sports racing car events such as the rally are rebroadcast, a relatively wide view angle is usually required to embody a large scene, and long-time playing of videos with wide view angles can lead viewers to tired emotion, thereby losing interest in live broadcasting. Conventional live broadcasting techniques usually employ time-sequential scene switching to realize live broadcasting from different angles, such as playing a first local close-up section at a first time, playing a panoramic section at a second time, and the like. Although pip technology is available, it occupies extra resources in the digital transmission mode, and thus has been rarely applied in digital tv systems.
Accordingly, there is a need for an apparatus, system, and associated method that enables wide-view live broadcasting of events.
Disclosure of Invention
Some embodiments of the present application relate to a wide-view live broadcasting system for an event, including a wide-view live broadcasting device, which may include a wide-view camera unit for acquiring a plurality of wide-view image frames; a memory for storing a first sequence of computer instructions; a processor for executing the first sequence of computer instructions to: obtaining an internal parameter matrix and a distortion coefficient matrix of the wide-view-angle camera unit; sending the internal parameter matrix and the distortion coefficient matrix to a streaming media server; encoding the plurality of wide view image frames into a generated video stream, and transmitting the video stream to a streaming media server; the streaming media server caches and schedules the internal parameter matrix, the distortion coefficient matrix and the wide-view-angle image frame; a playback device comprising a memory for storing a second sequence of computer instructions; a processor for executing the second sequence of computer instructions to: receiving the internal parameter matrix and distortion coefficient matrix from at least one wide view camera unit and the wide view video stream from the streaming media server; decoding the wide view video stream back into the plurality of wide view image frames; correcting the wide-view image frame based on the internal parameter matrix and the distortion coefficient matrix to obtain an expanded wide-view image frame; and playing the expanded wide-view image frame according to the generation time sequence of the wide-view image frame.
According to some embodiments of the present application, the wide-view live device pushes the wide-view video stream to the streaming server based on an RTMP or RTSP protocol, or the wide-view live device transmits the wide-view video stream to the streaming server based on an RTMP/RTSP/HTTP request from the streaming server.
According to some embodiments of the application, the event wide-view live broadcasting system further comprises a sound acquisition unit, which is used for acquiring sound field signals of the live broadcasting equipment of the racing cars.
According to some embodiments of the present application, the first memory further stores a third sequence of computer instructions that the processor executes to digitize the sound field signal into audio data; encoding the audio data and the wide-view image to be packaged into a wide-view audio/video stream; and transmitting the wide-view audio and video stream to the streaming media server.
According to some embodiments of the present application, the event comprises a vehicle athletic event, the event wide-view live device being disposed on a vehicle. According to some embodiments of the present application, the event comprises a vehicular athletic event, the event wide-view live device being disposed on an event floor.
According to some embodiments of the present application, the event wide-view live broadcasting system may further include at least one common-view live broadcasting device, where the common-view live broadcasting device generates a common-view video stream and transmits the common-view video stream to the streaming media server.
According to some embodiments of the present application, the streaming media system concurrently transmits the normal view video stream received from the normal view live device and the expanded wide view video stream received from the wide view live device to a live server.
Still other embodiments of the present application provide a playback device for a wide-view live event system, comprising a memory for storing a sequence of computer instructions; a processor for executing the sequence of computer instructions to: receiving an internal parameter matrix and a distortion coefficient matrix from at least one wide-view camera unit from a streaming media server; receiving a wide-view video stream from the wide-view camera unit from a streaming server; decoding the wide view video stream to restore into a plurality of wide view image frames; correcting the wide-view image frame based on the internal parameter matrix and the distortion coefficient matrix to obtain an expanded wide-view image frame; and playing the expanded wide-view image frame according to the generation time sequence of the wide-view image frame.
According to the playing device of the event wide-view live broadcasting system, the streaming media server can transmit the wide-view video stream to the playing device based on RTMP/RTSP/HTTP protocol.
Some embodiments of the present application relate to a live event method, which may include: sending the measured internal parameters and distortion coefficients to a streaming media server; generating a wide-view video stream or an audio/video stream by at least one event wide-view live broadcast device; transmitting the wide-view video stream or the audio/video stream to the streaming media server; generating a common visual angle video stream or an audio/video stream by at least one common live broadcast device; transmitting the common visual angle video stream or the audio and video stream to the streaming media server; the streaming media server transmits the wide-view video stream or the audio/video stream and the common-view video stream or the audio/video stream to a live broadcast server in parallel; and the live broadcast server synchronously or asynchronously provides the wide view video stream or the audio and video stream and the common view video stream or the audio and video stream to a playing device.
In some embodiments, sending the measured intrinsic parameters and distortion coefficients to the streaming media server includes sending the measured intrinsic parameters and distortion coefficients to the streaming media server based on the RTMP metadata packet
Some embodiments of the present application relate to a live event method that may include sending measured intrinsic parameters and distortion coefficients to a streaming server; generating an expanded wide-view video stream or audio/video stream by at least one event wide-view live broadcast device; transmitting the expanded wide-view video stream or audio/video stream to the streaming media server; generating a common visual angle video stream or an audio/video stream by at least one common live broadcast device; transmitting the common visual angle video stream or the audio and video stream to the streaming media server; the streaming media server transmits the wide-view video stream or the audio/video stream and the common-view video stream or the audio/video stream to a live broadcast server in parallel; and the live broadcast server provides the wide visual angle video stream or audio and video stream and the common visual angle video stream or audio and video stream to a playing device according to the selected operation of the playing device.
According to some embodiments of the present application, sending the measured intrinsic parameters and distortion coefficients to the streaming server includes sending the measured intrinsic parameters and distortion coefficients to the streaming server via an RTMP-based metadata packet.
The live broadcast system and the live broadcast method with the wide viewing angles are particularly suitable for scenes such as live broadcast of events, suitable solutions are provided for live broadcast of off-road events and the like, and particularly, application of live broadcast equipment with the wide viewing angles of the events to a streaming media live broadcast platform is provided.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments made with reference to the following drawings:
FIG. 1 is a schematic diagram of a scenario according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a live broadcast device according to an embodiment of the present invention;
fig. 3 is a flow diagram of image processing of a live device in accordance with an embodiment of the invention;
fig. 4 is a schematic diagram of a live system according to an embodiment of the invention;
fig. 5 is a schematic diagram of a playing interface of a playing device according to an embodiment of the present invention; and
fig. 6 is a flowchart illustrating the expanding function of a playback device according to an embodiment of the present invention.
Detailed Description
Before discussing exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although a flowchart may describe the operations as a sequential process, many of the operations can be performed in parallel, concurrently, or simultaneously. In addition, the order of the operations may be re-arranged. The process may be terminated when its operations are completed, but may have additional steps not included in the figure. The processes may correspond to methods, functions, procedures, subroutines, and the like.
The term "server" in this context refers to an intelligent electronic device that can perform predetermined processes such as numerical calculation and/or logic calculation by executing predetermined programs or instructions, and may include a processor and a memory, wherein the predetermined processes are performed by the processor executing program instructions prestored in the memory, or the predetermined processes are performed by hardware such as ASIC, FPGA, DSP, or a combination thereof.
Wherein the user equipment includes but is not limited to smart phones, PDAs, PCs, notebook computers, etc.; the network device includes, but is not limited to, a single network server, a server group consisting of a plurality of network servers, or a Cloud Computing (Cloud Computing) based Cloud consisting of a large number of computers or network servers, wherein Cloud Computing is one of distributed Computing, a super virtual computer consisting of a collection of loosely coupled computers. Wherein the computer device can be operated alone to implement the invention, or can be accessed to a network and implement the invention through interoperation with other computer devices in the network. The network in which the computer device is located includes, but is not limited to, the internet, a wide area network, a metropolitan area network, a local area network, a VPN network, and the like.
It should be noted that the user equipment, the network device, the network, etc. are only examples, and other existing or future computer devices or networks may also be included in the scope of the present invention, and are included by reference.
The methodologies discussed hereinafter, some of which are illustrated by flow diagrams, may be implemented by hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware or microcode, the program code or code segments to perform the necessary tasks may be stored in a machine or computer readable medium such as a storage medium. The processor(s) may perform the necessary tasks.
Specific structural and functional details disclosed herein are merely representative and are provided for purposes of describing example embodiments of the present invention. The present invention may, however, be embodied in many alternate forms and should not be construed as limited to only the embodiments set forth herein.
It will be understood that, although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first element may be termed a second element, and, similarly, a second element may be termed a first element, without departing from the scope of example embodiments. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used herein, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It should also be noted that, in some alternative implementations, the functions/acts noted may occur out of the order noted in the figures. For example, two figures shown in succession may, in fact, be executed substantially concurrently, or the figures may sometimes be executed in the reverse order, depending upon the functionality/acts involved.
The live event system of one embodiment of the present invention may be adapted for use in live events such as vehicle racing which may include, for example, field racing, such as slot and cross-country racing, such as rallies. FIG. 1 illustrates a schematic view of a live system scenario for a vehicle athletic event. As shown in FIG. 1, a typical vehicle athletic event includes at least two vehicles, at least one of which has a live device mounted thereon. The field of the event may have live broadcast devices oriented at different shooting angles, the live broadcast devices 10 may generate video streams, audio streams, or audio/video streams, the audio streams, or the audio/video streams may be transmitted to a streaming media server, and the streaming media server 20 may transmit the video streams, the audio streams, or the audio/video streams to a playing device of a user equipment, such as a smart phone, through a live broadcast platform for playing. The live platform 30 may be a live server that may have a director system controlling the video or audio-video streams transmitted to the user devices and selecting the video, audio or audio-video streams received by the live platform 30. The live platform 30 may store the received video stream, audio stream, or audio/video stream in different storage locations, and the user device 40 may view the video stream or audio/video stream by accessing an address specified by the live platform.
Among the live devices shown in fig. 1, some of the live devices may be live devices 10 having wide-angle camera units. A wide viewing angle is an acquisition device with a viewing angle greater than 170 degrees. The wide-angle camera unit can be, for example, an image with a fisheye lens, a video capture module, such as a fisheye camera. These devices with wide-angle lenses may be deployed, for example, inside a vehicle, such as an instrument desk or a roof, or anywhere on a racing track.
In the illustration shown in fig. 1, a part of the vehicles is equipped with a wide-view live broadcasting device 10 having a fisheye camera, which may also be referred to as a fisheye live broadcasting device, and the fisheye live broadcasting devices 10A, 10B, 10C, and 10D may be mounted on the vehicles, for example, or may be mounted in a stadium, as shown by live broadcasting devices 10E and 10F.
Fig. 2 shows a schematic diagram of a live device 10 with a fisheye camera in an embodiment. Referring to fig. 2, the live broadcasting device includes a fisheye camera 11, a memory 12, a wired or wireless output unit 13, and a processor 14. The processor may communicate with the fisheye camera, memory, wired or wireless output unit via bus 16. The image acquired by the fisheye camera 11 is an image with a viewing angle substantially larger than 180 degrees, and a program may be stored in the memory 12, and when the processor runs the program, the image acquired by the fisheye camera is firstly acquired or received S101 and the camera intrinsic parameters and distortion parameters S102, and then the acquired or received image data is subjected to pre-processing S200.
The preprocessing may include one or more of vignetting processing and image position correction of the image.
After the pre-processing, the wide-view live broadcast device encodes a plurality of image frames to form a wide-view video stream. And transmitting the wide-view video stream to user equipment, namely playing equipment.
Through the above procedures, a plurality of wide view image frames may be generated at the wide view live broadcast device 10, and the wide view image frames may be encoded and compressed according to the image acquisition timing to generate a wide view video stream, and transmitted to the streaming media server 20 based on, for example, an RTMP (real time messaging protocol) or RTSP (real time streaming protocol).
The streaming media server 20 may be geographically located remotely from the live platform 30. For example, the streaming server 20 may be located in the center of the track, or on any vehicle.
The streaming media server 20 may store the received video stream segment in the storage location 31 of the live platform 30 for the user equipment to retrieve, or may also directly transmit the video stream segment to the user equipment through the live platform based on the RTMP, RTSP, HLS protocol or HTTP FLV protocol.
The live device 10 may be provided with a sound collection unit 15, the sound collection unit 15 collecting sound field signals, and the sound field signals may be digitized into audio data by a program run by a processor, and the audio data may be pre-processed, e.g., de-noised, at the live device. The audio data and the image after the pre-processing are synchronized, and are encapsulated into audio and video streams based on the RTMP or RTSP protocol after being encoded and compressed, and then the audio and video streams are transmitted to the streaming media server 20.
A request for transmitting a video stream or an audio/video stream may also be sent to the live device 10 by the streaming media server 20 at regular time, and the live device 10 may transmit the video stream or the audio/video stream generated by the encoding compression to the streaming media server 20 in response to the request.
The streaming server 20 stores and simultaneously plays the obtained stream live to the user device 40 of the viewer through the live platform 30.
The live system 30 may also obtain video streams or audio-video streams from a common viewing angle live device 50. One or more of the normal-view live devices 50 may be arranged. May be arranged on the vehicle or at any position of the playing field. The normal view live broadcast device may have a similar structure to the wide view live broadcast device. For example, an image acquisition unit, a processor, a memory, a bus, a wired or wireless output unit, etc. may be included as well. The normal view live broadcasting device 50 can generate a video stream only by correcting the acquired image, and push the video stream to the streaming media server.
Similarly, the normal-view live broadcasting device 50 may also include a sound collection unit for acquiring sound field signals and digitally generating audio data. The audio data can be encoded to form an audio stream and pushed to the streaming media server or encapsulated into an audio/video stream together with the video stream and pushed to the streaming media server.
The delivery of the expanded wide view video stream or audio-visual stream and the normal view expanded video stream or audio-visual stream may be controlled via the director system 32 of the live platform.
The video stream, the audio stream, or the audio/video stream generated by the normal viewing angle live broadcasting device 50 may be pushed to the streaming media server 20 in parallel with the expanded wide viewing angle video stream or the expanded audio/video stream generated by the wide viewing angle live broadcasting device. The streaming media server 20 may synchronize the expanded wide view video stream or audio/video stream with the normal view video stream or audio/video stream and then transmit the synchronized stream to the live broadcast platform for storage, or transmit the synchronized stream to a playing device such as a smart phone through the live broadcast platform.
The expanded wide-view video stream or audio-video stream and the common-view video stream or audio-video stream can also be asynchronously transmitted to the playing device.
The user equipment, namely the playing equipment can request the internal parameters and the distortion coefficient matrix of the specific wide-view live broadcast equipment from the streaming media server; the user equipment can also request the streaming media server for the wide-view video stream or the wide-view audio and video stream of the wide-view live broadcast equipment. Conversely, the streaming media server may also push the internal parameter and distortion coefficient matrix of the specific wide-view live broadcast device and the wide-view video stream or wide-view audio/video stream of the wide-view live broadcast device to the specific user equipment.
The user equipment can have a wide-view video stream or wide-view audio and video stream expansion playing function. As shown in fig. 6, the function may include decoding a wide view video stream or a wide view audio/video stream to obtain a plurality of wide view image frames S602 after receiving the wide view video stream or the wide view audio/video stream S601. And expanding the wide-view image frame based on the received internal parameters and distortion coefficient matrix of the wide-view live broadcast equipment to obtain an expanded wide-view image frame S603. Finally, the multiple expanded wide-view image frames are subjected to mapping operation, such as rectangular mapping operation, on the expanded wide-view image frames according to a time sequence, so as to form a video frame on a playing interface of the playing device S604.
The image expansion process may use, for example, an equal arc length mapping technique of a cylindrical model, i.e., a cylindrical model with an equal radius to a spherical model of a fisheye lens, wherein the spherical model of the fisheye lens is based on CCS (camera coordinate system) and the cylindrical model is based on WCS (world coordinate system). The coordinate system of the cylindrical model and the coordinate system of the spherical model need to be corresponded, and the corresponding relation between the point on the cylindrical model in the WCS coordinate system and the point on the spherical model in the CCS is established. For example, the height and width of the cylinder can be first determined from the maximum range of the horizontal viewing angle of the cylinder model; and then, corresponding a certain point of the image plane to a point of a spherical surface with the radius of R according to an equidistant imaging model, mapping the point to a cylindrical surface in an equal arc length manner, and establishing the relation between the inclination angle and the horizontal azimuth angle as well as the height and width positions of the expanded graph. Therefore, the one-to-one mapping relation between the image plane points and the cylindrical plane points can be established, so that the corresponding relation between the fisheye image and the cylindrical image is completed, and the color value of each point of the fisheye image is obtained according to a bilinear interpolation method to give a more accurate gray value to the point on the cylindrical surface.
The expanded wide view video stream may be played at a play interface providing multiple angles at the play device 40, for example, a play interface as shown in fig. 5 may be provided, which includes a play screen in the middle according to the aspect ratio of the expanded wide view image, for example, an expanded wide view screen 1, an expanded wide view screen 2, and an expanded wide view screen 3. And normal-viewing-angle screens 1 to 4 having a normal viewing angle aspect ratio on both sides of the unfolded wide-viewing-angle screen. The video source played by the common visual angle screen can be a video stream pushed by the common visual angle live broadcasting equipment arranged on the competition field or the common visual angle live broadcasting equipment arranged on the racing car.
For example, a video stream or an audio/video stream may be played on one of the expanded wide view screen and the expanded normal view screen, and a segment of the video stream in a time cycle is repeatedly played on the other screens, and when the segment played in the cycle is selected by a user through, for example, a click operation or a touch operation, the video stream or the audio/video stream stored by the corresponding live broadcast equipment is pulled from the memory of the live broadcast platform, or the video stream or the audio/video stream just obtained by the streaming media server from the live broadcast equipment is pulled from the streaming media server in real time.
The playing interface can also judge whether the screen needs to be switched from the expanded wide view angle screen to the common view angle screen or vice versa when receiving the operation of a user on the screen corresponding to a certain direct playing device, and if the switching is judged to be needed, the playing interface is replaced.
The implementation example of the present application includes:
1. a wide-viewing-angle live broadcasting system for events comprises
Wide viewing angle live broadcast equipment, comprising
The wide-view-angle shooting unit is used for acquiring a plurality of wide-view-angle image frames;
a memory for storing a first sequence of computer instructions;
a processor for executing the first sequence of computer instructions to:
obtaining an internal parameter matrix and a distortion coefficient matrix of the wide-view-angle camera unit;
sending the internal parameter matrix and the distortion coefficient matrix to a streaming media server;
encoding the plurality of wide view image frames into a generated video stream,
sending the video stream to a streaming media server;
the streaming media server caches and schedules the internal parameter matrix, the distortion coefficient matrix and the wide-view-angle image frame;
a playback apparatus includes
A memory for storing a second sequence of computer instructions;
a processor for executing the second sequence of computer instructions to:
receiving the internal parameter matrix and distortion coefficient matrix from at least one wide view camera unit and the wide view video stream from the streaming media server;
decoding the wide view video stream back into the plurality of wide view image frames;
correcting the wide-view image frame based on the internal parameter matrix and the distortion coefficient matrix to obtain an expanded wide-view image frame;
and playing the expanded wide-view image frame according to the generation time sequence of the wide-view image frame.
2. The event wide-view live broadcasting system according to example 1, characterized in that: the wide-view live broadcast equipment pushes the wide-view video stream to the streaming media server based on RTMP or RTSP protocol, or the wide-view live broadcast equipment transmits the wide-view video stream to the streaming media server based on RTMP/RTSP/HTTP based on the request of the streaming media server.
3. The event wide-view live broadcasting system according to example 1, characterized in that: the sports racing car sound field signal acquisition device further comprises a sound acquisition unit used for acquiring sound field signals of the live broadcasting equipment of the sports racing cars.
4. The event wide-view live broadcasting system according to example 3, characterized in that: the memory also stores a third sequence of computer instructions that the processor executes to
Digitizing the sound field signals into audio data;
encoding the audio data and the wide-view image to be packaged into a wide-view audio/video stream; and
and transmitting the wide-view audio and video stream to the streaming media server.
5. The event wide-view live broadcasting system according to example 1, characterized in that: the events include vehicle athletic events, and the event wide-view live broadcast device is arranged on a vehicle.
6. The event wide-view live broadcasting system according to example 1, characterized in that: the events comprise vehicle athletic events, and the event wide-view live broadcast equipment is arranged on the event field.
7. The event wide-view live broadcasting system according to example 1, characterized in that: the system also comprises at least one common visual angle live broadcast device, wherein the common visual angle live broadcast device generates a common visual angle video stream and transmits the common visual angle video stream to the streaming media server.
8. The event wide-view live broadcasting system according to example 7, characterized in that: and the streaming media system transmits the common visual angle video stream received from the common visual angle live broadcast equipment and the expanded wide visual angle video stream received from the wide visual angle live broadcast equipment to a live broadcast server in parallel.
9. A playback device for a wide-viewing-angle live broadcast system for events, includes
A memory for storing a sequence of computer instructions;
a processor for executing the sequence of computer instructions to:
receiving an internal parameter matrix and a distortion coefficient matrix from at least one wide-view camera unit from a streaming media server;
receiving a wide-view video stream from the wide-view camera unit from a streaming server;
decoding the wide view video stream to restore into a plurality of wide view image frames;
correcting the wide-view image frame based on the internal parameter matrix and the distortion coefficient matrix to obtain an expanded wide-view image frame;
and playing the expanded wide-view image frame according to the generation time sequence of the wide-view image frame.
10. A playback device for a wide-view live event system according to example 9, characterized in that: and the streaming media server transmits the wide-view video stream to the playing device based on RTMP/RTSP/HTTP protocol.
11. A live broadcast method for an event is characterized in that:
sending the measured internal parameters and distortion coefficients to a streaming media server;
generating a wide-view video stream or an audio/video stream by at least one event wide-view live broadcast device;
transmitting the wide-view video stream or the audio/video stream to the streaming media server;
generating a common visual angle video stream or an audio/video stream by at least one common live broadcast device;
transmitting the common visual angle video stream or the audio and video stream to the streaming media server;
the streaming media server transmits the wide-view video stream or the audio/video stream and the common-view video stream or the audio/video stream to a live broadcast server in parallel; and
and the live broadcast server synchronously or asynchronously provides the wide visual angle video stream or the audio and video stream and the common visual angle video stream or the audio and video stream to a playing device.
12. A live event broadcasting method according to example 11, characterized by: sending the measured intrinsic parameters and distortion coefficients to the streaming media server includes sending the measured intrinsic parameters and distortion coefficients to the streaming media server based on the RTMP metadata packet
13. A live broadcast method for an event is characterized in that:
sending the measured internal parameters and distortion coefficients to a streaming media server;
generating an expanded wide-view video stream or audio/video stream by at least one event wide-view live broadcast device;
transmitting the expanded wide-view video stream or audio/video stream to the streaming media server;
generating a common visual angle video stream or an audio/video stream by at least one common live broadcast device;
transmitting the common visual angle video stream or the audio and video stream to the streaming media server;
the streaming media server transmits the wide-view video stream or the audio/video stream and the common-view video stream or the audio/video stream to a live broadcast server in parallel; and
and the live broadcast server provides the wide visual angle video stream or audio and video stream and the common visual angle video stream or audio and video stream to the playing equipment according to the selected operation of the playing equipment.
14. A live event broadcasting method according to example 13, characterized by: sending the measured intrinsic parameters and distortion coefficients to the streaming server includes sending the measured intrinsic parameters and distortion coefficients to the streaming server via an RTMP-based metadata packet.
15. A computer readable storage medium having stored thereon sequences of instructions executable by a processor to perform the method of any of claims 11 to 14.
It should be understood by those skilled in the art that the above-mentioned function options and their corresponding live event applications are only examples for illustrating the present invention, and should not be construed as any limitation to the present invention, and other existing or future function options and their corresponding live event applications, such as may be applied to the present invention, should be included in the scope of the present invention.
It should be noted that the processing portions of the present invention may be implemented in software and/or a combination of software and hardware, for example, as an Application Specific Integrated Circuit (ASIC), a general purpose computer or any other similar hardware device. In one embodiment, the software program of the present invention may be executed by a processor to implement the steps or functions described above. Also, the software programs (including associated data structures) of the present invention can be stored in a computer readable recording medium, such as RAM memory, magnetic or optical drive or diskette and the like. Further, some of the steps or functions of the present invention may be implemented in hardware, for example, as circuitry that cooperates with the processor to perform various steps or functions.
In addition, some of the present invention can be applied as a computer program product, such as computer program instructions, which when executed by a computer, can invoke or provide the method and/or technical solution according to the present invention through the operation of the computer. Program instructions which invoke the methods of the present invention may be stored on a fixed or removable recording medium and/or transmitted via a data stream on a broadcast or other signal-bearing medium and/or stored within a working memory of a computer device operating in accordance with the program instructions. An embodiment according to the invention herein comprises an apparatus comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein the computer program instructions, when executed by the processor, trigger the apparatus to perform a method and/or solution according to embodiments of the invention as described above.
Claims (10)
1. A wide-viewing-angle live broadcasting system for events comprises
Wide viewing angle live broadcast equipment, comprising
The wide-view-angle shooting unit is used for acquiring a plurality of wide-view-angle image frames;
a memory for storing a first sequence of computer instructions;
a processor for executing the first sequence of computer instructions to:
obtaining an internal parameter matrix and a distortion coefficient matrix of the wide-view-angle camera unit;
sending the internal parameter matrix and the distortion coefficient matrix to a streaming media server;
encoding the plurality of wide view image frames into a generated video stream,
sending the video stream to a streaming media server;
the streaming media server caches and schedules the internal parameter matrix, the distortion coefficient matrix and the wide-view video stream;
a playback apparatus includes
A memory for storing a second sequence of computer instructions;
a processor for executing the second sequence of computer instructions to:
receiving the internal parameter matrix and distortion coefficient matrix from at least one wide view camera unit and the wide view video stream from the streaming media server;
decoding the wide view video stream back into the plurality of wide view image frames;
correcting the wide-view image frame based on the internal parameter matrix and the distortion coefficient matrix to obtain an expanded wide-view image frame;
and playing the expanded wide-view image frame according to the generation time sequence of the wide-view image frame.
2. A wide viewing angle live event system according to claim 1, wherein: the wide-view live broadcast equipment pushes the wide-view video stream to the streaming media server based on RTMP or RTSP protocol, or the wide-view live broadcast equipment transmits the wide-view video stream to the streaming media server based on RTMP/RTSP/HTTP based on the request of the streaming media server.
3. A wide viewing angle live event system according to claim 1, wherein: the wide-view-angle live broadcast equipment further comprises a sound acquisition unit used for acquiring sound field signals of the live broadcast equipment.
4. A wide viewing angle live event system according to claim 3, wherein: the memory of the wide-view live broadcast device further stores a third sequence of computer instructions that the processor executes to
Digitizing the sound field signals into audio data;
encoding the audio data and the wide-view image to be packaged into a wide-view audio/video stream; and
and transmitting the wide-view audio and video stream to the streaming media server.
5. A wide viewing angle live event system according to claim 1, wherein: the events include vehicle athletic events, and the wide-view live broadcast device is arranged on a vehicle.
6. A wide viewing angle live event system according to claim 1, wherein: the event comprises a vehicle athletic event, and the wide-view live broadcast device is arranged on the field of the event.
7. A wide viewing angle live event system according to claim 1, wherein: the system also comprises at least one common visual angle live broadcast device, wherein the common visual angle live broadcast device generates a common visual angle video stream and transmits the common visual angle video stream to the streaming media server.
8. A wide viewing angle live event system according to claim 7, wherein: and the streaming media server transmits the common visual angle video stream received from the common visual angle live broadcast equipment and the wide visual angle video stream received from the wide visual angle live broadcast equipment to a live broadcast server in parallel.
9. A playback device for a wide-viewing-angle live broadcast system for events, includes
A memory for storing a sequence of computer instructions;
a processor for executing the sequence of computer instructions to:
receiving an internal parameter matrix and a distortion coefficient matrix from at least one wide-view camera unit from a streaming media server;
receiving a wide-view video stream from the wide-view camera unit from a streaming server;
decoding the wide view video stream to restore into a plurality of wide view image frames;
correcting the wide-view image frame based on the internal parameter matrix and the distortion coefficient matrix to obtain an expanded wide-view image frame;
and playing the expanded wide-view image frame according to the generation time sequence of the wide-view image frame.
10. A playback device for a wide-view live event system according to claim 9, further comprising: and the streaming media server transmits the wide-view video stream to the playing device based on RTMP/RTSP/HTTP protocol.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710432123.6A CN107835433B (en) | 2017-06-09 | 2017-06-09 | Event wide-view live broadcasting system, associated equipment and live broadcasting method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710432123.6A CN107835433B (en) | 2017-06-09 | 2017-06-09 | Event wide-view live broadcasting system, associated equipment and live broadcasting method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107835433A CN107835433A (en) | 2018-03-23 |
| CN107835433B true CN107835433B (en) | 2020-11-20 |
Family
ID=61642987
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710432123.6A Active CN107835433B (en) | 2017-06-09 | 2017-06-09 | Event wide-view live broadcasting system, associated equipment and live broadcasting method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107835433B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113766266B (en) * | 2021-09-10 | 2024-02-13 | 阿波罗智联(北京)科技有限公司 | Audio and video processing method, device, equipment and storage medium |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| IL134881A (en) * | 1997-09-04 | 2004-05-12 | Discovery Communicat Inc | Apparatus for video access and control over computer network, including image correction |
| JP2004153750A (en) * | 2002-11-01 | 2004-05-27 | Matsushita Electric Ind Co Ltd | Image distribution system |
| US9185391B1 (en) * | 2014-06-17 | 2015-11-10 | Actality, Inc. | Adjustable parallax distance, wide field of view, stereoscopic imaging system |
| CN105847851A (en) * | 2016-04-19 | 2016-08-10 | 北京金山安全软件有限公司 | Panoramic video live broadcast method, device and system and video source control equipment |
| CN106101503A (en) * | 2016-07-18 | 2016-11-09 | 优势拓展（北京）科技有限公司 | Real time panoramic Living Network video camera and system and method |
| US9547883B1 (en) * | 2016-08-19 | 2017-01-17 | Intelligent Security Systems Corporation | Systems and methods for dewarping images |
| CN106327478A (en) * | 2016-08-31 | 2017-01-11 | 北京的卢深视科技有限公司 | Fisheye panoramic image synthesis processing method and device |
| CN106604042A (en) * | 2016-12-22 | 2017-04-26 | Tcl集团股份有限公司 | Panorama webcasting system and panorama webcasting method based on cloud server |
-
2017
- 2017-06-09 CN CN201710432123.6A patent/CN107835433B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN107835433A (en) | 2018-03-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210006770A1 (en) | Methods and apparatus for receiving and/or using reduced resolution images | |
| JP7045856B2 (en) | Video transmission based on independent coded background update | |
| WO2015174501A1 (en) | 360-degree video-distributing system, 360-degree video distribution method, image-processing device, and communications terminal device, as well as control method therefor and control program therefor | |
| US8639046B2 (en) | Method and system for scalable multi-user interactive visualization | |
| KR20190008901A (en) | Method, device, and computer program product for improving streaming of virtual reality media content | |
| CN107835435B (en) | Event wide-view live broadcasting equipment and associated live broadcasting system and method | |
| US20030086003A1 (en) | Video data processing apparatus and method, data distributing apparatus and method, data receiving apparatus and method, storage medium, and computer program | |
| JP2015220595A (en) | Perimeter motion picture distribution system, perimeter motion picture distribution method, communication terminal device, and method and program for controlling the same | |
| JP6002191B2 (en) | All-round video distribution system, all-round video distribution method, communication terminal device, and control method and control program thereof | |
| KR101528863B1 (en) | Method of synchronizing tiled image in a streaming service providing system of panoramic image | |
| CN107211081B (en) | Video transmission based on independently coded background updates | |
| CN112235585B (en) | Live broadcasting method, device and system for virtual scene | |
| JP2015173424A (en) | Video distribution system and video display device | |
| US11282169B2 (en) | Method and apparatus for processing and distributing live virtual reality content | |
| JP7177034B2 (en) | Method, apparatus and stream for formatting immersive video for legacy and immersive rendering devices | |
| CN113891117B (en) | Immersion medium data processing method, device, equipment and readable storage medium | |
| CN111669518A (en) | Multi-angle free visual angle interaction method and device, medium, terminal and equipment | |
| CN111669567A (en) | Multi-angle free visual angle video data generation method and device, medium and server | |
| CN111669561A (en) | Multi-angle free visual angle image data processing method and device, medium and equipment | |
| EP3621309A1 (en) | Transmission system for multi-channel image, control method therefor, and multi-channel image playback method and apparatus | |
| CN107835433B (en) | Event wide-view live broadcasting system, associated equipment and live broadcasting method | |
| CN107835434B (en) | Event wide-view live broadcasting equipment and associated live broadcasting system and method | |
| CN111669569A (en) | Video generation method and device, medium and terminal | |
| CN111669604A (en) | Acquisition equipment setting method and device, terminal, acquisition system and equipment | |
| US11706375B2 (en) | Apparatus and system for virtual camera configuration and selection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20211202 Address after: Room 501, gate 9, 9 / F, zhixueyuan, No. 29, Xierqi Road West, Haidian District, Beijing 100085 Patentee after: Zhi Xiaomu Address before: Room 1513, building 3, No. 3, Xijing Road, Badachu high tech park, Shijingshan District, Beijing 100041 Patentee before: FBLIFE (BEIJING) MEDIA TECHNOLOGY CO.,LTD. |
|
| TR01 | Transfer of patent right |