WO2018222974A1 - Procédé et appareil de transformation de multiples flux vidéo en un seul flux vidéo - Google Patents
Procédé et appareil de transformation de multiples flux vidéo en un seul flux vidéo Download PDFInfo
- Publication number
- WO2018222974A1 WO2018222974A1 PCT/US2018/035571 US2018035571W WO2018222974A1 WO 2018222974 A1 WO2018222974 A1 WO 2018222974A1 US 2018035571 W US2018035571 W US 2018035571W WO 2018222974 A1 WO2018222974 A1 WO 2018222974A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- video
- content view
- stream
- video stream
- client device
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 110
- 230000008569 process Effects 0.000 claims description 55
- 230000004044 response Effects 0.000 claims description 41
- 230000008859 change Effects 0.000 claims description 18
- 238000004519 manufacturing process Methods 0.000 description 11
- 238000004891 communication Methods 0.000 description 5
- 230000008901 benefit Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 210000003813 thumb Anatomy 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000010189 synthetic method Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
- H04N21/2365—Multiplexing of several video streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
- H04N21/4728—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
Definitions
- This disclosure relates to providing multiple views from multiple video streams in a single view based on a multi-view video stream.
- the methods, processes, and systems can comprise live capturing or receiving a plurality of monoscopic or stereoscopic video frame feeds; encoding and converting frame streams into video bunch streams or files, consisting of a plurality video frames in a plurality of resolution qualities; and streaming bunch frames to a viewer client device as determined by request from a viewer client device.
- Video content includes video and audio data.
- Video content from multiple sources may reflect different views of the same area of interest or different areas of interest.
- a consumer of video content from multiple sources may desire to select a view from multiple views and/or an area of interest from multiple areas of interest.
- the displayed video content shifts in response to the viewer selecting a desired view and/or selecting a desired area of interest.
- Current systems may require specialized equipment to enable the desired experience from multiple video sources. Such specialized equipment may include extensive, redundant hardware and/or specialized video processors.
- the methods, processes, and systems can comprise live capturing or receiving of a plurality of monoscopic or stereoscopic video frame feeds; encoding and converting the frame streams into video bunch frame streams or files, each video bunch frame stream including a plurality of video frames in a plurality of resolution qualities represented quantitatively in the form of a bitrate; and streaming bunch frames to a viewer client device as determined by a request from the viewer client device.
- a method of morphing multiple video streams into a single video stream includes: receiving multiple video streams at an encoding device, wherein each multiple video stream is formed by a plurality of video frames, wherein the multiple video streams are configured such that each multiple video stream provides a different content view than other streams of the multiple video streams; encoding the plurality of video frames for each multiple video stream at multiple bitrates to form multiple resolution qualities for each multiple video stream, wherein the multiple resolution qualities range from a highest resolution quality to a lowest resolution quality; and combining streams for the multiple resolution qualities of the multiple video streams to form a video bunch frame stream.
- an apparatus for morphing multiple video streams into a single video stream includes: an encoding device configured to receive multiple video streams, wherein each multiple video stream is formed by a plurality of video frames, wherein the multiple video streams are configured such that each multiple video stream provides a different content view than other streams of the multiple video streams; wherein the encoding device is configured to encode the plurality of video frames for each multiple video stream at multiple bitrates to form multiple resolution qualities for each multiple video stream, wherein the multiple resolution qualities range from a highest resolution quality to a lowest resolution quality; wherein the encoding device is configured to combine streams for the multiple resolution qualities of the multiple video streams to form a video bunch frame stream.
- a non-transitory computer-readable medium storing program instructions that, when executed by at least one computer or processor, cause a computer- controlled system to perform a method of morphing multiple video streams into a single video stream.
- the method includes: receiving multiple video streams at an encoding device, wherein each multiple video stream is formed by a plurality of video frames, wherein the multiple video streams are configured such that each multiple video stream provides a different content view than other streams of the multiple video streams; encoding the plurality of video frames for each multiple video stream at multiple bitrates to form multiple resolution qualities for each multiple video stream, wherein the multiple resolution qualities range from a highest resolution quality to a lowest resolution quality; and combining streams for the multiple resolution qualities of the multiple video streams to form a video bunch frame stream.
- FIG. 1 illustrates an exemplary hardware and software environment for the methods, processes, and systems for streaming monoscopic or stereoscopic videos simultaneously from multiple sources.
- FIG. 2 illustrates an exemplary method, process, and system for streaming monoscopic or stereoscopic videos utilizing camera feeds and manual switching between cameras video feeds at the production level.
- FIG. 3 illustrates an exemplary method, process, and system for streaming monoscopic or stereoscopic videos utilizing multiple camera sets, each with a designated production station, and manual switching between video feeds at the playback level.
- FIG. 4 illustrates an exemplary method, process, and system for uploading and streaming previously produced monoscopic or stereoscopic video files.
- FIG. 5 is a flowchart of an exemplary method and process for multi encoding a full monoscopic or stereoscopic video frame, thereby converting the video frame into a bunch frame set.
- FIG. 6 is a flowchart of an exemplary method and process for identifying, streaming, and displaying monoscopic or stereoscopic video bunch frames, consisting of Intra Frames (Key Frames) and Predicted Frames (Delta Frames), as determined by the request from the viewer client device and displayed separately in a viewer client device.
- Key Frames Intra Frames
- Delta Frames Predicted Frames
- FIG. 7 is a flowchart of an exemplary method, process, and system for streaming and displaying monoscopic or stereoscopic video bunch frames when M levels of qualities are utilized.
- FIG. 8 is a flowchart of an exemplary method, process, and system for identifying and streaming a monoscopic or stereoscopic video bunch frame set, consisting of a particular combination of Intra Frames (Key Frames) and Predicted Frames (Delta Frames), as determined by the Region of Interest selected by the viewer client device.
- Key Frames Intra Frames
- Delta Frames Predicted Frames
- FIG. 9 is a flow chart of an exemplary embodiment of a process for morphing multiple video streams into a single video stream.
- FIG. 10, in combination with FIG. 9, provide a flow chart of another exemplary embodiment of a process for morphing multiple video streams into a single video stream.
- FIG. 11 in combination with FIG. 9, provide a flow chart of yet another exemplary embodiment of a process for morphing multiple video streams into a single video stream.
- FIG. 12, in combination with FIG. 9, provide a flow chart of still another exemplary embodiment of a process for morphing multiple video streams into a single video stream.
- FIG. 13A and 13B in combination with FIG. 9, provide a flow chart of still yet another exemplary embodiment of a process for morphing multiple video streams into a single video stream.
- FIG. 14A and 14B in combination with FIG. 9, provide a flow chart of another exemplary embodiment of a process for morphing multiple video streams into a single video stream.
- FIGs. 15A and 15B in combination with FIG. 9, provide a flow chart of yet another exemplary embodiment of a process for morphing multiple video streams into a single video stream.
- FIG. 16 is a block diagram of an exemplary embodiment of a system configured to morph multiple video streams into a single video stream.
- the present invention is directed to methods, processes, and systems for capturing or uploading, encoding, and streaming monoscopic and stereoscopic multiple video streams morphed into single view for presentation.
- the methods, processes, and systems can comprise live capturing or receiving a plurality of monoscopic or stereoscopic video frame feeds; encoding and converting frame streams into a video bunch streams or files, consisting of a plurality of video frames in a plurality of resolution qualities; and streaming bunch frames to a viewer client device as determined by request from viewer client device.
- FIG. 1 illustrates an exemplary hardware and software environment implemented in accordance with some embodiments of the methods, processes, and systems for streaming monoscopic or stereoscopic videos simultaneously from multiple sources.
- the exemplary hardware and software environment includes a camera system 101 ; a production station 102, consisting of a video capture module 103, an encoding module 105, and a broadcasting module 106; a storage device assembly 107; a streaming server 108; a communications network 109, and a viewer client device 110.
- the network connecting the camera system 101 , production station 102, storage device assembly 107, streaming server 108, communications network 109, and the viewer client device 110 may be partially or entirely interconnected via a network comprised of, for example, a local area network (LAN), a wireless LAN (WLAN), a metropolitan area network (MAN), a universal serial bus (USB), a hybrid fiber-coaxial (HFC) network, a satellite network, an Ethernet network, a wireless MAN (WMAN), a wide area network (WAN), a wireless WAN (WWAN), a personal area network (PAN), a wireless PAN (WPAN), the internet, and/or wireless standards such as, for example, IEEE 802.11 standards, WiFi, Bluetooth, infrared, WiMax, Long Term Evolution (LTE), or any suitable communication network in any suitable combination.
- the exemplary hardware and software environment formed by these components may also include any other suitable elements to facilitate capturing or uploading, encoding, and streaming monoscopic and stereoscopic videos for viewing.
- the camera system 101 consists of individual cameras numbering 1 to N and supporting monoscopic or stereoscopic video, depending on the specific implementation of the present invention. Each camera sends a video feed to the video capture module 103.
- the production station 102 comprised of a video capture module 103, an encoding module 105, and a broadcasting module 106, may be one computing device or each module may be implemented on separate computing devices.
- the video capture module 103 is comprised of a plurality of internal or external video capture devices. A video capture device is needed to capture video from each camera. The number of video capture devices is dependent on the number of inputs per video capture device and the number of cameras. The video capture module captures and temporarily stores video feeds from the camera system 101 before transmission to the encoding module 105.
- the encoding module 105 consists of a plurality of computer CPUs and/or GPUs that converts the monoscopic or stereoscopic video stream, received from the video capture device(s) 103, into a video bunch frame streams or files.
- the encoding module produces the video bunch frame stream by encoding each piece in a plurality of resolution qualities.
- the encoding module may utilize any suitable codec, including any current or future standard codec (e.g., H.264, H.265, etc.).
- the storage device assembly 107 is comprised of a plurality of storage devices that can optionally receive and save a video bunch frame streams from the encoding module 105.
- the plurality of storage devices may include internal hard drives, external hard drives, solid state drives (SSDs), VRAMs, DRAMs, network attached storage (NAS) devices, USB thumb drives, flash drives, memory cards, optical drives (e.g., CD drives, DVD drives, etc.), cloud storage (e.g., Amazon S3, Google Drive, Sky Drive, etc.), and other suitable storage devices in any suitable combination.
- the broadcasting module 106 receives and temporarily stores a video bunch frame streams from the encoding module 105. Based on request from the viewer client device 110 the broadcasting module sends pieces of the video bunch frame streams to the streaming server 108.
- the streaming server 108 is comprised of a plurality of streaming servers that receive video bunch frame streams from the broadcasting module 106 or read from the storage device assembly 107 based on request by the viewer client device 110.
- the streaming server assembly sends the video bunch frame streams, as request by viewer, to the viewer client device via the communications network 109.
- the viewer client device 110 communicates to the streaming server 108, and displays the received video bunch frame streams from the streaming server 108 via the communications network 109.
- the viewer client device 110 operates on a plurality of viewer devices with each viewer device communicating and receiving a unique set of pieces of the video bunch frame streams from the streaming server 108.
- the viewer client device 110 can operate on any user device capable of receipt and playback of monoscopic or stereoscopic video, e.g., personal computing device, smartphone, virtual reality headset, etc.
- FIG. 2 illustrates an exemplary method, process, and system for streaming monoscopic or stereoscopic videos utilizing camera feeds and manual switching between camera video feeds at the production level.
- FIG. 2 shows a further embodiment of FIG. 1 , in which the camera system 112 is comprised of a plurality of camera sets, each consisting of individual cameras numbering 1 to N and supporting the capture of monoscopic or stereoscopic video.
- each camera set sends video streams to a producer subsystem 113.
- the producer subsystem allows for the manual selection of a single camera set. Only the camera set actively selected by the producer subsystem sends the video feeds for the cameras comprising the selected camera set to the video capture module for processing and eventual playback in the viewer client device.
- FIG. 3 illustrates an exemplary method, process, and system for streaming monoscopic or stereoscopic videos utilizing multiple camera sets, each with a designated production station 114, and manual switching between video feeds 115 at the playback level.
- FIG. 3 shows a further embodiment of FIG. 1 , in which a plurality of camera sets each have a dedicated production station 114. In this embodiment, each camera set and dedicated production station produces a unique video bunch frame stream.
- the viewer client device 117 communicates a manual camera set selection to the streaming server 116, which switches between the unique video bunch frame streams 115 and sends the video bunch frame stream from the selected camera set to the viewer client device 117 for playback.
- FIG. 4 illustrates an exemplary method, process, and system for uploading and streaming previously produced monoscopic or stereoscopic video files.
- FIG. 4 shows a further embodiment of the production station in FIG. 1 , in which the production station 122 is comprised of a file upload module 123, and an encoding module 124.
- a previously produced monoscopic or stereoscopic video file is uploaded to the file upload module 123, eliminating the need for a camera system and video capture module.
- the encoding module 124 consists of a plurality of computer CPUs and/or GPUs that converts the monoscopic or stereoscopic video stream, received from the file upload module 123, into a video bunch frame stream or file.
- the encoding module produces the video bunch frame stream by encoding each video in a plurality of resolution qualities.
- the encoding module may utilize any suitable codec, including any current or future standard codec (e.g., H.264, H.265, etc.).
- the storage device assembly 125 is comprised of a plurality of storage devices that can optionally receive and save a video bunch frame stream from the encoding module 124.
- the plurality of storage devices may include internal hard drives, external hard drives, solid state drives (SSDs), VRAMs, DRAMs, network attached storage (NAS) devices, USB thumb drives, flash drives, memory cards, optical drives (e.g., CD drives, DVD drives, etc.), cloud storage (e.g., Amazon S3, Google Drive, Sky Drive, etc.), and other suitable storage devices in any suitable combination.
- the streaming server 126 is comprised of a plurality of streaming servers that read pieces of the video bunch frame stream read from the storage device assembly 125 based on request from the viewer client device 127. The streaming server sends the video bunch frame stream based on viewer client device 127 request.
- FIG. 5 is a flowchart of an exemplary method and process for multi encoding a full monoscopic or stereoscopic video frame, thereby converting the video frame into a bunch frame set.
- FIG. 5 shows an embodiment of the method and process utilized by an encoding module 200.
- Each video capture device being associated with a corresponding camera of a camera set.
- the encoding module 200 encodes the full frame into a plurality of resolution qualities [(QL1) ... (QLM)] frames 203, the number of which are variable based on the desired configuration.
- After encoding 203 the plurality of resolution qualities of each frame is combined into a video bunch frame set 206.
- the bunch frame set consists of two types of frames, Intra frames (key frames) and Predicted frames (delta frames).
- the video bunch frame stream consists of a combination of Intra and Predicted frames (IPPPIPPPI).
- FIG. 6 is a flowchart of an exemplary method and process for identifying, streaming, and displaying monoscopic or stereoscopic video bunch frames, consisting of Intra Frames (Key Frames) and Predicted Frames (Delta Frames), as determined by request from the viewer client device and displayed separately in a viewer client device.
- monoscopic or stereoscopic video bunch frames are streamed from a storage device 300 or a broadcast module 301.
- the memory and broadcast modules contain a videos bunch frame set, which together includes the plurality of resolution qualities for each video source frame.
- the video bunch stream consists of a combination of Intra and Predicted frames (IPPPIPPPI).
- the streaming server 302 receives the video bunch frame stream from the memory 300 and broadcast 301 modules.
- the viewer client device 304 connects to the streaming server 302 and, after the connection is established, sends request for quality combination to the streaming server.
- Quality combination based specific implementation of the request in the viewer client device 303.
- the streaming server selects and stream a unique set of streams from each source with the necessary resolution qualities, which is sent to the viewer client device for display 304. This process results in high quality frames being displayed separately in larger area from low quality frames in the viewer client device 303.
- the streaming server adjusts quality of the each streamed source in real time.
- the frames from each source are displayed in the viewer client device based on application.
- the frames can be positioned in very combinations (305, 306).
- FIG. 7 is a flowchart of an exemplary method, process, and system for streaming and displaying monoscopic or stereoscopic video bunch frames from video sources when (QL1)...(QLM) quality levels of bunch frames are utilized.
- FIG. 7 shows another embodiment of FIG. 6, that illustrates the video bunch frame stream containing a plurality of resolution qualities for each frame source.
- FIG. 8 is a flowchart of an exemplary method, process, and system for identifying and streaming a monoscopic or stereoscopic video bunch frame sets, consisting of a particular combination of Intra Frames (Key Frames) and Predicted Frames (Delta Frames), as determined by request from the viewer client device.
- the current embodiment explains the process accounting for changes in the resolution qualities of frame sources comprising the unique video bunch frame streams for the viewer client device, i.e.
- the composition of the frames comprising the unique video bunch frame streams changes based on new Request and on the current Intra frame so that high quality frame pieces are displayed in the viewer client device.
- the streaming server receives new frame bunch data from the broadcast module 400.
- the streaming server checks to see if an Intra (I) frame is received 401.
- the streaming server skips new frame data until an Intra frame is received 401. If an Intra frame is received and the Request from the viewer client device did not change, then the frame quality output does not change 402. If an Intra frame is received and the Request received from the viewer client device have changed, then the frame quality output is changed by the streaming server 403.
- Each frame in the video frame bunch is checked 404. If a frame is corresponded to high resolution request, the video bunch frame stream output to the viewer client device is the high resolution version of the frame 405. If a frame is corresponded to low resolution request, the video bunch frame stream output to the viewer client device is the low resolution version of the frame 406.
- an exemplary embodiment of a process 900 for morphing multiple video streams into a single video stream begins at 902 where multiple video streams are received at an encoding device.
- Each multiple video stream is formed by a plurality of video frames.
- the multiple video streams are configured such that each multiple video stream provides a different content view than other streams of the multiple video streams.
- the plurality of video frames for each multiple video stream are encoded at multiple bitrates to form multiple resolution qualities for each multiple video stream.
- the multiple resolution qualities range from a highest resolution quality to a lowest resolution quality.
- streams for the multiple resolution qualities of the multiple video streams are combined to form a video bunch frame stream (906).
- each multiple video stream is defined by a predetermined configuration such that the predetermined configuration is consistent from frame-to-frame.
- the predetermined configuration for each multiple video stream includes shape and size parameters for the corresponding content view in a manner that permits different multiple video streams to have different shapes and different sizes.
- the multiple bitrates for each multiple video stream include at least three bitrates such that the multiple resolution qualities for each multiple video stream include a high resolution quality, at least one intermediate resolution quality, and a low resolution quality.
- an embodiment of a process 1000 for morphing multiple video streams into a single video stream begins at 1002 where multiple video feeds from a plurality of video cameras are captured at one or more video capture devices.
- the plurality of video cameras are configured such that each multiple video feed provides a different content view than other feeds of the multiple video feeds.
- the captured multiple video feeds are sent as the multiple video streams from the one or more video capture devices to the encoding device.
- an embodiment of a process 1100 for morphing multiple video streams into a single video stream begins at 1102 where one or more electronic files associated with multiple video feeds are uploaded from a storage device to a file upload device.
- the multiple video feeds are configured such that each multiple video feed provides a different content view than other feeds of the multiple video feeds.
- the one or more electronic files are processed to form the multiple video streams.
- the multiple video streams are sent from the file upload device to the encoding device (1106).
- an embodiment of a process 1200 for morphing multiple video streams into a single video stream includes the process 900 of FIG. 9 and continues from 906 to 1202 where the video bunch frame stream is sent from the encoding device to a broadcasting device.
- the video bunch frame stream is at least temporarily stored in a storage device accessible to the broadcasting device.
- identifying information for a first content view is received from a first viewer client device at the broadcasting device (1206).
- the first content view is representative of a corresponding first video stream of the multiple video streams.
- identifying information for a second content view is received from the first viewer client device at the broadcasting device.
- the second content view is different from the first content view and representative of a corresponding second video stream of the multiple video streams.
- a higher resolution quality in the video bunch frame stream is selected for the first video stream associated with the first content view (1210).
- a lower resolution quality in the video bunch frame stream is selected for the second video stream associated with the second content view.
- the first video stream at the higher resolution quality and the second video stream at the lower resolution quality are combined to form a first customized multi-view video stream (1214).
- the first customized multi-view video stream is sent from the broadcasting device to the first viewer client device.
- the process 1200 also includes receiving identifying information for a first preferred content view from a second viewer client device at the broadcasting device.
- the first preferred content view is different from the first content view selected by the first viewer client device and representative of a corresponding first preferred video stream of the multiple video streams.
- Identifying information for a second preferred content view is received from the second viewer client device at the broadcasting device.
- the second preferred content view is different from the first preferred content view and representative of a corresponding second preferred video stream of the multiple video streams.
- a higher resolution quality in the video bunch frame stream is selected for the first preferred video stream associated with the first preferred content view.
- a lower resolution quality in the video bunch frame stream is selected for the second preferred video stream associated with the second preferred content view.
- the first preferred video stream at the higher resolution quality and the second preferred video stream at the lower resolution quality are combined to form a second customized multi-view video stream.
- the second customized multi-view video stream is sent from the broadcasting device to the second viewer client device.
- an embodiment of a process 1300 for morphing multiple video streams into a single video stream includes the process 900 of FIG. 9 and continues from 906 to 1302 where the video bunch frame stream, content view information associated with the multiple video streams, and resolution quality information associated with the video bunch frame stream are at least temporarily stored in a storage device accessible to the encoding device.
- the content view and resolution quality information are uploaded from the storage device to a streaming server assembly.
- identifying information for a first content view is received from a first viewer client device at the streaming server assembly (1306).
- the first content view is representative of a corresponding first video stream of the multiple video streams.
- identifying information for a second content view is received from the first viewer client device at the streaming server assembly.
- the second content view is different from the first content view and representative of a corresponding second video stream of the multiple video streams.
- a higher resolution quality in the video bunch frame stream is uploaded from the storage device to the streaming server assembly for the first video stream associated with the first content view (1310).
- a lower resolution quality in the video bunch frame stream is uploaded from the storage device to the streaming server assembly for the second video stream associated with the second content view.
- the first video stream at the higher resolution quality and the second video stream at the lower resolution quality are combined to form a first customized multi-view video stream (1314).
- the first customized multi-view video stream is sent from the streaming server assembly to the first viewer client device.
- the process 1300 also includes receiving identifying information for a first preferred content view from a second viewer client device at the streaming server assembly.
- the first preferred content view is different from the first content view selected by the first viewer client device and representative of a corresponding first preferred video stream of the multiple video streams.
- Identifying information for a second preferred content view is received from the second viewer client device at the streaming server assembly.
- the second preferred content view is different from the first preferred content view and representative of a corresponding second preferred video stream of the multiple video streams.
- a higher resolution quality in the video bunch frame stream is uploaded from the storage device to the streaming server assembly for the first preferred video stream associated with the first preferred content view.
- a lower resolution quality in the video bunch frame stream is uploaded from the storage device to the streaming server assembly for the second preferred video stream associated with the second preferred content view.
- the first preferred video stream at the higher resolution quality and the second preferred video stream at the lower resolution quality are combined to form a second customized multi-view video stream.
- the second customized multi-view video stream is sent from the streaming server assembly to the second viewer client device.
- an embodiment of a process 1400 for morphing multiple video streams into a single video stream includes the process 900 of FIG. 9 and continues from 906 to 1402 where the video bunch frame stream is sent from the encoding device to a broadcasting device.
- the video bunch frame stream is at least temporarily storing in a storage device accessible to the broadcasting device.
- identifying information for a first content view is periodically received from a viewer client device at the broadcasting device at least each time the first content view changes (1406).
- the viewer client device is configured to permit the first content view to dynamically change in response to user actions at the viewer client device to select which actual content view of the multiple video streams is to be displayed on the viewer client device at a higher resolution quality.
- the first content view is representative of a corresponding first video stream of the multiple video streams.
- identifying information for a second content view is periodically received from the viewer client device at the broadcasting device at least each time the second content view changes.
- the viewer client device is configured to permit the second content view to dynamically change in response to user actions at the viewer client device to select which actual content view of the multiple video streams is to be displayed on the viewer client device at a lower resolution quality.
- the second content view is different from the first content view and representative of a corresponding second video stream of the multiple video streams.
- the higher resolution quality in the video bunch frame stream is dynamically selected for the first video stream associated with the first content view in response to dynamic changes to the first content view (1410).
- the lower resolution quality in the video bunch frame stream is dynamically selected for the second video stream associated with the second content view in response to dynamic changes to the second content view.
- the first video stream at the higher resolution quality and the second video stream at the lower resolution quality are dynamically combined to form a customized multi-view video stream that is dynamically adjusted in response to dynamic changes to at least one of the first and second content views (1414).
- the customized multi-view video stream is sent from the broadcasting device to the viewer client device.
- an embodiment of a process 1500 for morphing multiple video streams into a single video stream includes the process 900 of FIG. 9 and continues from 906 to 1502 where the video bunch frame stream, content view information associated with the multiple video streams, and resolution quality information associated with the video bunch frame stream are at least temporarily stored in a storage device accessible to the encoding device.
- the content view and resolution quality information are uploaded from the storage device to a streaming server assembly.
- identifying information for a first content view is periodically received from a viewer client device at the streaming server assembly at least each time the first content view changes (1506).
- the viewer client device is configured to permit the first content view to dynamically change in response to user actions at the viewer client device to select which actual content view of the multiple video streams is to be displayed on the viewer client device at a higher resolution quality.
- the first content view is representative of a corresponding first video stream of the multiple video streams.
- identifying information for a second content view is periodically received from the viewer client device at the streaming server assembly at least each time the second content view changes.
- the viewer client device is configured to permit the second content view to dynamically change in response to user actions at the viewer client device to select which actual content view of the multiple video streams is to be displayed on the viewer client device at a lower resolution quality.
- the second content view is different from the first content view and representative of a corresponding second video stream of the multiple video streams.
- the higher resolution quality in the video bunch frame stream is dynamically uploaded from the storage device to the streaming server assembly for the first video stream associated with the first content view in response to dynamic changes to the first content view (1510).
- the lower resolution quality in the video bunch frame stream is dynamically uploaded from the storage device to the streaming server assembly for the second video stream associated with the second content view in response to dynamic changes to the second content view.
- the first video stream at the higher resolution quality and the second video stream at the lower resolution quality are dynamically combined to form a customized multi-view video stream that is dynamically adjusted in response to dynamic changes to at least one of the first and second content views (1514).
- the customized multi-view video stream is sent from the streaming server assembly to the viewer client device.
- a system 1600 for morphing multiple video streams into a single video stream includes an encoding device 1602 configured to receive multiple video streams.
- Each multiple video stream is formed by a plurality of video frames.
- the multiple video streams are configured such that each multiple video stream provides a different content view than other streams of the multiple video streams.
- the encoding device 1602 is configured to encode the plurality of video frames for each multiple video stream at multiple bitrates to form multiple resolution qualities for each multiple video stream.
- the multiple resolution qualities range from a highest resolution quality to a lowest resolution quality.
- the encoding device 1602 is configured to combine streams for the multiple resolution qualities of the multiple video streams to form a video bunch frame stream.
- the system 1600 also includes one or more video capture devices 1604 configured to capture multiple video feeds from a plurality of video cameras 1606.
- the plurality of video cameras 1606 are configured such that each multiple video feed provides a different content view than other feeds of the multiple video feeds.
- the one or more video capture devices 1604 are configured to send the captured multiple video feeds as the multiple video streams to the encoding device 1602.
- the system 1600 also includes a file upload device 1608 configured to upload one or more electronic files associated with multiple video feeds from a storage device 1610.
- the multiple video feeds are configured such that each multiple video feed provides a different content view than other feeds of the multiple video feeds.
- the file upload device 1608 is configured to process the one or more electronic files to form the multiple video streams.
- the file upload device 1608 is configured to send the multiple video streams to the encoding device 1602.
- the system 1600 also includes a broadcasting device 1612.
- the encoding device 1602 is configured to send the video bunch frame stream to the broadcasting device 1612.
- the broadcasting device 1612 is configured to at least temporarily store the video bunch frame stream in a storage device 1614.
- the broadcasting device 1612 is configured to receive identifying information for a first content view from a first viewer client device 1616.
- the first content view is representative of a corresponding first video stream of the multiple video streams.
- the broadcasting device 1612 is configured to receive identifying information for a second content view from the first viewer client device 1616.
- the second content view is different from the first content view and representative of a corresponding second video stream of the multiple video streams.
- the broadcasting device 1612 is configured to select a higher resolution quality in the video bunch frame stream for the first video stream associated with the first content view.
- the broadcasting device 1612 is configured to select a lower resolution quality in the video bunch frame stream for the second video stream associated with the second content view.
- the broadcasting device 1612 is configured to combine the first video stream at the higher resolution quality and the second video stream at the lower resolution quality to form a first customized multi-view video stream.
- the broadcasting device 1612 is configured to send the first customized multi-view video stream to the first viewer client device 1616.
- the system 1600 also includes a streaming server assembly 1618.
- the encoding device 1602 is configured to at least temporarily store the video bunch frame stream, content view information associated with the multiple video streams, and resolution quality information associated with the video bunch frame stream in a storage device 1620.
- the streaming server assembly 1618 is configured to upload the content view and resolution quality information from the storage device 1620.
- the streaming server assembly 1618 is configured to receive identifying information for a first content view from a first viewer client device 1622.
- the first content view is representative of a corresponding first video stream of the multiple video streams.
- the streaming server assembly 1618 is configured to receive identifying information for a second content view from the first viewer client device 1622.
- the second content view is different from the first content view and representative of a corresponding second video stream of the multiple video streams.
- the streaming server assembly 1618 is configured to upload a higher resolution quality in the video bunch frame stream from the storage device 1620 for the first video stream associated with the first content view.
- the streaming server assembly 1618 is configured to upload a lower resolution quality in the video bunch frame stream from the storage device 1620 for the second video stream associated with the second content view.
- the streaming server assembly 1618 is configured to combine the first video stream at the higher resolution quality and the second video stream at the lower resolution quality to form a first customized multi-view video stream.
- the streaming server assembly 1618 is configured to send the first customized multi-view video stream to the first viewer client device 1622.
- the system 1600 also includes a broadcasting device 1612.
- the encoding device 1602 is configured to send the video bunch frame stream to the broadcasting device 1612.
- the broadcasting device 1612 is configured to at least temporarily store the video bunch frame stream in a storage device 1614.
- the broadcasting device 1612 is configured to periodically receive identifying information for a first content view from a viewer client device 1616 at least each time the first content view changes.
- the viewer client device 1616 is configured to permit the first content view to dynamically change in response to user actions at the viewer client device 1616 to select which actual content view of the multiple video streams is to be displayed on the viewer client device 1616 at a higher resolution quality.
- the first content view is representative of a corresponding first video stream of the multiple video streams.
- the broadcasting device 1612 is configured to periodically receive identifying information for a second content view from the viewer client device 1616 at least each time the second content view changes.
- the viewer client device 1616 is configured to permit the second content view to dynamically change in response to user actions at the viewer client device 1616 to select which actual content view of the multiple video streams is to be displayed on the viewer client device 1616 at a lower resolution quality.
- the second content view is different from the first content view and representative of a corresponding second video stream of the multiple video streams.
- the broadcasting device 1612 is configured to dynamically select the higher resolution quality in the video bunch frame stream for the first video stream associated with the first content view in response to dynamic changes to the first content view.
- the broadcasting device 1612 is configured to dynamically select the lower resolution quality in the video bunch frame stream for the second video stream associated with the second content view in response to dynamic changes to the second content view.
- the broadcasting device 1612 is configured to dynamically combine the first video stream at the higher resolution quality and the second video stream at the lower resolution quality to form a customized multi-view video stream that is dynamically adjusted in response to dynamic changes to at least one of the first and second content views.
- the broadcasting device 1612 is configured to send the customized multi-view video stream to the viewer client device 1616.
- the system 1600 also includes a streaming server assembly 1618.
- the encoding device 1602 is configured to at least temporarily store the video bunch frame stream, content view information associated with the multiple video streams, and resolution quality information associated with the video bunch frame stream in a storage device 1620.
- the streaming server assembly 1618 is configured to upload the content view and resolution quality information from the storage device 1620.
- the streaming server assembly 1618 is configured to periodically receive identifying information for a first content view from a viewer client device 1622 at least each time the first content view changes.
- the viewer client device 1622 is configured to permit the first content view to dynamically change in response to user actions at the viewer client device 1622 to select which actual content view of the multiple video streams is to be displayed on the viewer client device 1622 at a higher resolution quality.
- the first content view is representative of a corresponding first video stream of the multiple video streams.
- the streaming server assembly 1618 is configured to periodically receive identifying information for a second content view from the viewer client device 1622 at least each time the second content view changes.
- the viewer client device 1622 is configured to permit the second content view to dynamically change in response to user actions at the viewer client device 1622 to select which actual content view of the multiple video streams is to be displayed on the viewer client device 1622 at a lower resolution quality.
- the second content view is different from the first content view and representative of a corresponding second video stream of the multiple video streams.
- the streaming server assembly 1618 is configured to dynamically upload the higher resolution quality in the video bunch frame stream from the storage device 1620 for the first video stream associated with the first content view in response to dynamic changes to the first content view.
- the streaming server assembly 1618 is configured to dynamically upload the lower resolution quality in the video bunch frame stream from the storage device 1620 for the second video stream associated with the second content view in response to dynamic changes to the second content view.
- the streaming server assembly 1618 is configured to dynamically combine the first video stream at the higher resolution quality and the second video stream at the lower resolution quality to form a customized multi-view video stream that is dynamically adjusted in response to dynamic changes to at least one of the first and second content views.
- the streaming server assembly 1618 is configured to send the customized multi-view video stream to the viewer client device 1622.
- FIGs. 9-16 various exemplary embodiments of non- transitory computer-readable medium storing program instructions that, when executed by at least one computer or processor cause a corresponding processor-controlled system (e.g., system 1600) to perform a method of morphing multiple video streams into a single video stream.
- a processor-controlled system e.g., system 1600
- FIG. 16 various embodiments of the processor-controlled system are described above with reference to FIG. 16.
- FIGs. 9-12, 13A, 13B, 14A, 14B, 15A, and 15B are described above with reference to FIGs. 9-12, 13A, 13B, 14A, 14B, 15A, and 15B.
- the program instructions of the various exemplary embodiments of non-transitory computer-readable medium are defined by any suitable combination of the processes 900, 1000, 1 100, 1200, 1300, 1400, and 1500 described above with reference to FIGs. 9-12, 13A, 13B, 14A, 14B, 15A, and 15B.
- the processor-controlled system 1600 associated with the various exemplary embodiments of non-transitory computer-readable medium are defined by any suitable combination of components described above with reference to FIG. 16.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Human Computer Interaction (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
L'invention concerne un procédé de transformation de multiples flux vidéo en un seul flux vidéo. Selon un mode de réalisation, le procédé consiste à recevoir des flux vidéo au niveau d'un dispositif de codage, chaque flux vidéo multiple étant formé par des trames vidéo; à coder les trames vidéo pour chaque flux vidéo à de multiples débits binaires pour former de multiples qualités de résolution pour chaque flux vidéo; et à combiner les flux pour les multiples qualités de résolution des flux vidéo afin de former un flux de trames de grappes vidéo. L'invention concerne également un appareil de transformation de multiples flux vidéo en un seul flux vidéo. L'invention concerne également un support non transitoire lisible par ordinateur mémorisant des instructions de programme. Les instructions du programme, lorsqu'elles sont exécutées par un ordinateur ou un processeur, amènent l'appareil à mettre en œuvre le procédé.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/618,726 US11388455B2 (en) | 2016-06-02 | 2018-06-01 | Method and apparatus for morphing multiple video streams into single video stream |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201762513826P | 2017-06-01 | 2017-06-01 | |
PCT/US2017/035478 WO2017210444A1 (fr) | 2016-06-02 | 2017-06-01 | Procédé et appareil de diffusion en continu de vidéo panoramique |
US62/513,826 | 2017-06-01 | ||
USPCT/US2017/035478 | 2017-06-01 |
Related Parent Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2017/035478 Continuation-In-Part WO2017210444A1 (fr) | 2016-06-02 | 2017-06-01 | Procédé et appareil de diffusion en continu de vidéo panoramique |
US16/306,065 Continuation-In-Part US11153615B2 (en) | 2016-06-02 | 2017-06-01 | Method and apparatus for streaming panoramic video |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2018222974A1 true WO2018222974A1 (fr) | 2018-12-06 |
Family
ID=64456129
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2018/035571 WO2018222974A1 (fr) | 2016-06-02 | 2018-06-01 | Procédé et appareil de transformation de multiples flux vidéo en un seul flux vidéo |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2018222974A1 (fr) |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
FR2927758A1 (fr) * | 2008-02-15 | 2009-08-21 | Ateme Sa Sa | Procede et dispositif de codage-decodage d'images video successives selon un flux video principal en pleine resolution et selon un flux video secondaire en qualite reduite |
EP2555517A1 (fr) * | 2011-08-01 | 2013-02-06 | Hunt Electronic Co., Ltd. | Serveur vidéo de réseau et son procédé de contrôle vidéo |
US20150201197A1 (en) * | 2014-01-15 | 2015-07-16 | Avigilon Corporation | Streaming multiple encodings with virtual stream identifiers |
US20150199366A1 (en) * | 2014-01-15 | 2015-07-16 | Avigilon Corporation | Storage management of data streamed from a video source device |
-
2018
- 2018-06-01 WO PCT/US2018/035571 patent/WO2018222974A1/fr active Application Filing
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
FR2927758A1 (fr) * | 2008-02-15 | 2009-08-21 | Ateme Sa Sa | Procede et dispositif de codage-decodage d'images video successives selon un flux video principal en pleine resolution et selon un flux video secondaire en qualite reduite |
EP2555517A1 (fr) * | 2011-08-01 | 2013-02-06 | Hunt Electronic Co., Ltd. | Serveur vidéo de réseau et son procédé de contrôle vidéo |
US20150201197A1 (en) * | 2014-01-15 | 2015-07-16 | Avigilon Corporation | Streaming multiple encodings with virtual stream identifiers |
US20150199366A1 (en) * | 2014-01-15 | 2015-07-16 | Avigilon Corporation | Storage management of data streamed from a video source device |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11153615B2 (en) | Method and apparatus for streaming panoramic video | |
US20150124048A1 (en) | Switchable multiple video track platform | |
US20140219634A1 (en) | Video preview creation based on environment | |
JP6979035B2 (ja) | 仮想現実メディアコンテンツのストリーミングを向上させる方法、デバイス及びコンピュータプログラム | |
US20160150212A1 (en) | Live selective adaptive bandwidth | |
US20150208103A1 (en) | System and Method for Enabling User Control of Live Video Stream(s) | |
KR101528863B1 (ko) | 파노라마 영상의 스트리밍 서비스 제공 시스템에서 타일링 영상 동기화 방법 | |
WO2014193996A2 (fr) | Diffusion vidéo en continu sur un réseau, avec trick play basé sur des fichiers trick play séparés | |
US20200213631A1 (en) | Transmission system for multi-channel image, control method therefor, and multi-channel image playback method and apparatus | |
JP7515546B2 (ja) | 中断可能な映像トランスコーディング | |
US20140294080A1 (en) | Image processing device, image processing method, and computer program | |
CN113438495A (zh) | Vr直播方法、装置、系统、设备及存储介质 | |
US20180227504A1 (en) | Switchable multiple video track platform | |
US11388455B2 (en) | Method and apparatus for morphing multiple video streams into single video stream | |
US10380077B2 (en) | System and method for upload and synchronization of media content to cloud based media services | |
CN108141564B (zh) | 用于视频广播的系统和方法 | |
WO2018222974A1 (fr) | Procédé et appareil de transformation de multiples flux vidéo en un seul flux vidéo | |
Junior et al. | A software-based solution for distributing and displaying 3d uhd films | |
KR101954298B1 (ko) | 하이브리드 스트리밍 방법, 스트리밍 서버 및 시스템 | |
US10271075B2 (en) | Cloud encoding system | |
Cho et al. | 360-degree video traffic reduction using cloud streaming in mobile | |
US11328453B2 (en) | Device and method for image processing | |
US10893331B1 (en) | Subtitle processing for devices with limited memory | |
US20180262790A1 (en) | Systems and methods for adaptive streaming using jpeg 2000 | |
CN112203101A (zh) | 一种远程视频直播方法、装置及电子设备 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 18730205 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 27.02.2020) |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 18730205 Country of ref document: EP Kind code of ref document: A1 |