WO2022143212A1 - 由组合传送的多个串流取出特定串流播放的系统及方法 - Google Patents

由组合传送的多个串流取出特定串流播放的系统及方法 Download PDF

Info

Publication number
WO2022143212A1
WO2022143212A1 PCT/CN2021/139040 CN2021139040W WO2022143212A1 WO 2022143212 A1 WO2022143212 A1 WO 2022143212A1 CN 2021139040 W CN2021139040 W CN 2021139040W WO 2022143212 A1 WO2022143212 A1 WO 2022143212A1
Authority
WO
WIPO (PCT)
Prior art keywords
stream
view frame
source
view
signal
Prior art date
Application number
PCT/CN2021/139040
Other languages
English (en)
French (fr)
Inventor
陈韦廷
Original Assignee
光阵三维科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 光阵三维科技有限公司 filed Critical 光阵三维科技有限公司
Priority to JP2023563142A priority Critical patent/JP2024504513A/ja
Publication of WO2022143212A1 publication Critical patent/WO2022143212A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/21805—Source of audio or video content, e.g. local disk arrays enabling multiple viewpoints, e.g. using a plurality of cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/2187—Live feed
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/174—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a slice, e.g. a line of blocks or a group of blocks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
    • H04N21/2365—Multiplexing of several video streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/262—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists
    • H04N21/26208—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/65—Transmission of management data between client and server
    • H04N21/658—Transmission by the client directed to the server
    • H04N21/6587—Control parameters, e.g. trick play commands, viewpoint selection
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845—Structuring of content, e.g. decomposing content into time segments
    • H04N21/8455—Structuring of content, e.g. decomposing content into time segments involving pointers to the content, e.g. pointers to the I-frames of the video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding

Definitions

  • the present invention relates to a stream playing system and method, in particular to a system and method for extracting a specific stream from a plurality of streams transmitted in combination.
  • Streaming media refers to a technology that compresses a multimedia file containing a series of frames and transmits the compressed multimedia file in segments through the Internet, so as to provide the content of the multimedia file in real time.
  • the packet of the file can be transmitted from the server to the receiving end like a stream, and the receiving end can play the downloaded content while downloading the multimedia file, without having to download the complete multimedia file in advance.
  • one of the applications using streaming media is webcasting.
  • the live broadcaster or the live broadcast unit transmits audio and video signals to the streaming server, and the client can receive the audio and video stream containing the audio and video signals sent by the streaming server (streaming media). , so as to play the live screen of the live broadcaster or the live broadcaster.
  • the live broadcast unit provides live images from different viewing angles, the live broadcast unit can transmit multiple video and audio signals to the streaming server, and the streaming server can transmit the video and audio streams of the viewing angle that the user wants to watch to the client.
  • just sending a single video stream to a large number of different clients may bring the streaming server close to the load limit, so the streaming server usually does not provide video streams of different viewing angles to the same client at the same time. , therefore, when the user of the client needs to switch the viewing angle, he can only send the viewing angle switching request to the streaming server, so that the streaming server sends the corresponding viewing angle changed video stream to the streaming server according to the received viewing angle switching request.
  • the streaming server since the load of the streaming server is close to the upper limit, the streaming server may not be able to receive or immediately process the viewing angle switching request sent by the client, resulting in the delayed transmission of the video stream after the viewing angle change to the client.
  • the client even if the client receives the video stream after the viewing angle has been changed, the client will not start playing the viewing angle change until the number of frames (playable time) contained in the received video stream has not reached the predetermined value. After the video and audio streaming, the screen played by the client stops, and the users of the client may not be able to watch the live screen for a period of time.
  • the present invention discloses a system and method for extracting a specific stream from a plurality of streams transmitted in combination, wherein:
  • the system for extracting a specific stream from a plurality of streams transmitted in combination disclosed by the present invention at least includes: a plurality of first signal sources, respectively outputting the first audio and video streams; a stream server; and an array server for receiving each The first video and audio streams, and continuously extract each first single-view frame from each first video and audio stream, respectively, and combine all the extracted first single-view frames into a first multi-view frame, and combine the first multi-view frames.
  • the view frame is sent to the streaming server by the first multi-source stream; the client is used to connect to the streaming server, receive the first multi-source stream, and continuously obtain the same first signal source from the first multi-view frame
  • the generated first single-view frame is displayed, and the obtained first single-view frame is displayed, so that the first video and audio stream is played, and when the first switching signal is generated, the first multi-view frame is changed continuously. extracting a first single-view frame generated by another first signal source corresponding to the first switching signal, and displaying another extracted first single-view frame, so that another first video and audio stream is played.
  • the method for playing a specific stream from a plurality of streams transmitted in combination disclosed in the present invention at least includes the following steps: the array server receives the first video and audio streams output by the plurality of first signal sources; Each of the first single-view frames is extracted from an audio and video stream, and all the extracted first single-view frames are combined into a first multi-view frame, and each first multi-view frame is sent to the first multi-source stream.
  • the client connects to the streaming server and receives the first multi-source stream; the client continuously obtains the first single-view frame generated by the same first signal source from the first multi-view frame, and displays the obtained the first single-view frame, so that the first video stream is played; when the first switching signal is generated, the client continuously extracts another first signal corresponding to the first switching signal from the first multi-viewing frame The first single-view frame generated by the source is displayed, and another first single-view frame extracted is displayed, so that another first video and audio stream is played.
  • the system and method disclosed in the present invention are as above, and the difference between the system and the prior art is that the present invention combines the audio and video streams output by multiple signal sources through the array server into a multi-source stream including all
  • the multi-source stream generated by the combination is sent to the stream server, so that the stream server provides the multi-source stream to the client, and the client extracts the audio and video stream to be played from the received multi-source stream, so as to solve the problem of the current situation.
  • FIG. 1 is a system architecture diagram of the present invention for extracting a specific stream from a plurality of streams transmitted in combination.
  • FIG. 2 is a schematic diagram of M*N blocks according to an embodiment of the present invention.
  • FIG. 3 is a flowchart of a method for extracting a specific stream from a plurality of streams transmitted in combination according to the present invention.
  • FIG. 4 is a schematic diagram of an operation interface according to an embodiment of the present invention.
  • FIG. 5 is a schematic diagram of a signal source setup according to an embodiment of the present invention.
  • FIG. 6 is a flowchart of another method for extracting a specific stream from a plurality of streams transmitted in combination according to the present invention.
  • FIG. 7 is a schematic diagram of another signal source setup according to an embodiment of the present invention.
  • the present invention can combine the audio and video streams sent by multiple signal sources into another multi-source stream with higher resolution and including all audio and video streams by the array server without changing the structure of the existing streaming service.
  • the multi-source streaming can be sent to the existing streaming server, so that the client can obtain the multi-source streaming from the streaming server, and can retrieve and play the audio and video stream from a specific signal source from the multi-source streaming, and According to the user's operation, another video and audio stream can be retrieved and played from the multi-source stream, so that the user can quickly switch to another video and audio stream displaying different viewing angles for viewing.
  • the array server, the streaming server and the client can be computing devices.
  • the computing device mentioned in the present invention includes, but is not limited to, one or more processing modules, one or more memory modules, and hardware elements such as a bus connecting different hardware elements (including memory modules and processing modules).
  • a computing device can load and execute an operating system, so that the operating system runs on the computing device, and can also execute software or programs.
  • the computing device also includes a housing, and the above-mentioned hardware components are arranged in the housing.
  • the bus of the computing device mentioned in the present invention may include one or more types, such as a data bus, an address bus, a control bus, an expansion bus, and / or local bus (local bus) and other types of bus.
  • the bus of the computing device includes, but is not limited to, the Industry Standard Architecture (ISA) bus, the Peripheral Component Interconnect (PCI) bus, the Video Electronics Standards Association (VESA) local bus, And serial universal serial bus (Universal Serial Bus, USB), fast peripheral component interconnect (PCI Express, PCI-E/PCIe) bus and so on.
  • ISA Industry Standard Architecture
  • PCI Peripheral Component Interconnect
  • VESA Video Electronics Standards Association
  • USB serial universal serial bus
  • PCI Express PCI Express
  • PCI-E/PCIe fast peripheral component interconnect
  • the processing module of the computing device of the present invention is coupled to the bus.
  • the processing module includes a register bank or a register space, and the register bank or register space can be completely set on the processing chip of the processing module, or all or part of it can be set outside the processing chip and via Dedicated electrical connections and/or via a bus are coupled to the processing chip.
  • the processing module may be a central processing unit, a microprocessor or any suitable processing element. If the computing device is a multi-processor device, that is, the computing device includes multiple processing modules, the processing modules included in the computing device are all the same or similar, and are coupled and communicated through a bus.
  • the processing module can interpret a computer instruction or a series of multiple computer instructions to perform specific operations or operations, such as mathematical operations, logical operations, data comparison, copy/move data, etc., to drive other hardware elements in the computing device or Run the operating system or execute various programs and/or modules.
  • a computing device usually also includes one or more chipsets.
  • the processing module of the computing device may be coupled to the chipset or electrically connected to the chipset through a bus.
  • Chipset is composed of one or more integrated circuits (Integrated Circuit, IC), including memory controller and peripheral input/output (I/O) controller, that is to say, memory controller and peripheral I/O controller may include Within one integrated circuit, two or more integrated circuits can also be used for implementation.
  • Chipsets typically provide I/O and memory management functions, and provide a number of general-purpose and/or special-purpose registers, timers, etc., where the above-mentioned general-purpose and/or special-purpose registers and timers can be Access or use by one or more processing modules that are sexually connected to the chipset.
  • the processing modules of the computing device may also access data in the memory modules and mass storage areas installed on the computing device through the memory controller.
  • the above-mentioned memory module includes any type of volatile memory (volatile memory) and/or non-volatile memory (non-volatile memory, NVRAM) memory, such as static random access memory (Static Random Access Memory, SRAM), dynamic random access memory Access memory (Dynamic Random Access Memory, DRAM), read-only memory (Read-Only Memory, ROM), flash memory (Flash memory), etc.
  • the above-mentioned large-capacity storage area can include any type of storage device or storage medium, such as a hard drive, an optical disc, a flash drive, a memory card, a solid state disk (Solid State Disk, SSD) ), or any other storage device, etc. That is, the memory controller can access data in static random access memory, dynamic random access memory, flash memory, hard drive, solid state drive.
  • a hard drive such as a hard drive, an optical disc, a flash drive, a memory card, a solid state disk (Solid State Disk, SSD) ), or any other storage device, etc. That is, the memory controller can access data in static random access memory, dynamic random access memory, flash memory, hard drive, solid state drive.
  • SSD Solid State Disk
  • the processing module of the computing device can also be connected and communicated with peripheral devices or interfaces such as peripheral output devices, peripheral input devices, communication interfaces, and GPS receivers through the peripheral I/O controller via the peripheral I/O bus.
  • peripheral input device can be any type of input device, such as a keyboard, mouse, trackball, touchpad, joystick, etc.
  • the peripheral output device can be any type of output device, such as a monitor, printer, etc.
  • the peripheral output device can also be the same device, such as a touch screen.
  • the communication interface may include a wireless communication interface and/or a wired communication interface
  • the wireless communication interface may include support for wireless local area networks (such as Wi-Fi, Zigbee, etc.), Bluetooth, infrared, Near-field communication (NFC), 3G /4G/5G and other mobile communication network (cellular network) or other wireless data transmission protocol interface
  • wired communication interface can be Ethernet device, DSL modem, cable modem, asynchronous transfer mode (Asynchronous Transfer Mode) Mode, ATM) devices, or optical fiber communication interfaces and/or components, etc.
  • the processing module can periodically poll various peripheral devices and interfaces, so that the computing device can input and output data through various peripheral devices and interfaces, and can also communicate with another computing device having the hardware elements described above. to communicate.
  • the system of the present invention includes a first signal source 110 , an array server 151 , a streaming server 153 , a client 160 , and an attachable second signal source 130 .
  • the first signal source 110, the second signal source 130, and the streaming server 153 can be connected to the array server 151 through a physical connection line or wireless communication, and the streaming server 153 and the client 160 can be connected through a wireless network , but the present invention is not limited to this.
  • the first signal source 110 is responsible for outputting the first audio and video stream, and the second signal source 130 can output the second audio and video stream.
  • the audio and video streams (the first audio and video stream, the second audio and video stream) output by the signal sources (the first signal source 110, the second signal source 130) are usually generated by the signal sources themselves
  • the signal source can continuously capture external images to generate continuous frames, and output the generated frames as an audio and video stream in sequence according to the generated sequence, but the invention is not limited to this.
  • the signal source can be a computing device capable of capturing images and sounds, converting the captured images and sounds into audio and video signals and outputting them, such as a camera, but the present invention is not limited to this, for example , the signal source may also be a computing device capable of storing video and audio files and transmitting the stored video and audio files to the array server 151 .
  • the signal source can use various existing protocols to transmit the audio-visual signal/audio-audio file to the array server 151, for example, using the Network Device Interface (NDI) protocol, but the present invention is not limited to this.
  • NDI Network Device Interface
  • first signal sources 110 may be included, and two or more second signal sources 130 may also be included.
  • the number of the second signal sources 130 may be the same as the number of the first signal sources 110 , and each different first signal source 110 may correspond to a different second signal source 130 , that is, the first signal source 110 There may be a one-to-one correspondence with the second signal source 130 .
  • the array server 151 is responsible for receiving the first video and audio stream output by the first signal source 110, and is responsible for extracting a different first single-view frame from each received first video and audio stream, and extracting a different first single-view frame from the different first video and audio streams. Each of the first single-view frames extracted from an audio and video stream is combined into a first multi-view frame.
  • the array server 151 may first generate a multi-view frame 200 with M*N blocks as shown in FIG. 2 , and the number of blocks in the multi-view frame 200 is greater than or equal to the number of the first signal sources 110 .
  • M and N are greater than 1.
  • M and N are the same numerical value and are a power of 2, but the present invention is not limited to this, that is to say, M and N can be 2, 4 and other numerical values except for In addition, it may be 3, 5, etc., and M and N are not necessarily the same.
  • the array server 151 can also define the correspondence between each acquired first single-view frame and each block in the multi-view frame 200 according to the source identification data of the first signal source 110 , and can convert the acquired first single-view The frames are placed in corresponding blocks in the multi-view frame 200 to generate the first multi-view frame.
  • the source identification data is data that can represent a specific signal source, usually composed of a specific number of words, letters, numbers, and symbols arranged in a specific manner, such as the network address used by the signal source, the interface number to which the signal source is connected , the product serial number of the signal source, the specific data preset on the signal source, etc., but the present invention is not limited to this.
  • the array server 151 can arrange each signal source according to the source identification data of each signal source (the first signal source 110 ), and can arrange the frames (the first single-view frame) generated by the arranged signal sources with the left first
  • the sequence from top to bottom on the rear right defines the correspondence between each first single-view frame and each block in the multi-view frame 200 .
  • first signal sources there are four first signal sources (111-114), if the array server 151 can define 2*2 blocks (arranged in the upper left, lower left, upper right, and lower right positions), and The order of the first signal sources that can be arranged according to the source identification data of the first signal sources (111-114) is the first signal source 114, the first signal source 113, the first signal source 112, the first signal source 111, Then the first single-view frame generated by the first signal source 114 may correspond to the block located at the upper left corner, the first single-view frame generated by the first signal source 113 may correspond to the block located at the upper right corner, and the The first single-view frame generated by a signal source 112 may correspond to the block in the lower left corner, and the first single-view frame generated by the first signal source 111 may correspond to the block in the lower right corner.
  • the array server 151 may also adjust (enlarge or reduce) the resolution of the first single-view frame , so that the resolution of the first single-view frame is the same as the resolution of the blocks in the multi-view frame 200 .
  • the array server 151 does not only perform the action of generating one first multi-view frame once, but can repeatedly perform the above-mentioned actions continuously, thereby generating a series of first multi-view frames. That is to say, after the array server 151 generates a first multi-view frame, it can continue to retrieve another different first single-view frame (usually arranged in another first single-view frame after the first single-view frame that has been taken out and adjacent to the existing first single-view frame), and another different first single-view frame that can be taken out again Combining into another different first multi-view frame, the process is repeated continuously until the first video and audio stream transmitted by any signal source is not received.
  • the array server 151 is also responsible for generating the first multi-source stream.
  • the first multi-source stream generated by the array server 151 includes the first multi-view frames generated by the array server 151 , and the first multi-view frames in the first multi-source stream are arranged according to the sequence generated by the array server 151 .
  • the array server 151 is also responsible for sending the generated first multi-source stream to the stream server 153 .
  • the array server 151 can use the Real-Time Messaging Protocol (RTMP) to send the first multi-source stream to the stream server 153, but the present invention is not limited to this, any data that can be used to transmit Any video streaming method can be used between the array server 151 and the streaming server 153 .
  • the array server 151 can generally transmit the generated multi-view frame to the streaming server 153 every time a multi-view frame is generated.
  • the array server 151 can also generate the first multi-source stream as described above, combine the second audio and video streams output by all the second signal sources 130 into a second multi-source stream, and can combine the generated second multi-source stream.
  • the streaming is sent to the streaming server 153 . That is to say, the array server 151 can receive the second multi-source stream output by each second signal source 130, and can continuously extract a second single-view frame from each second multi-source stream, and can continue to All the extracted second single-view frames are combined into a second multi-view frame, and a second multi-source stream including the sequentially generated second multi-view frames can be generated.
  • the stream server 153 may receive the first multi-source stream generated by the array server 151 . If the array server 151 also generates the second multi-source stream, the stream server 153 can also receive the second multi-source stream generated by the array server 151 .
  • the streaming server 153 is also responsible for providing the client 160 connection and providing the first multi-source stream to the client 160 requesting the first multi-source stream. In some embodiments, the streaming server 153 may also provide the first multi-source stream. The second multi-source stream is provided together with the first multi-source stream to the client 160 requesting the first multi-source stream.
  • the client 160 is usually a mobile phone, a tablet computer and other devices that are convenient for users to carry, but the present invention is not limited to this.
  • the client 160 is responsible for connecting to the streaming server 153 and for requesting the first multi-source stream from the streaming server 153 , so as to receive the first multi-source stream provided by the streaming server 153 .
  • the client 160 may also receive the second multi-source stream from the streaming server 153 when receiving the first multi-source stream.
  • the client 160 can use the HTTP Live Streaming transmission protocol (HLS) to receive the first multi-source stream and the second multi-source stream, but the present invention is not limited to this, any transmission method that can transmit audio and video signals can be Can be used between the streaming server 153 and the client 160 .
  • HLS HTTP Live Streaming transmission protocol
  • the client 160 is also responsible for fetching a first multi-view frame from the first multi-source stream, obtaining a first single-view frame from the fetched first multi-view frame, and displaying the obtained first frame. perspective frame.
  • the client 160 can continuously extract the positions from each of the first multi-view frames in the first multi-source stream according to the sequence of the first multi-view frames included in the first multi-source stream (the same area). block) and sequentially display the extracted first single-view frames, so that the first video and audio stream including the extracted first single-view frames is played.
  • the client 160 can retrieve the first single-view frame in the corresponding position (block) according to a predetermined value, wherein the predetermined value can correspond to a block in a specific corner (eg, upper left corner or upper right corner), corresponding to The block in the center, or the corresponding block arranged in half of the total number of blocks (eg, the block arranged in the 8th position when there are 16 blocks), etc., but the present invention is not limited to this.
  • the first switching signal or the second switching signal or pauses The control signal may be generated as the user of the client 160 performs a specific operation on the client 160 .
  • the first switching signal/second switching signal may be a touch generated by the user of the client 160 sliding a moving distance in the moving direction on the main display area 410 of the touch screen 400 of the client 160 as shown in FIG. 4 .
  • the signal can also be a rotation signal generated by the user turning the client terminal 160 in the moving direction or angle, but the specific operation for generating the first switching signal/second switching signal is not limited to the above; the pause control signal can be used
  • the client 160 can obtain another different first single-view frame corresponding to the first switching signal from the first multi-view frame extracted from the first multi-source stream, and display the first single-view frame.
  • Another first single-view frame obtained is made to include another different first video and audio stream (in the present invention, the first single-view frame corresponding to the first switching signal is also used as the first video and audio stream corresponding to the first switching signal).
  • Video streaming representation is played.
  • the client 160 can obtain the arrangement in the first multi-view frame according to the direction corresponding to the moving direction indicated by the first switching signal.
  • the first single-view frame output by the first signal source 110 of the currently displayed first single-view frame is output by another first signal source 110 at a certain interval.
  • the block corresponding to the currently displayed first single-view frame is the block 231 (see FIG. 2 ) arranged at the ninth position, and the arrangement order of the blocks in the first multi-view frame is left first Back right from top to bottom (that is, in two adjacent blocks, the order of the blocks on the left/top is earlier than the block on the right/bottom), then when the first switching signal is left When the touch signal is slid, the first switching signal corresponds to the block 224 arranged at the eighth position, that is, the client 160 can instead continuously obtain the first block 224 from the first multi-view frame.
  • a single-view frame and when the first switching signal is a touch signal for sliding to the right, the first switching signal corresponds to the block 232 arranged at the tenth position, and the client 160 can change to continue from the first multi-view
  • the first monoscopic frame placed in block 232 is obtained in the frame.
  • the currently displayed first single-view frame corresponds to the block 224 arranged at the eighth position
  • the first switch signal corresponds to the block 224 arranged at the eighth position.
  • the seven positions correspond to the block 223, that is, the client 160 can instead continuously obtain the first single-view frame placed in the block 223 from the first multi-view frame, and when the first switching signal is a right-sliding touch, When the control signal is activated, the first switching signal corresponds to the block 231 arranged at the ninth position, and the client 160 may instead continuously obtain the first single-view frame placed in the block 231 from the first multi-view frame.
  • the block corresponding to the first switching signal may also be related to the moving direction and the moving distance indicated by the first switching signal, that is, it is assumed that the block corresponding to the currently displayed first single-view frame is arranged In the block 231 at the ninth position, and there are a plurality of distance thresholds with different length values, when the first switching signal is a touch signal for sliding to the left, if the moving distance of the first switching signal does not meet the first distance threshold , the first switching signal may correspond to the block 224 arranged in the previous position (the eighth one), and if the moving distance of the first switching signal only meets the first distance threshold, the first switching signal may correspond to the block 224 arranged in the first two Corresponding to the block 223 of the first position (the seventh), if the moving distance of the first switching signal complies with the first distance threshold and the second distance threshold, the first switching signal can be arranged with the first three positions (the sixth) corresponding to block 222 of , and so on.
  • the first switching signal when the first switching signal is a touch signal sliding to the right, if the moving distance of the first switching signal does not meet the first distance threshold, the first switching signal can be The block 232 arranged in the latter position (the tenth) corresponds to the block 232. If the moving distance of the first switching signal only meets the first distance threshold, the first switching signal can be arranged in the latter two positions (the eleventh). corresponding to the block 233 of the first switching signal, if the moving distance of the first switching signal complies with the first distance threshold and the second distance threshold, the first switching signal may correspond to the block 234 arranged in the last three positions (the twelfth), So on and so forth.
  • the client 160 can also pause the first multi-view frame taken from the first multi-source stream when the pause control signal is generated, can also pause to take the first single-view frame from the first multi-view frame, or can not.
  • Displaying the first single-view frame obtained from the first multi-view frame may continuously display the last obtained first single-view frame before the pause control signal is generated, so as to pause the playback of the first video stream.
  • the client 160 may also extract the first switching control signal from the multi-view frame including the currently displayed first single-view frame after the pause control signal is generated and before the playback control signal is generated, and the first switch control signal is generated. Another different first single-view frame corresponding to the switching signal is switched, and another single-view frame taken out is displayed.
  • the client 160 can also select a different first single-view frame from the multi-view frame of the currently displayed first single-view frame according to the moving direction or the moving direction and the moving distance indicated by the first switching control signal in the above-mentioned manner.
  • Single view frame The client 160 can also select to continue to display the first multi-view frame extracted from the currently received first multi-source stream according to a preset value when the playback control signal is generated after the pause control signal is generated, or select The display starts from the first multi-view frame extracted from the first multi-source stream received after the pause control signal is generated, and there is no particular limitation in the present invention.
  • the client 160 may also set the predetermined value to correspond to the block corresponding to the generated first switching signal after the first switching signal is generated. In this way, when the client 160 continues to obtain the first single-view frame from the first multi-view frame extracted from the first multi-source stream, the first single-view frame is obtained from the block corresponding to the first switching signal. , that is, the first single-view frame is continuously obtained from the blocks located in the same position in the first multi-view frame.
  • the client 160 may instead continuously obtain the second single-view frame corresponding to the currently displayed first single-view frame from the second multi-view frame extracted from the second multi-source stream. frame, and display the obtained second single-view frame, so that the second video stream including the second single-view frame (also represented by the second video stream corresponding to the second switching signal in the present invention) is played.
  • the second single-view frame corresponding to the currently displayed first single-view frame is the block in the second multi-view frame corresponding to the position of the currently displayed first single-view frame in the first multi-view frame.
  • the block corresponding to the currently displayed first single-view frame is arranged at the ninth position in the first multi-view frame
  • the second single-view frame corresponding to the currently displayed first single-view frame The corresponding block is also arranged at the ninth position in the second multi-view frame.
  • the moving distance represented by the second switching signal can make the second switching signal correspond to different groups of second signal sources 130.
  • the client 160 may have a distance threshold, when the moving distance represented by the second switching signal does not exceed At the distance threshold, the second switching signal may correspond to a group of second signal sources 130 that are closer to the first signal source 110.
  • the second switching signal It may correspond to another group of second signal sources 130 that are far away from the first signal source 110 .
  • the first signal source 110 is a camera, and there are sixteen first signal sources 110 in total, but the present invention is not limited to this.
  • the first signal source 110 needs to be set up.
  • the first signal sources (111-126) are erected around the performance stage, and the first signal sources (111-126) are erected in the same circular (or arc) track.
  • the network addresses of the first signal sources (111-126) are 172.0.0.1-16 respectively, and the first signal sources (111-126) are in ascending order of network addresses.
  • the first signal source 111 with the network address 172.0.0.1 is adjacent to the first signal source 112 with the network address 172.0.0.2 and the first signal source 126 with the network address 172.0.0.16, and the network address is 172.0.0.2
  • the first signal source 112 is adjacent to the first signal source 111 with the network address 172.0.0.1 and the first signal source 113 with the network address 172.0.0.3, ..., the first signal source 125 with the network address 172.0.
  • the first signal source 124 of 172.0.0.14 and the first signal source 126 of 172.0.0.16 are adjacent to the first signal source 126 with the network address of 172.0.0.16 and the first signal source 125 and 172.0 of the network address of 172.0.0.15.
  • the first signal source 111 of 0.1 is adjacent.
  • the first signal source 110 can start shooting images on the performance stage, and can transmit the captured images to the array server 151 in the form of video and audio streaming (streaming media).
  • the resolution of the first audio and video stream generated by the first signal source (111-126) may be 960*540, and the first signal source (111-126)
  • the video and audio streaming is sent to the array server 151 .
  • the array server 151 can continuously extract a different first single-view frame from each of the first audio and video streams, and the array The server 151 may also combine the different first single-view frames extracted from the first video and audio streams each time into a first multi-view frame, and may transmit the first multi-view frame including the first multi-view frame as the first multi-source stream to the streaming server 153 (step 330).
  • the array server 151 can generate a first multi-view frame with a resolution of 3840*2160, and define 4*4 blocks as shown in FIG.
  • each area The resolution of the block is 960*540
  • the network address (source identification data) of the first signal source (111-126) the first single-view frame and the first single-view frame extracted from different first video and audio streams are defined.
  • the array server 151 can define the first signal generated by the camera (the first signal source 111 ) whose network address is 172.0.0.1.
  • the first single-view frame in the video stream corresponds to the block 211, and the first single-view frame in the first video stream generated by the camera with the network address of 172.0.0.2 (the first signal source 112) corresponds to the block 212,
  • the first single-view frame in the first video and audio stream generated by the camera with the network address of 172.0.0.3 (the first signal source 113 ) corresponds to blocks 213, . . .
  • the camera with the network address of 172.0.0.5 (the first signal source 115)
  • the first single-view frame in the generated first video stream corresponds to blocks 221, .
  • a single-view frame corresponds to block 241, ..., and the first single-view frame in the first video stream generated by the camera with network address 172.0.0.15 (the first signal source 125) corresponds to block 243, and the network address is 172.0 .0.16 camera (second signal source 126) in the first video stream generated by the first single-view frame corresponding to the block 244, then, the array server 151 can place each first single-view frame into the corresponding area In blocks (211-244), in this way, the array server 151 can generate a first multi-view frame including sixteen first single-view frames, and then can generate a first multi-source stream including a plurality of first multi-view frames , and the first multi-source streaming can be sent to the streaming server 153 through an instant message protocol in a push streaming manner.
  • the client 160 can connect to the streaming server 153 and request the first multi-source stream from the streaming server 153 .
  • the streaming server 153 may send the first multi-source stream to the client 160, so that the client 160 may receive the first multi-source stream (step 350).
  • the client 160 can continuously extract the first multi-view frame from the received first multi-source stream, and can continuously extract the first multi-view frame from each first multi-view frame
  • the first single-view frame is obtained, and the obtained first single-view frame can be continuously displayed, so that the first video and audio stream is played (step 360 ).
  • the client 160 can start the calculation from the upper left corner (if the pixel in the upper left corner is in the first multi-view frame The coordinates are (1,1)), and for each first multi-view frame, take out the image with the X-axis from the 1st to the 960th pixel and the Y-axis as the 1st to 540th pixel (the taken out image is The first single-view frame), and then the retrieved image (the first single-view frame) is displayed. In this way, the first video and audio stream captured by the first signal source 111 can be played on the client 160 .
  • the client 160 may use the frames per second (FPS) of the first multi-source stream as the frequency of fetching the first multi-view frame from the first multi-source stream.
  • FPS frames per second
  • the client 160 plays the first video stream
  • the client 160 can continue to extract another first single-view frame from the received multi-source stream, and can continue to display it
  • the obtained first single-view frame causes another first video stream to be played (step 370).
  • the first switching signal is a touch signal generated by the user of the client terminal 160 sliding his finger to the right on the main display area 410 (refer to FIG.
  • the client 160 can, after the first switching signal is generated, according to the touch signal swiping right (the first switching signal switch signal) select the first signal source 112 that is adjacent to the first signal source 111 that outputs the first audio and video stream currently being played and is arranged after the first signal source 111 (assuming a counterclockwise direction).
  • One video stream that is to say, the client 160 can start the calculation from the upper left corner, and then continuously extract the pixels from the 961st to 1920th pixels on the X axis and the 1st to 540th pixels on the Y axis for each first multi-source frame.
  • pixel image (the extracted image is the first single-view frame generated by the first signal source 112 ), and then the extracted first single-view frame is displayed. In this way, the first single-view frame captured by the first signal source 112
  • the video stream can then be played on the main display area 410 .
  • the client 160 plays the second video stream captured by the first signal source 112 if the touch signal (first switching signal) that slides to the right is generated again, the client 160 can also continue
  • the first multi-view frame is obtained from the received first multi-source stream, and the first multi-view frame generated by obtaining another first signal source 113 corresponding to the first switching signal in each first multi-view frame can be continuously obtained.
  • a single-view frame that is, for each first multi-view frame, images with the 1921st to 2880th pixels on the X axis and the 1st to 540th pixels on the Y axis are taken out, and the obtained images can be displayed continuously, so that The first video and audio stream captured by the first signal source 113 is played; and if the first switching signal generated again is a touch signal for sliding to the left, the client 160 can continue to use the received first multi-source
  • the first multi-view frame is extracted from the stream, and the first single-view frame generated by the first signal source 111 corresponding to the first switching signal can be continuously obtained from each first multi-view frame, that is, for each In the first multi-view frame, images with pixels from the 1st to 960th pixels on the X axis and pixels from the 1st to 540th on the Y axis are taken out, and the obtained images can be continuously displayed, so that the first signal generated by the first signal source 111 is captured.
  • a video stream is played.
  • the client 160 can also select and output the first signal source 111 of the currently playing first audio-video stream after the first switching signal is generated, in a position corresponding to the moving direction and arranged at an interval corresponding to the moving distance
  • the client 160 can play the first audio and video stream output by the first signal source 112 in the main display area 410 instead, that is, the client 160 can use
  • the calculation starts from the upper left corner, and instead, the image with the 961st to 1920th pixels on the X axis and the 1st to 540th pixels on the Y axis is continuously extracted for each first multi-view frame (the extracted image is the first signal.
  • the client 160 can instead play the first signal source 115 in the main display area 410
  • the outputted first audio and video stream that is, the client 160 can change to continuously extract images from the first multi-view frame whose X-axis is the 1st to 960th pixels and the Y-axis is 541st to 1080th pixels, ...; So on and so forth.
  • the array server 151 can combine the audio and video streams output by multiple signal sources at a higher resolution into a multi-source stream including all audio and video streams, and then transmit them to the client by the streaming server 153 160, so that the client 160 can switch and play the audio and video streams output by different signal sources locally, so as to switch and play the audio and video streams of different viewing angles with almost no delay time.
  • the client 160 when the client 160 plays the first video stream, if the pause control signal is generated first, the client 160 can pause the first multi-view frame extracted from the first multi-source stream, and also The acquisition of the first single-view frame from the first multi-view frame can be paused, and the first single-view frame obtained last before the pause control signal is generated can be continuously displayed, so that the playback of the first video and audio stream is suspended.
  • the client 160 may also extract a different one corresponding to the first switching signal from the multi-view frame including the currently displayed first single-view frame when the first switching control signal is generated after the pause control signal is generated.
  • the first monoscopic frame, and another monoscopic frame that is fetched is displayed.
  • the client 160 may, with each generation of the first switching signal, take out and display the first single-view frame from the block in front or rear of the arrangement position; or may follow the movement indicated by the first switching signal
  • the direction and moving distance are selected by taking out and displaying the first single-view frame from the blocks whose arrangement position is at a certain interval in the front or back.
  • FIG. 6 For another method of extracting and playing a specific stream from a plurality of streams transmitted in combination provided by the present invention.
  • FIG. 7 it is assumed that as shown in FIG. 7 , there are two groups of signal sources including a first signal source 110 and a second signal source 130 .
  • the first signal source 110 and the second signal source 130 are built around the performance stage. camera, and each of the first signal source 110 and the second signal source 130 has sixteen, but the invention is not limited to this.
  • the network addresses of the first signal sources (111-126) are 172.0.0.1-16
  • the network addresses of the second signal sources (131-146) are 172.0.1.1-16
  • the first signal sources (111-126) In the order of network addresses, they are arranged in a ring on a plane about 150 cm away from the performance stage, and the second signal sources (131-146) are arranged in a ring in the order of network addresses on another plane about 200 cm away from the performance stage.
  • the second signal source 131 whose network address is 172.0.1.1 corresponds to the first signal source 111 whose network address is 172.0.0.1
  • the second signal source 132 whose network address is 172.0.1.2 corresponds to the first signal source 132 whose network address is 172.0.0.2
  • Signal source 112 corresponds to, . . . , and so on.
  • the first signal source 110 and the second signal source 130 can start to capture the images on the stage, and the captured images can be streamed in the form of video and audio. mode is transmitted to the array server 151 .
  • the resolutions of the video streams (the first video stream and the second video stream) generated by all the signal sources (the first signal source 110 and the second signal source 130 ) are all 960*540. , and all signal sources can transmit video and audio streaming to the array server 151 through the NDI protocol.
  • the array server 151 After the array server 151 receives the first audio and video stream transmitted by the first signal source 110 and the second audio and video stream transmitted by the second signal source 130 (step 610 ), the array server 151 can be as in the first embodiment Description, continuously extracting a first single-view frame from each first video and audio stream, and combining the different first single-view frames taken out each time into a first multi-view frame, and the first multi-view frame
  • the first multi-source stream is sent to the streaming server 153.
  • the array server 151 can also generate the first multi-source stream from each of the first video and audio streams in the same process, and continuously extract them from each of the second video and audio streams.
  • a second single-view frame, and different second single-view frames taken out each time can be combined into a second multi-view frame, and the generated second multi-view frame Streaming server 153 (step 630).
  • the process of generating the first multi-source stream by the array server 151 will not be described again.
  • the array server 151 can also generate 4*4 regions with a resolution of 960*540 as shown in FIG. 2 .
  • the second multi-view frame of the block, and the corresponding relationship between each second single-view frame and each block can be defined according to the network address (source identification data) of the second signal source (131-146), or can be based on the second signal
  • the position of the first single-view frame generated by the first signal source 110 corresponding to the source 130 in the first multi-view frame defines the correspondence between the second single-view frame generated by the corresponding second signal source 130 and each block . That is to say, if the array server 151 defines the blocks to be arranged in the order from left to right and from top to bottom, the array server 151 can define the second signal generated by the camera (second signal source 131 ) whose network address is 172.0.1.1.
  • the second single-view frame in the video stream corresponds to the block 211, and the second single-view frame in the second video stream generated by the camera with the network address of 172.0.1.2 (the second signal source 132) corresponds to the block 212, ..., the second single-view frame in the second video stream generated by the camera whose network address is 172.0.1.16 corresponds to the block 244, and then the array server 151 can place each second single-view frame into the corresponding block In this way, the array server 151 can generate a second multi-view frame including sixteen second single-view frames, and can generate a second multi-source stream including the second multi-view frame, and can use the push streaming method to pass
  • the instant messaging protocol sends the first multi-source stream and the second multi-source stream to the streaming server 153 .
  • the client 160 can connect to the streaming server 153 , and can request the first multi-source streaming from the streaming server 153 . source stream.
  • the streaming server 153 can send the first multi-source stream and the second multi-source stream to the client 160, so that the client 160 receives the first multi-source stream and the second multi-source stream.
  • Two multi-source streams (step 650). In this embodiment, it is assumed that the client 160 can use the streaming method to receive the first multi-source stream and the second multi-source stream through the HLS protocol.
  • the client 160 can continuously extract the first multi-view frame from the received first multi-source stream, and can continue to The first single-view frame is obtained from each of the first multi-view frames, and the obtained first single-view frame can be continuously displayed, so that the first video and audio stream is played (step 660 ).
  • the client 160 can start the calculation from the upper left corner, and extract the X-axis for each first multi-view frame It is the image of the 1st to 960th pixels and the Y-axis is the 1st to 540th pixel (the extracted image is the first single-view frame), and then the extracted image (the first single-view frame) is displayed, so, The first audio and video stream captured by the first signal source 111 can be played on the client 160 .
  • the client 160 plays the first video stream, if the first switching signal is generated, the client 160 can continuously extract the first multi-view from the received first multi-source stream as described in the first embodiment frame, and can continuously obtain another different first single-view frame corresponding to the first switching signal from each of the first multi-view frames, and can continuously display the obtained first single-view frame, so that another first single-view frame can be continuously displayed.
  • the client 160 can instead extract the second multi-view frame from the second multi-source stream corresponding to the second switching signal, and can continue to use the In each second multi-view frame, a second single-view frame corresponding to the currently playing first single-view frame is obtained, and the obtained second single-view frame can be continuously displayed, so that the second video stream is played (step 680 ) ).
  • the second switching signal is a touch signal generated by the user of the client 160 sliding upward on the main display area 410 (refer to FIG. 4 ) of the touch screen 400 of the client 160 .
  • the touch signal sliding up and down on the main display area 410 can change the viewing angle of the currently playing video stream.
  • the client 160 can select it according to the touch signal sliding up (second switching signal).
  • the second audio and video stream output by the second signal source 131 corresponding to the first signal source 111 outputting the currently played first audio and video stream that is to say, the client 160 can start the calculation from the upper left corner, From the second single-view frame, extract the images whose X-axis is the 1st to 960th pixels and the Y-axis is the 1st to 540th pixels (the extracted images are the second single-view frames generated by the second signal source 131 ) , and then display the second single-view frame taken out. In this way, the second video and audio stream captured and output by the second signal source 131 can be played on the client 160 .
  • the client 160 plays the second video stream captured by the second signal source 131, if the touch signal (second switching signal) for sliding down is generated, the client 160 can change to continuous
  • the first multi-view frame is obtained from the first multi-source stream corresponding to the second switching signal, and the second signal of the currently playing second single-view frame can be continuously obtained and output from each first multi-view frame.
  • the client 160 can continuously extract the second multi-view frame from the received second multi-source stream, and can continuously obtain the second signal source corresponding to the first switching signal from each second multi-view frame
  • the second single-view frame generated by 132 that is, for each second multi-view frame, the image with the 961st to 1920th pixels on the X axis and the 1st to 540th pixels on the Y axis can be continuously displayed.
  • the acquired image causes the second video and audio stream generated by the second signal source 132 to be played.
  • the first switching signal is not limited to correspond to the second signal source 132, and the client 160 can also communicate with other second signal sources other than the second signal source 131 ( 133 to 146), and can continuously extract and display images at the corresponding positions from the second multi-view frame.
  • the array server combines the audio and video streams output by multiple signal sources into a multi-source stream including all audio and video
  • the source stream is provided to the client, and the client extracts the audio and video stream to be played from the received multi-source stream, and this technical method can solve the existing need to switch the viewing angle when watching live broadcasts in the prior art.
  • the problem of response time is achieved, and the technical effect of reducing the load of the live service server is achieved.
  • the method for extracting a specific stream from a plurality of streams transmitted in combination of the present invention can be implemented in hardware, software, or a combination of hardware and software, or can be implemented in a centralized manner in a computer system or distributed with different components It is implemented in a decentralized manner over several interconnected computer systems.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Human Computer Interaction (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

本发明公开一种由组合传送的多个串流取出特定串流播放的系统及方法，其通过阵列服务器将多个信号源所输出的影音串流组合为一个包含所有影音串流的多源串流并将组合产生的多源串流传送给串流服务器，使串流服务器将多源串流提供给客户端，客户端由所接收到的多源串流中取出欲播放的影音串流的技术手段，可以在观赏直播时即时切换播放不同视角的影音串流，并达成降低直播的服务器的负载的技术功效。

Description

由组合传送的多个串流取出特定串流播放的系统及方法 技术领域
本发明涉及一种串流播放系统及其方法，特别是指一种由组合传送的多个串流取出特定串流播放的系统及方法。
背景技术
串流媒体(Streaming media)是指将包含一连串的帧的多媒体文件压缩后，经过网际网络分段传送被压缩的多媒体文件，借以即时提供观赏多媒体文件的内容的一种技术，此技术使得封装多媒体文件的封包得以像流水一样由服务器端被传送到接收端，且接收端可以在下载多媒体文件的同时播放以下载的内容，而不需要预先下载完整的多媒体文件。
目前，使用串流媒体的其中一种应用是网络直播，直播主或直播单位传送影音信号给串流服务器，客户端可以接收串流服务器所传送的包含影音信号的影音串流(串流媒体)，借以播放直播主或直播单位所直播的画面。若直播单位提供以不同的视角观看直播的现场画面，则直播单位可以传送多个影音信号给串流服务器，串流服务器可以传送使用者欲观看的视角的影音串流给客户端。
事实上，光是传送一个影音串流给大量的不同客户端就可能让串流服务器接近可负载的上限，因此，串流服务器通常不会将不同视角的影音串流同时提供给同一个客户端，因此，客户端的使用者在需要切换视角时，只能将视角切换请求发送给串流服务器，使得串流服务器依据所接收到的视角切换请求将相对应的视角变更后的影音串流传送给客户端，然而，由于串流服务器的负载接近上限，导致串流服务器可能无法接收到或无法即时处理客户端所发送的视角切换请求，导致视角变更后的影音串流迟迟无法传送到客户端； 另外，即使客户端接收到视角变更后的影音串流，但在所接收到的影音串流包含的帧数(即可播放的时间)尚未达到预定值前，客户端不会开始播放视角变更后的影音串流，造成客户端所播放的画面停止，导致客户端的使用者可能有一段时间无法观看到直播画面。
综上所述，可知现有技术中长期以来一直存在观赏直播时切换视角需要反应时间的问题，因此有必要提出改进的技术手段，来解决此问题。
发明内容
有鉴于现有技术存在观赏直播时切换视角需要反应时间的问题，本发明遂公开一种由组合传送的多个串流取出特定串流播放的系统及方法，其中：
本发明所公开的由组合传送的多个串流取出特定串流播放的系统，至少包含：多个第一信号源，分别输出第一影音串流；串流服务器；阵列服务器，用以接收各第一影音串流，并持续由各第一影音串流中分别取出各第一单视角帧，及将所有被取出的第一单视角帧帧组合为第一多视角帧，并将第一多视角帧以第一多源串流传送到串流服务器；客户端，用以连接至串流服务器，并接收第一多源串流，及持续由第一多视角帧中取得相同第一信号源所产生的第一单视角帧，并显示被取得的第一单视角帧，使第一影音串流被播放，及用以于第一切换信号被产生时，改为持续由第一多视角帧中取出与第一切换信号对应的另一第一信号源所产生的第一单视角帧，并显示被取出的另一第一单视角帧，使另一第一影音串流被播放。
本发明所公开的由组合传送的多个串流取出特定串流播放的方法，其步骤至少包括：阵列服务器接收多个第一信号源所输出的第一影音串流；阵列服务器持续由各第一影音串流中分别取出各第一单视角帧，并将所有被取出的第一单视角帧组合为第一多视角帧，及将各第一多视角帧以第一多源串流传送给串流服务器；客户端连接至串流服务器，并接收第一多源串流；客户端持续由第一多视角帧中取得相同第一信号源所产生的第一单视角帧，并显 示被取得的第一单视角帧，使第一影音串流被播放；客户端于第一切换信号被产生时，改为持续由第一多视角帧中取出与第一切换信号对应的另一第一信号源所产生的第一单视角帧，并显示被取出的另一第一单视角帧，使另一第一影音串流被播放。
本发明所公开的系统与方法如上，与现有技术的间的差异在于本发明通过阵列服务器将多个信号源所输出的影音串流组合为一个包含所有影音串流的多源串流并将组合产生的多源串流传送给串流服务器，使串流服务器将多源串流提供给客户端，客户端由所接收到的多源串流中取出欲播放的影音串流，借以解决现有技术所存在的问题，并可以达成降低直播的服务服务器的负载的技术功效。
附图说明
图1为本发明所提的由组合传送的多个串流取出特定串流播放的系统架构图。
图2为本发明实施例所提的M*N个区块的示意图。
图3为本发明所提的由组合传送的多个串流取出特定串流播放的方法流程图。
图4为本发明实施例所提的操作界面的示意图。
图5为本发明实施例所提的信号源架设的示意图。
图6为本发明所提的另一种由组合传送的多个串流取出特定串流播放的方法流程图。
图7为本发明实施例所提的另一种信号源架设的示意图。
附图标记说明：
110～126   第一信号源
130～146  第二信号源
151       阵列服务器
153       串流服务器
160       客户端
200       多视角帧
211～244  区块
400       触控屏幕
410       主显示区域
具体实施方式
以下将配合图式及实施例来详细说明本发明的特征与实施方式，内容足以使任何本领域技术人员能够轻易地充分理解本发明解决技术问题所应用的技术手段并据以实施，借此实现本发明可达成的功效。
本发明可以在不改变现有串流服务的架构下，由阵列服务器将多个信号源所发出的影音串流组合为另一个解析度较大且包含所有影音串流的多源串流，并可以将多源串流传送给既有的串流服务器，使得客户端可以由串流服务器取得多源串流并可以由多源串流中取出并播放特定信号源所发出的影音串流，且能够依据使用者的操作由多源串流中取出并播放另一个影音串流，借以让使用者快速的切换到显示不同视角的另一个影音串流观赏。其中，阵列服务器、串流服务器与客户端可以是计算设备。
本发明所提的计算设备包含但不限于一个或多个处理模块、一条或多条存储器模块、以及连接不同硬件元件(包括存储器模块和处理模块)的总线等硬件元件。通过所包含的多个硬件元件，计算设备可以载入并执行作业系统，使作业系统在计算设备上运行，也可以执行软件或程序。另外，计算设备也包含一个外壳，上述的各个硬件元件设置于外壳内。
本发明所提的计算设备的总线可以包含一种或多个类型，例如包含数据总线(data bus)、地址总线(address bus)、控制总线(control bus)、扩充功能总线(expansion bus)、和/或局域总线(local bus)等类型的总线。计算设备的总线包括但不限于的工业标准架构(Industry Standard Architecture,ISA)总线、周边元件互连(Peripheral Component Interconnect,PCI)总线、视频电子标准协会(Video Electronics Standards Association,VESA)局域总线、以及串列的通用序列总线(Universal Serial Bus,USB)、快速周边元件互连(PCI Express,PCI-E/PCIe)总线等。
本发明所提的计算设备的处理模块与总线耦接。处理模块包含暂存器(Register)组或暂存器空间，暂存器组或暂存器空间可以完全的被设置在处理模块的处理芯片上，或全部或部分被设置在处理芯片外并经由专用电气连接和/或经由总线耦接至处理芯片。处理模块可为中央处理器、微处理器或任何合适的处理元件。若计算设备为多处理器设备，也就是计算设备包含多个处理模块，则计算设备所包含的处理模块都相同或类似，且通过总线耦接与通信。处理模块可以解释一个计算机指令或一连串的多个计算机指令以进行特定的运算或操作，例如，数学运算、逻辑运算、数据比对、复制/移动数据等，借以驱动计算设备中的其他硬件元件或运行作业系统或执行各种程序和/或模块。
计算设备中通常也包含一个或多个芯片组(Chipset)。计算设备的处理模块可以与芯片组耦接或通过总线与芯片组电性连接。芯片组是由一个或多个集成电路(Integrated Circuit,IC)组成，包含存储器控制器以及周边输出入(I/O)控制器等，也就是说，存储器控制器以及周边输出入控制器可以包含在一个集成电路内，也可以使用两个或更多的集成电路实现。芯片组通常提供了输出入和存储器管理功能、以及提供多个通用和/或专用暂存器、计时器等，其中，上述的通用和/或专用暂存器与计时器可以让耦接或电性连接至芯片组的一个或多个处理模块存取或使用。
计算设备的处理模块也可以通过存储器控制器存取安装于计算设备上的存储器模块和大容量储存区中的数据。上述的存储器模块包含任何类型的易失性存储器(volatile memory)和/或非易失性(non-volatile memory,NVRAM)存储器，例如静态随机存取存储器(Static Random Access Memory,SRAM)、动态随机存取存储器(Dynamic Random Access Memory,DRAM)、只读存储器(Read-Only Memory,ROM)、快闪存储器(Flash memory)等。上述的大容量储存区可以包含任何类型的储存装置或储存介质，例如，硬盘机、光盘(optical disc)、随身盘(flash drive)、存储卡(memory card)、固态硬盘(Solid State Disk,SSD)、或任何其他储存装置等。也就是说，存储器控制器可以存取静态随机存取存储器、动态随机存取存储器、快闪存储器、硬盘机、固态硬盘中的数据。
计算设备的处理模块也可以通过周边输出入控制器经由周边输出入总线与周边输出装置、周边输入装置、通信界面、及GPS接收器等周边装置或界面连接并通信。周边输入装置可以是任何类型的输入装置，例如键盘、鼠标、轨迹球、触控板、摇杆等，周边输出装置可以是任何类型的输出装置，例如显示器、印表机等，周边输入装置与周边输出装置也可以是同一装置，例如触控屏幕等。通信界面可以包含无线通信界面和/或有线通信界面，无线通信界面可以包含支持无线区域网络(如Wi-Fi、Zigbee等)、蓝牙、红外线、近场通信(Near-field communication,NFC)、3G/4G/5G等移动通信网络(蜂巢式网络)或其他无线数据传输协议的界面，有线通信界面可为以太网络装置、DSL数据机、缆线(Cable)数据机、非同步传输模式(Asynchronous Transfer Mode,ATM)装置、或光纤通信界面和/或元件等。处理模块可以周期性地轮询(polling)各种周边装置与界面，使得计算设备能够通过各种周边装置与界面进行数据的输入与输出，也能够与具有上面描述的硬件元件的另一个计算设备进行通信。
以下先以图1本发明所提的由组合传送的多个串流取出特定串流播放的系统架构图来说明本发明的系统运作。如图1所示，本发明的系统含有第一信号源110、阵列服务器151、串流服务器153、客户端160，及可附加的第二信号源130。其中，第一信号源110、第二信号源130、串流服务器153可以通过实体连接线或无线通信的方式与阵列服务器151连接，且串流服务器153与客户端160之间可以通过无线网络连接，但本发明并不以此为限。
第一信号源110负责输出第一影音串流、第二信号源130可以输出第二影音串流。一般而言，在本发明中，信号源(第一信号源110、第二信号源130)所输出的影音串流(第一影音串流、第二影音串流)通常是由信号源自身产生，例如，信号源可以持续撷取外部影像以产生连续的帧(frame)，并将所产生的帧依据被产生的先后顺序依序输出为影音串流，但本发明并不以此为限。
一般而言，信号源可以是能够撷取影像与声音并将所撷取的影像与声音转换为影音信号并输出的计算设备，如摄影机等，但本发明并不以此为限，举例来说，信号源也可以是能够储存影音文件并可以将所储存的影音文件传送给阵列服务器151的计算设备。其中，信号源可以使用现有的各种协议将影音信号/影音文件传送给阵列服务器151，例如使用Network Device Interface(NDI)协议，但本发明并不以此为限。
要说明的是，在本发明中，可以包含两个或两个以上的第一信号源110，也可以包含两个或两个以上的第二信号源130。一般而言，第二信号源130的数量通常可以与第一信号源110的数量相同，且每一个不同的第一信号源110可以与不同的第二信号源130对应，即第一信号源110与第二信号源130可以是一对一的对应关系。
阵列服务器151负责接收第一信号源110所输出的第一影音串流，并负责分别由所接收到的每一个第一影音串流中取出一个不同的第一单视角帧， 及将从不同第一影音串流所取出的各个第一单视角帧组合为一个第一多视角帧。
更详细的，阵列服务器151可以先产生如图2所示的M*N个区块的多视角帧200，且多视角帧200中的区块的数量大于或等于第一信号源110的数量。其中，M、N大于1。在大部分的实施例中，M、N为相同的数值，且为2的次方数，但本发明并不以此为限，也就是说，M、N除了可以是2、4等数值的外，也可以是3或5等，且M、N并不一定相同。
阵列服务器151也可以依据第一信号源110的来源识别数据定义所取得的各个第一单视角帧与多视角帧200中的各个区块的对应关系，并可以将所取得的各个第一单视角帧置于多视角帧200中的相对应的区块中以产生第一多视角帧。其中，来源识别数据为可以表示特定信号源的数据，通常是由特定数量的文字、字母、数字、符号以特定方式排列而成，例如信号源所使用的网络地址、信号源所连接的界面编号、信号源的产品序号、预先被设定在信号源上的特定数据等，但本发明并不以此为限。
举例来说，阵列服务器151可以依据各个信号源(第一信号源110)的来源识别数据排列各个信号源，并可以将排列后的信号源所产生的帧(第一单视角帧)以先左后右由上到下的顺序依序定义各第一单视角帧与多视角帧200中的各区块的对应关系。例如，在某些实施例中，存在四个第一信号源(111～114)，若阵列服务器151可以定义2*2个区块(以左上、左下、右上、右下的位置排列)，并可以依据第一信号源(111～114)的来源识别数据排列出的各第一信号源的顺序为第一信号源114、第一信号源113、第一信号源112、第一信号源111，则第一信号源114所产生的第一单视角帧可以与位置在左上角的区块对应、第一信号源113所产生的第一单视角帧可以与位在右上角的区块对应、第一信号源112所产生的第一单视角帧可以与左下角的区块对应、第一信号源111所产生的第一单视角帧可以与右下角的区块对应。
在部分的实施例中，若第一单视角帧的解析度与多视角帧200中的区块的解析度不同，则阵列服务器151也可以调整(放大或缩小)第一单视角帧的解析度，使得第一单视角帧的解析度与多视角帧200中的区块的解析度相同。
事实上，阵列服务器151并不只执行一次产生一个第一多视角帧的动作，而是可以持续不断的重复执行上述动作，借以产生一连串的第一多视角帧。也就是说，阵列服务器151在产生一个第一多视角帧后，可以继续由接收从不同信号源的每一个第一影音串流中取出另一个不同的第一单视角帧(通常是排列在现有所取出的第一单视角帧的后且与现有所取出的第一单视角帧相邻的另一个第一单视角帧)，并可以将再次取出的另一个不同的第一单视角帧组合为另一个不同的第一多视角帧，如此不断重复，直到没有接收到任何信号源所传送的第一影音串流为止。
阵列服务器151也负责产生第一多源串流。阵列服务器151所产生的第一多源串流中包含阵列服务器151所产生的第一多视角帧，且第一多源串流中的各个第一多视角帧依照阵列服务器151产生的顺序排列。
阵列服务器151也负责将所产生的第一多源串流传送给串流服务器153。一般而言，阵列服务器151可以使用即时消息协议(Real-Time Messaging Protocol,RTMP)将第一多源串流传送给串流服务器153，但本发明并不以此为限，凡可以用来传送影音串流的方式都可以在阵列服务器151与串流服务器153之间被使用。其中，阵列服务器151通常可以在每产生一个多视角帧时，便将所产生的多视角帧传送给串流服务器153。
阵列服务器151也可以如上述产生第一多源串流的过程，将所有第二信号源130所输出的第二影音串流组成第二多源串流，并可以将所产生的第二多源串流传送到串流服务器153。也就是说，阵列服务器151可以接收每个第二信号源130所输出的第二多源串流，并可以分别持续由各个第二多源串 流中取出一个第二单视角帧，及可以持续将所有被取出的第二单视角帧组合为一个第二多视角帧，并可以产生包含依序产生的第二多视角帧的第二多源串流。
串流服务器153可以接收阵列服务器151所产生的第一多源串流。若阵列服务器151也有产生第二多源串流，则串流服务器153也可以接收阵列服务器151所产生的第二多源串流。
串流服务器153也负责提供客户端160连接，并负责将第一多源串流提供给请求第一多源串流的客户端160，在部分的实施例中，串流服务器153也可以将第二多源串流与第一多源串流一同提供给请求第一多源串流的客户端160。
客户端160通常是手机、平板电脑等方便使用者携带的装置，但本发明并不以此为限。
客户端160负责连接至串流服务器153，并负责向串流服务器153请求第一多源串流，借以接收串流服务器153所提供的第一多源串流。客户端160也可以在接收第一多源串流时一并接收来自串流服务器153的第二多源串流。其中，客户端160可以使用HTTP Live Streaming的传输协议(HLS)接收第一多源串流与第二多源串流，但本发明并不以此为限，凡可以传输影音信号的传输方式都可以在串流服务器153与客户端160之间被使用。
客户端160也负责由第一多源串流中取出一个第一多视角帧，并负责由所取出的第一多视角帧中取得一个第一单视角帧，及负责显示被取得的第一单视角帧。实务上，客户端160可以持续的依据第一多源串流所包含的各个第一多视角帧的顺序，由第一多源串流中的每一个第一多视角帧取出位置相同(相同区块)的第一单视角帧并依序显示所取出的第一单视角帧，使得包含所取出的第一单视角帧的第一影音串流被播放。在本发明中，客户端160可以依据预定值取出在对应位置(区块)中的第一单视角帧，其中，预定值 可以对应特定角落(如左上角或右上角)的区块、对应位于中央的区块、或对应排列在总区块数一半的区块(如在有16个区块时排列在第8个位置的区块)等，但本发明并不以此为限。
另外，客户端160在持续由第一多源串流中取出第一多视角帧并由第一多视角帧取得并播放第一单视角帧的同时，第一切换信号或第二切换信号或暂停控制信号可能随着客户端160的使用者对客户端160进行特定操作而被产生。其中，第一切换信号/第二切换信号可以是客户端160的使用者在如图4的客户端160的触控屏幕400的主显示区域410上朝移动方向滑动一段移动距离所产生的触控信号，也可以是使用者将客户端160朝移动方向或角度转动所产生的转动信号，但产生第一切换信号/第二切换信号的特定操作并不以上述为限；暂停控制信号可以是使用者在主显示区域410上单击或双击所产生的触控信号，但本发明也不以上述为限。
客户端160可以在第一切换信号被产生时，由第一多源串流所取出的第一多视角帧中取得与第一切换信号对应的另一个不同的第一单视角帧，并显示所取得的另一个第一单视角帧，使包含与第一切换信号对应的第一单视角帧的另一个不同的第一影音串流(在本发明中也以与第一切换信号对应的第一影音串流表示)被播放。
在部分的实施例中，若所有第一信号源110以一定顺序排列，则客户端160可以依据与第一切换信号所表示的移动方向对应的方向，由第一多视角帧中取得排列在与输出当前被显示的第一单视角帧的第一信号源110相距一定间隔的另一个第一信号源110所输出的第一单视角帧。
例如，若与当前被显示的第一单视角帧对应的区块为排列在第九个位置的区块231(图2参照)，且第一多视角帧中的各区块的排列顺序为先左后右由上往下(也就是在两个相邻的区块中，在左/上的区块的排列顺序比在右/下的区块为先)，则当第一切换信号为向左滑动的触控信号时，第一切换信号 即与排列在第八个位置的区块224对应，即客户端160可以改为持续由第一多视角帧中取得置于区块224中的第一单视角帧，而当第一切换信号为向右滑动的触控信号时，第一切换信号即与排列在第十个位置的区块232对应，客户端160可以改为持续由第一多视角帧中取得置于区块232中的第一单视角帧。相似的，若当前被显示的第一单视角帧与排列在第八个位置的区块224对应，则当第一切换信号为向左滑动的触控信号时，第一切换信号与排列在第七个位置的区块223对应，即客户端160可以改为持续由第一多视角帧中取得置于区块223中的第一单视角帧，而当第一切换信号为向右滑动的触控信号时，第一切换信号即与排列在第九个位置的区块231对应，客户端160可以改为持续由第一多视角帧中取得置于区块231中的第一单视角帧。
又如，第一切换信号所对应的区块也可以与第一切换信号所表示的移动方向与移动距离有关，也就是说，假设与当前被显示的第一单视角帧对应的区块为排列在第九个位置的区块231，且存在多个不同长度值的距离阈值，当第一切换信号为向左滑动的触控信号时，若第一切换信号的移动距离未符合第一距离阈值，则第一切换信号可以与排列在前一个位置(第八个)的区块224对应，若第一切换信号的移动距离仅符合第一距离阈值，则第一切换信号可以与排列在前二个位置(第七个)的区块223对应，若第一切换信号的移动距离符合第一距离阈值与第二距离阈值，则第一切换信号可以与排列在前三个位置(第六个)的区块222对应，依此类推，相似的，当第一切换信号为向右滑动的触控信号时，若第一切换信号的移动距离未符合第一距离阈值，则第一切换信号可以与排列在后一个位置(第十个)的区块232对应，若第一切换信号的移动距离仅符合第一距离阈值，则第一切换信号可以与排列在后两个位置(第十一个)的区块233对应，若第一切换信号的移动距离符合第一距离阈值与第二距离阈值，则第一切换信号可以与排列在后三个位置(第十二个)的区块234对应，依此类推。
客户端160也可以在暂停控制信号被产生时，暂停由第一多源串流中取出的第一多视角帧，也可以暂停由第一多视角帧中取得第一单视角帧，或可以不显示由第一多视角帧中取得的第一单视角帧，例如可以持续显示暂停控制信号被产生前的最后取得的第一单视角帧，使第一影音串流暂停播放。客户端160也可以在暂停控制信号被产生后，播放控制信号未被产生前，第一切换控制信号被产生时，由包含当前被显示的第一单视角帧的多视角帧中取出与第一切换信号对应的另一个不同的第一单视角帧，并显示被取出的另一个单视角帧。其中，客户端160同样可以上述的方式，依据第一切换控制信号所表示的移动方向或移动方向与移动距离选择由取出当前被显示的第一单视角帧的多视角帧中取出不同的第一单视角帧。客户端160也可以在暂停控制信号被产生后，播放控制信号被产生时，依据预设值选择继续显示由当前所接收到的第一多源串流中取出的第一多视角帧，或选择由暂停控制信号被产生后接收到的第一多源串流中取出的第一多视角帧开始显示，本发明没有特别的限制。
客户端160也可以在第一切换信号被产生后，将预定值设定为与被产生的第一切换信号所对应的区块对应。如此，在客户端160继续由第一多源串流取出的第一多视角帧中取得第一单视角帧时，便是由与第一切换信号所对应的区块中取得第一单视角帧，也就是持续由第一多视角帧中的位在同一个位置的区块取得第一单视角帧。
客户端160也可以在第二切换信号被产生时，改为持续由第二多源串流所取出的第二多视角帧中取得与当前被显示的第一单视角帧对应的第二单视角帧，并显示所取得的第二单视角帧，使包含第二单视角帧的第二影音串流(在本发明中也以与第二切换信号对应的第二影音串流表示)被播放。一般而言，与当前被显示的第一单视角帧对应的第二单视角帧即为与在第二多视角帧中位置与当前被显示的第一单视角帧对应的区块在第一多视角帧中的位置相同(排列顺序相同)的区块对应的第二单视角帧。例如，若与当前被显 示的第一单视角帧对应的区块排列在第一多视角帧中的第九个位置，则与当前被显示的第一单视角帧对应的第二单视角帧相对应的区块也排列在第二多视角帧中的第九个位置。
在本发明中，也可能并不只一组第二信号源130，当有多组第二信号源130时，第二切换信号所表示的移动距离可以让第二切换信号对应到不同组的第二信号源130，例如，若有两组设置在第一信号源110的平面上方的第二信号源130，则客户端160可以有一个距离阀值，当第二切换信号所表示的移动距离未超过距离阀值时，第二切换信号可以对应到与第一信号源110距离较近的一组第二信号源130，当第二切换信号所表示的移动距离超过距离阀值时，第二切换信号可以对应到与第一信号源110距离较远的另一组第二信号源130。
接着以第一实施例来解说本发明的运作系统与方法，并请参照图3本发明所提的由组合传送的多个串流取出特定串流播放的方法流程图。在本实施例中，假设第一信号源110为摄影机，且第一信号源110共有十六个，但本发明并不以此为限。
首先，在使用本发明提供串流服务前，需要先架设第一信号源110。在本实施例中，假设第一信号源(111～126)被架设在表演舞台的周围，且第一信号源(111～126)以一个圆形(或弧形)的轨迹被架设在同一个水平面上，如图5所示，其中，第一信号源(111～126)的网络地址分别为172.0.0.1～16，且第一信号源(111～126)依网络地址的顺序由小至大排列，也就是说，网络地址为172.0.0.1的第一信号源111与网络地址为172.0.0.2的第一信号源112及172.0.0.16的第一信号源126相邻、网络地址为172.0.0.2的第一信号源112与网络地址为172.0.0.1的第一信号源111及172.0.0.3的第一信号源113相邻、…、网络地址为172.0.0.15的第一信号源125与网络地址为172.0.0.14的第一信号源124及172.0.0.16的第一信号源126相邻、网络地址为172.0.0.16 的第一信号源126与网络地址为172.0.0.15的第一信号源125及172.0.0.1的第一信号源111相邻。
在完成第一信号源110的架设后，第一信号源110可以开始拍摄表演舞台上的影像，并可以将所拍摄到的影像以影音串流(串流媒体)的方式传送给阵列服务器151。在本实施例中，假设第一信号源(111～126)所产生的第一影音串流的解析度可以是960*540，且第一信号源(111～126)可以通过NDI协议将第一影音串流传送给阵列服务器151。
在阵列服务器151接收到第一信号源110所传送的第一影音串流(步骤310)后，阵列服务器151可以持续由各个第一影音串流中分别取出一个不同的第一单视角帧，阵列服务器151也可以将每一次由各个第一影音串流取出的各个不同的第一单视角帧组合为一个第一多视角帧，并可以将包含第一多视角帧以第一多源串流传送给串流服务器153(步骤330)。在本实施例中，假设阵列服务器151可以产生解析度为3840*2160的第一多视角帧，并在第一多视角帧中定义如图2所示的4*4个区块(每个区块的解析度为960*540)，及依据第一信号源(111～126)的网络地址(来源识别数据)定义各个由不同第一影音串流中所取出的第一单视角帧与第一多视角帧中的各区块的对应关系。若阵列服务器151定义4*4的区块以先左后右由上至下的顺序排列，则阵列服务器151可以定义网络地址为172.0.0.1的摄影机(第一信号源111)所产生的第一影音串流中的第一单视角帧对应区块211、网络地址为172.0.0.2的摄影机(第一信号源112)所产生的第一影音串流中的第一单视角帧对应区块212、网络地址为172.0.0.3的摄影机(第一信号源113)所产生的第一影音串流中的第一单视角帧对应区块213、…、网络地址为172.0.0.5的摄影机(第一信号源115)所产生的第一影音串流中的第一单视角帧对应区块221、…、网络地址为172.0.0.13的摄影机(第一信号源123)所产生的第一影音串流中的第一单视角帧对应区块241、…、网络地址为172.0.0.15的摄影机(第一信号源125)所产生的第一影音串流中的第一单视角帧对应区块243、 网络地址为172.0.0.16的摄影机(第二信号源126)所产生的第一影音串流中的第一单视角帧对应区块244，接着，阵列服务器151可以将各个第一单视角帧置入相对应的区块(211～244)中，如此，阵列服务器151可以产生包含十六个第一单视角帧的一个第一多视角帧，进而可以产生包含多个第一多视角帧的第一多源串流，及可以使用推流的方式通过即时消息协议将第一多源串流传送给串流服务器153。
在串流服务器153接收到阵列服务器151所产生第一多源串流后，客户端160可以连接到串流服务器153，并可以向串流服务器153请求第一多源串流。串流服务器153在接收到客户端160所发出的请求后，可以将第一多源串流传送给客户端160，使得客户端160可以接收第一多源串流(步骤350)。在本实施例中，假设客户端160可以使用拉流的方式，通过HLS协议接收第一多源串流。
在客户端160接收到多源串流后，客户端160可以持续的由所接收到的第一多源串流中取出第一多视角帧，并可以持续的由每一个第一多视角帧中取得第一单视角帧，及可以持续显示所取得的第一单视角帧，使得第一影音串流被播放(步骤360)。在本实施例中，假设预设值表示取得排列在第一个位置的区块中的单视角帧，则客户端160可以由左上角开始计算(若左上角的像素在第一多视角帧中的座标为(1,1))，对每一个第一多视角帧中取出X轴为第1到第960个像素且Y轴为第1到540个像素的影像(被取出的影像即为第一单视角帧)，再显示所取出的影像(第一单视角帧)，如此，第一信号源111所拍摄产生的第一影音串流便可以在客户端160上被播放。其中，客户端160可以将第一多源串流的每秒帧数(frames per second,FPS)作为由第一多源串流中取出第一多视角帧的频率。
在客户端160播放第一影音串流时，若第一切换信号被产生，则客户端160可以持续由接收到的多源串流中改为取出另一个第一单视角帧，并可以持续显示所取得的第一单视角帧，使得另一个第一影音串流被播放(步骤 370)。在本实施例中，假设第一切换信号为客户端160的使用者的手指在客户端160的触控屏幕400的主显示区域410(图4参照)上向右滑动而产生的触控信号，若在主显示区域410上左右滑动的触控信号可以改变当前被播放的影音串流的视角，则客户端160可以在第一切换信号被产生后，依据向右滑动的触控信号(第一切换信号)选择与输出当前被播放的第一影音串流的第一信号源111位置相邻且排列在第一信号源111之后(假设为逆时针方向)的第一信号源112所输出的第一影音串流，也就是说，客户端160可以由左上角开始计算，改为持续对每一个第一多源帧中取出X轴为第961到第1920个像素且Y轴为第1到540个像素的影像(被取出的影像即为第一信号源112所产生的第一单视角帧)，再显示所取出的第一单视角帧，如此，第一信号源112所拍摄产生的第一影音串流便可以在主显示区域410上被播放。
相似的，若在客户端160播放第一信号源112所拍摄产生的第二影音串流时，若向右滑动的触控信号(第一切换信号)再次被产生，则客户端160同样可以持续由接收到的第一多源串流中取出第一多视角帧，并可以持续的由每一个第一多视角帧中取得与第一切换信号对应的另一个第一信号源113所产生的第一单视角帧，也就是对每一个第一多视角帧中取出X轴为第1921到第2880个像素且Y轴为第1到540个像素的影像，及可以持续显示所取得的影像，使得第一信号源113所拍摄产生的第一影音串流被播放；而若再次被产生的第一切换信号为向左滑动的触控信号，则客户端160可以持续由接收到的第一多源串流中取出第一多视角帧，并可以持续的由每一个第一多视角帧中取得与第一切换信号对应的第一信号源111所产生的第一单视角帧，也就是对每一个第一多视角帧中取出X轴为第1到第960个像素且Y轴为第1到540个像素的影像，及可以持续显示所取得的影像，使得第一信号源111所拍摄产生的第一影音串流被播放。
另外，客户端160也可以在第一切换信号被产生后，选择与输出当前被播放的第一影音串流的第一信号源111在与移动方向对应且位置排列在与移 动距离对应的间隔位置的另一个第一信号源所输出的第一影音串流，若向右的移动方向表示以逆时针方向选择第一信号源110，也就是表示选择相对应的区块排列在后的第一单视角帧，当移动距离未大于第一距离阈值时，客户端160可以在主显示区域410中改为播放第一信号源112所输出的第一影音串流，也就是说，客户端160可以由左上角开始计算，改为持续对每一个第一多视角帧中取出X轴为第961到第1920个像素且Y轴为第1到540个像素的影像(被取出的影像即为第一信号源112所输出的第一单视角帧)，再于主显示区域410中显示所取出的第一单视角帧；而当移动距离大于第一距离阈值但未大于第二距离阈值时，客户端160可以改为在主显示区域410中播放第一信号源113所输出的第一影音串流，即客户端160可以改为持续由第一多视角帧中取出X轴为第961到第1920个像素且Y轴为第1到540个像素的影像；…；当移动距离大于第三距离阈值但未大于第四距离阈值时，客户端160可以改为在主显示区域410中播放第一信号源115所输出的第一影音串流，即客户端160可以改为持续由第一多视角帧中取出X轴为第1到第960个像素且Y轴为第541到1080个像素的影像，…；依此类推。
如此，通过本发明，阵列服务器151可以将多个信号源所输出的影音串流以较大的解析度组合为包含所有影音串流的多源串流后再由串流服务器153传送给客户端160，使得客户端160可以在本地(local)切换播放不同信号源所输出的影音串流，借以在几乎没有延迟时间的情况下切换播放不同视角的影音串流。
上述第一实施例中，在客户端160播放第一影音串流时，若暂停控制信号先被产生，则客户端160可以暂停由第一多源串流中取出的第一多视角帧，也可以暂停由第一多视角帧中取得第一单视角帧，并可以持续显示暂停控制信号被产生前最后取得的第一单视角帧，使第一影音串流暂停播放。客户端160也可以在暂停控制信号被产生后，第一切换控制信号被产生时，由包含当前被显示的第一单视角帧的多视角帧中取出与第一切换信号对应的另一个 不同的第一单视角帧，并显示被取出的另一个单视角帧。例如，客户端160可以随着第一切换信号的每一次产生，改由排列位置在前或后的区块中取出并显示第一单视角帧；或可以随着第一切换信号所表示的移动方向与移动距离选择由排列位置在前或后一定间隔的区块中取出并显示第一单视角帧。
继续以第二实施例来解说本发明的运作系统与方法，并请参照图6本发明所提的另一种由组合传送的多个串流取出特定串流播放的方法流程图。在本实施例中，假设如图7所示，包含第一信号源110与第二信号源130等两组信号源，第一信号源110、第二信号源130为被架设在表演舞台周围的摄影机，且第一信号源110与第二信号源130各有十六个，但本发明并不以此为限。其中，第一信号源(111～126)的网络地址为172.0.0.1～16、第二信号源(131～146)的网络地址为172.0.1.1～16，且第一信号源(111～126)依网络地址的顺序被环形设置在离表演舞台约150公分的平面上、第二信号源源(131～146)依网络地址的顺序被环形设置在离表演舞台约200公分的另一个平面上，同时，网络地址为172.0.1.1的第二信号源131与网络地址为172.0.0.1的第一信号源111对应、网络地址为172.0.1.2的第二信号源132与网络地址为172.0.0.2的第一信号源112对应、…、依此类推。
在完成第一信号源110、第二信号源130的设置后，第一信号源110、第二信号源130可以开始拍摄表演舞台上的影像，并可以将所拍摄到的影像以影音串流的方式传送给阵列服务器151。在本实施例中，假设所有信号源(第一信号源110、第二信号源130)所产生的影音串流(第一影音串流、第二影音串流)的解析度都是960*540，且所有信号源都可以通过NDI协议将影音串流传送给阵列服务器151。
在阵列服务器151接收到第一信号源110所传送的第一影音串流与第二信号源130所传送的第二影音串流(步骤610)后，阵列服务器151可以如第一实施例中的描述，持续由各个第一影音串流中分别取出一个第一单视角帧，并可以将每一次取出的不同第一单视角帧组合为一个第一多视角帧，并 可以将第一多视角帧以第一多源串流传送给串流服务器153，另外，阵列服务器151也可以由各第一影音串流产生第一多源串流相同的过程，持续由各个第二影音串流中分别取出一个第二单视角帧，并可以将每一次取出的不同第二单视角帧组合为一个第二多视角帧，并可以将所产生的第二多视角帧以第二多源串流传送到串流服务器153(步骤630)。在本实施例中，假设阵列服务器151产生第一多源串流的过程不再赘述，另外，阵列服务器151同样可以产生如图2所示的包含4*4个解析度为960*540的区块的第二多视角帧，及可以依据第二信号源(131～146)的网络地址(来源识别数据)定义每一个第二单视角帧与各个区块的对应关系，或可以依据第二信号源130所对应的第一信号源110所产生的第一单视角帧在第一多视角帧中的位置定义相对应的第二信号源130所产生的第二单视角帧与各区块的对应关系。也就是说，若阵列服务器151定义区块以先左后右由上至下的顺序排列，则阵列服务器151可以定义网络地址为172.0.1.1的摄影机(第二信号源131)所产生的第二影音串流中的第二单视角帧对应区块211、网络地址为172.0.1.2的摄影机(第二信号源132)所产生的第二影音串流中的第二单视角帧对应区块212、…、网络地址为172.0.1.16的摄影机所产生的第二影音串流中的第二单视角帧对应区块244，接着，阵列服务器151可以将各个第二单视角帧置入相对应的区块中，如此，阵列服务器151可以产生包含十六个第二单视角帧的第二多视角帧，并可以产生包含第二多视角帧的第二多源串流，及可以使用推流的方式通过即时消息协议将第一多源串流与第二多源串流传送给串流服务器153。
在串流服务器153接收到阵列服务器151所传送的第一多源串流与第二多源串流后，客户端160可以连接到串流服务器153，并可以向串流服务器153请求第一多源串流。串流服务器153在接收到客户端160所发出的请求后，可以将第一多源串流与第二多源串流传送给客户端160，使得客户端160 接收第一多源串流与第二多源串流(步骤650)。在本实施例中，假设客户端160可以使用拉流的方式，通过HLS协议接收第一多源串流与第二多源串流。
在客户端160接收到第一多源串流与第二多源串流后，客户端160可以持续的由所接收到的第一多源串流中取出第一多视角帧，并可以持续的由每一个第一多视角帧中取得第一单视角帧，及可以持续显示所取得的第一单视角帧，使得第一影音串流被播放(步骤660)。在本实施例中，假设预设值表示取得排列在第一个位置的区块中的单视角帧，则客户端160可以由左上角开始计算，对每一个第一多视角帧中取出X轴为第1到第960个像素且Y轴为第1到540个像素的影像(被取出的影像即为第一单视角帧)，再显示所取出的影像(第一单视角帧)，如此，第一信号源111所拍摄产生的第一影音串流便可以在客户端160上被播放。
在客户端160播放第一影音串流时，若第一切换信号被产生，则客户端160可以如第一实施例所述，持续由接收到的第一多源串流中取出第一多视角帧，并可以持续的由每一个第一多视角帧中取得与第一切换信号对应的另一个不同的第一单视角帧，及可以持续显示所取得的第一单视角帧，使得另一个第一影音串流被播放；而若第二切换信号被产生，则客户端160可以改为由与第二切换信号对应的第二多源串流中取出第二多视角帧，并可以持续的由每一个第二多视角帧中取得与当前播放的第一单视角帧对应的第二单视角帧，及可以持续显示所取得的第二单视角帧，使得第二影音串流被播放(步骤680)。在本实施例中，假设第二切换信号为客户端160的使用者的手指在客户端160的触控屏幕400的主显示区域410(图4参照)上向上滑动而产生的触控信号，若在主显示区域410上下滑动的触控信号可以改变当前所播放的影音串流的视角，客户端160可以在第二切换信号被产生后，依据向上滑动的触控信号(第二切换信号)选择与输出当前被播放的第一影音串流的第一信号源111对应的第二信号源131所输出的第二影音串流，也就是说，客户端160可以由左上角开始计算，对每一个第二单视角帧中取出X轴为第 1到第960个像素且Y轴为第1到540个像素的影像(被取出的影像即为第二信号源131所产生的第二单视角帧)，再显示所取出的第二单视角帧，如此，第二信号源131所拍摄输出的第二影音串流便可以在客户端160上被播放。
相似的，若在客户端160播放第二信号源131所拍摄产生的第二影音串流时，若向下滑动的触控信号(第二切换信号)被产生，则客户端160可以改为持续由与第二切换信号对应的第一多源串流中取出第一多视角帧，并可以持续的由每一个第一多视角帧中取得与输出当前播放的第二单视角帧的第二信号源131对应的第一信号源111所输出的第一单视角帧，也就是对每一个第一多视角帧中取出X轴为第1到第960个像素且Y轴为第1到540个像素的影像，及可以持续显示所取得的影像，使得第一信号源111所拍摄产生的第一影音串流被播放；另外，若是被产生的触控信号为第一切换信号而非第二切换信号，则客户端160可以持续由接收到的第二多源串流中取出第二多视角帧，并可以持续的由每一个第二多视角帧中取得与第一切换信号对应的第二信号源132所产生的第二单视角帧，也就是对每一个第二多视角帧中取出X轴为第961到第1920个像素且Y轴为第1到540个像素的影像，及可以持续显示所取得的影像，使得第二信号源132所拍摄产生的第二影音串流被播放。其中，第一切换信号并不限于与第二信号源132对应，客户端160也可以依据第一切换信号所表示的移动方向与移动距离而与其他非第二信号源131的第二信号源(133～146)对应，并可以持续由第二多视角帧中取出相对应的位置的影像并显示。
综上所述，可知本发明与现有技术之间的差异在于具有阵列服务器将多个信号源所输出的影音串流组合为一个包含所有影音串流的多源串流并将组合产生的多源串流提供给客户端，客户端由所接收到的多源串流中取出欲播放的影音串流的技术手段，借由此技术手段可以来解决现有技术所存在观赏直播时切换视角需要反应时间的问题，进而达成降低直播的服务服务器的负载的技术功效。
此外，本发明的由组合传送的多个串流取出特定串流播放的方法，可实现于硬件、软件或硬件与软件的组合中，也可在电脑系统中以集中方式实现或以不同元件散布于若干互连的电脑系统的分散方式实现。
虽然本发明所公开的实施方式如上，但所述的内容并非用以直接限定本发明的专利保护范围。任何本发明所属技术领域中的技术人员，在不脱离本发明所公开的精神和范围的前提下，对本发明的实施的形式上及细节上作些许的更动润饰，均属于本发明的专利保护范围。本发明的专利保护范围，仍须以所附的权利要求书所界定的范围为准。

Claims (10)

  1. 一种由组合传送的多个串流取出特定串流播放的方法，该方法至少包含下列步骤：
    阵列服务器接收多个第一信号源所输出的各第一影音串流；
    该阵列服务器产生包含多个第一多视角帧的第一多源串流，并将该第一多源串流传送至串流服务器，其中，该阵列服务器是由每一该第一影音串流中分别取出第一单视角帧，并将各该第一单视角帧组合为该第一多视角帧；
    客户端连接至该串流服务器，并接收该第一多源串流；
    该客户端持续由该些第一多视角帧中取得同一该第一信号源所产生的该第一单视角帧，并显示该被取得的第一单视角帧，使该第一影音串流被播放；及
    该客户端于第一切换信号被产生时，改为持续由该些第一多视角帧中取出与该第一切换信号对应的另一该第一信号源所产生的该第一单视角帧，并显示被取出的另一该第一单视角帧，使另一该第一影音串流被播放。
  2. 根据权利要求1所述的由组合传送的多个串流取出特定串流播放的方法，其中该方法于该第一影音串流被播放的步骤后，还包含该客户端于暂停控制信号被产生后该第一切换信号被产生时，由包含当前被显示的该第一单视角帧的该多视角帧中取出与该第一切换信号对应的另一该第一单视角帧，并显示被取出的另一该第一单视角帧。
  3. 根据权利要求1所述的由组合传送的多个串流取出特定串流播放的方法，其中该第一切换信号被产生的步骤为该客户端产生朝移动方向滑动一移动距离的触控信号；且该客户端改为持续由该些多视角帧中取出与该第一切换信号对应的另一该第一信号源所产生的该第一单视角帧的步骤为该客户端 取得与发出当前被显示的该第一单视角帧的该第一信号源在与该移动方向的对应方向且排列在与该移动距离对应的间隔位置的另一该第一信号源所输出的另一该第一单视角帧。
  4. 根据权利要求1所述的由组合传送的多个串流取出特定串流播放的方法，其中该阵列服务器将各该第一单视角帧组合为该第一多视角帧的步骤为该阵列服务器产生包含M*N个区块的该第一多视角帧，并依据各该第一信号源的来源识别数据分别将每一该第一单视角帧置于相对应的该区块中。
  5. 根据权利要求1所述的由组合传送的多个串流取出特定串流播放的方法，其中该方法还包含该阵列服务器接收多个第二信号源所输出的各第二影音串流，并持续由每一该第二影音串流中分别取出第二单视角帧，及将该些第二单视角帧组合为第二多视角帧，并将各该第二多视角帧以第二多源串流传送到该串流服务器，该客户端于接收该第一多源串流时一并接收该第二多源串流，及于第二切换信号被产生时，改为持续由与该第二切换信号对应的该第二多源串流所包含的该第二多视角帧中取出与当前被显示的该单视角帧对应的该第二单视角帧，并显示被取出的该第二单视角帧的步骤。
  6. 一种由组合传送的多个串流取出特定串流播放的系统，该系统至少包含：
    多个第一信号源，每一该第一信号用以输出第一影音串流；
    串流服务器；
    阵列服务器，用以接收各该第一影音串流，及用以产生包含多个第一多视角帧的第一多源串流，并将该第一多源串流传送至该串流服务器，其中，该阵列服务器是由每一该第一影音串流中分别取出第一单视角帧，及将各该第一单视角帧组合为该第一多视角帧；及
    客户端，用以连接至该串流服务器，并接收该第一多源串流，及持续由该些第一多视角帧中取得同一该第一信号源所产生的该第一单视角帧，并显示该被取得的第一单视角帧，使该第一影音串流被播放，及用以于第一切换信号被产生时，改为持续由该些第一多视角帧中取出与该第一切换信号对应的另一该第一信号源所产生的该第一单视角帧，并显示被取出的另一该第一单视角帧，使另一该第一影音串流被播放。
  7. 根据权利要求6所述的由组合传送的多个串流取出特定串流播放的系统，其中该客户端还用以于暂停控制信号被产生后该第一切换信号被产生时，由取出当前被显示的该第一单视角帧的该多视角帧中取出与该第一切换信号对应的另一该第一单视角帧，并显示被取出的另一该单视角帧。
  8. 根据权利要求6所述的由组合传送的多个串流取出特定串流播放的系统，其中该客户端是在朝移动方向滑动一移动距离的触控信号被产生时判断该第一切换信号被产生，并取得与发出当前被显示的该第一单视角帧的该第一信号源在与该移动方向的对应方向且排列在与该移动距离对应的间隔位置的另一该第一信号源所输出的另一该第一单视角帧。
  9. 根据权利要求6所述的由组合传送的多个串流取出特定串流播放的系统，其中每一该第一多视角帧包含M*N个区块，该阵列服务器是依据各该第一信号源的来源识别数据分别将每一该第一单视角帧置于相对应的该区块中以产生该第一多视角帧。
  10. 根据权利要求6所述的由组合传送的多个串流取出特定串流播放的系统，其中该系统还包含多个第二信号源，每一该第二信号源分别用以输出第二影音串流，该阵列服务器还用以接收各该第二影音串流，并持续由每一该第二影音串流中取出第二单视角帧，及将该些第二单视角帧组合为第二多视角帧，并将各该第二多视角帧以第二多源串流传送到该串流服务器，该客户端还用以于接收该第一多源串流时一并接收该第二多源串流，及于第二切 换信号被产生时，改为持续由与该第二切换信号对应的该第二多源串流所包含的该第二多视角帧中取出与当前被显示的该第一单视角帧对应的该第二单视角帧，并显示该被取出的另一该第二单视角帧。
PCT/CN2021/139040 2020-12-30 2021-12-17 由组合传送的多个串流取出特定串流播放的系统及方法 WO2022143212A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2023563142A JP2024504513A (ja) 2020-12-30 2021-12-17 組み合わせて伝送される複数のストリーミングメディアから特定のストリーミングメディアを抽出し再生するシステムとその方法

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011620239.0 2020-12-30
CN202011620239.0A CN114697690A (zh) 2020-12-30 2020-12-30 由组合传送的多个串流取出特定串流播放的系统及方法

Publications (1)

Publication Number Publication Date
WO2022143212A1 true WO2022143212A1 (zh) 2022-07-07

Family

ID=82118352

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/139040 WO2022143212A1 (zh) 2020-12-30 2021-12-17 由组合传送的多个串流取出特定串流播放的系统及方法

Country Status (4)

Country Link
US (1) US11997327B2 (zh)
JP (1) JP2024504513A (zh)
CN (1) CN114697690A (zh)
WO (1) WO2022143212A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114697690A (zh) * 2020-12-30 2022-07-01 光阵三维科技有限公司 由组合传送的多个串流取出特定串流播放的系统及方法
CN115174943B (zh) * 2022-07-08 2023-10-31 叠境数字科技（上海）有限公司 一种边云协同及客户端自适应的自由视角播放方法及系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101300840A (zh) * 2005-11-04 2008-11-05 微软公司 多视角视频传递
CN101742096A (zh) * 2010-01-21 2010-06-16 中华电信股份有限公司 多视角互动电视系统与方法
WO2011039814A1 (ja) * 2009-09-30 2011-04-07 ヒューレット－パッカード デベロップメント カンパニー エル．ピー． マルチビュー・ストリームデータ制御システム・方法
CN102833584A (zh) * 2012-05-22 2012-12-19 中华电信股份有限公司 多视角互动电视的影音串流方法
CN111447461A (zh) * 2020-05-20 2020-07-24 上海科技大学 多视角直播视频的同步切换方法、装置、设备和介质

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3744002B2 (ja) * 2002-10-04 2006-02-08 ソニー株式会社 表示装置、撮像装置、および撮像／表示システム
US20120212579A1 (en) * 2009-10-20 2012-08-23 Telefonaktiebolaget Lm Ericsson (Publ) Method and Arrangement for Multi-View Video Compression
US9838686B2 (en) * 2013-07-12 2017-12-05 Intel Corporation Techniques for inclusion of thumbnail images in compressed video data
WO2016183812A1 (zh) * 2015-05-20 2016-11-24 北京诺亦腾科技有限公司 一种混合运动捕捉系统及方法
US10219026B2 (en) * 2015-08-26 2019-02-26 Lg Electronics Inc. Mobile terminal and method for playback of a multi-view video
CN109074629A (zh) * 2015-10-29 2018-12-21 Oy沃肯视觉有限公司 使用联网照相机对关注的区域进行视频摄像
KR20180039529A (ko) * 2016-10-10 2018-04-18 엘지전자 주식회사 이동 단말기 및 그의 동작 방법
CN107018370B (zh) * 2017-04-14 2020-06-30 威盛电子股份有限公司 用于视频墙的显示方法及其系统
US11087549B2 (en) * 2018-10-15 2021-08-10 University Of Maryland, College Park Methods and apparatuses for dynamic navigable 360 degree environments
CN111447503A (zh) * 2020-04-26 2020-07-24 烽火通信科技股份有限公司 一种多视点视频的视点切换方法、服务器和系统
CN114697690A (zh) * 2020-12-30 2022-07-01 光阵三维科技有限公司 由组合传送的多个串流取出特定串流播放的系统及方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101300840A (zh) * 2005-11-04 2008-11-05 微软公司 多视角视频传递
WO2011039814A1 (ja) * 2009-09-30 2011-04-07 ヒューレット－パッカード デベロップメント カンパニー エル．ピー． マルチビュー・ストリームデータ制御システム・方法
CN101742096A (zh) * 2010-01-21 2010-06-16 中华电信股份有限公司 多视角互动电视系统与方法
CN102833584A (zh) * 2012-05-22 2012-12-19 中华电信股份有限公司 多视角互动电视的影音串流方法
CN111447461A (zh) * 2020-05-20 2020-07-24 上海科技大学 多视角直播视频的同步切换方法、装置、设备和介质

Also Published As

Publication number Publication date
JP2024504513A (ja) 2024-01-31
US20220210486A1 (en) 2022-06-30
US11997327B2 (en) 2024-05-28
CN114697690A (zh) 2022-07-01

Similar Documents

Publication Publication Date Title
US10003741B2 (en) System for processing data from an omnidirectional camera with multiple processors and/or multiple sensors connected to each processor
US6539547B2 (en) Method and apparatus for electronically distributing images from a panoptic camera system
CN106992959B (zh) 一种3d全景音视频直播系统及音视频采集方法
KR102157634B1 (ko) 영상 획득 방법 및 로컬 엔드포인트 호스트 장치
WO2022143212A1 (zh) 由组合传送的多个串流取出特定串流播放的系统及方法
US9843725B2 (en) Omnidirectional camera with multiple processors and/or multiple sensors connected to each processor
JP2020519094A (ja) ビデオ再生方法、デバイス、およびシステム
RU2015143011A (ru) Устройство серверного узла и способ
WO2018068481A1 (zh) 一种双目 720 度全景采集系统
KR20150029461A (ko) 파노라마 영상의 스트리밍 서비스 제공 시스템에서 타일링 영상 동기화 방법
KR20180004068A (ko) 가상 카메라를 이용한 개인화된 쇼핑몰 시스템
US9445142B2 (en) Information processing apparatus and control method thereof
KR101311463B1 (ko) 원격 영상정보 제공시스템
JP2008109364A (ja) カメラサーバシステム、データの処理方法、およびカメラサーバ
TWM610558U (zh) 由組合傳送之多個串流取出特定串流播放之系統
US20230239525A1 (en) Server, method and terminal
TWI830978B (zh) 由組合傳送之多個串流取出特定串流播放之系統及方法
US11930290B2 (en) Panoramic picture in picture video
KR20140111324A (ko) 비디오 감시 방법, 관련 시스템, 관련 감시 서버, 및 관련 감시 카메라
JP2021078114A (ja) 撮影装置、配信システム、配信方法および記録媒体
US20210160563A1 (en) Method and apparatus for preview decoding for joint video production
TWI822158B (zh) 沉浸式串流影像擷取與成像之裝置及方法
US20240129564A1 (en) Bandwidth-adaptive light field video transmission on mobile and portable devices
JP7495903B2 (ja) Ａｖ伝送装置
CN109005453B (zh) 一种视频流播放控制方法及电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21913953

Country of ref document: EP

Kind code of ref document: A1

DPE1 Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101)
WWE Wipo information: entry into national phase

Ref document number: 2023563142

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21913953

Country of ref document: EP

Kind code of ref document: A1