WO2015053157A1 - 送信装置、送信方法および受信装置 - Google Patents
送信装置、送信方法および受信装置 Download PDFInfo
- Publication number
- WO2015053157A1 WO2015053157A1 PCT/JP2014/076334 JP2014076334W WO2015053157A1 WO 2015053157 A1 WO2015053157 A1 WO 2015053157A1 JP 2014076334 W JP2014076334 W JP 2014076334W WO 2015053157 A1 WO2015053157 A1 WO 2015053157A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- picture
- image data
- sequence
- display
- encoded image
- Prior art date
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 59
- 238000000034 method Methods 0.000 title description 37
- 238000012545 processing Methods 0.000 claims description 64
- 238000003780 insertion Methods 0.000 claims description 18
- 230000037431 insertion Effects 0.000 claims description 18
- 238000001514 detection method Methods 0.000 claims description 13
- 101001073193 Homo sapiens Pescadillo homolog Proteins 0.000 description 81
- 102100035816 Pescadillo homolog Human genes 0.000 description 81
- 230000002123 temporal effect Effects 0.000 description 31
- 230000008569 process Effects 0.000 description 30
- 230000007704 transition Effects 0.000 description 24
- 238000005516 engineering process Methods 0.000 description 20
- 230000008859 change Effects 0.000 description 19
- 238000012805 post-processing Methods 0.000 description 19
- 238000004458 analytical method Methods 0.000 description 13
- 101150028534 cpb-1 gene Proteins 0.000 description 9
- 239000000284 extract Substances 0.000 description 9
- 230000003139 buffering effect Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 230000010354 integration Effects 0.000 description 6
- 102100036848 C-C motif chemokine 20 Human genes 0.000 description 4
- 238000000605 extraction Methods 0.000 description 4
- 101150085553 cpb-2 gene Proteins 0.000 description 3
- 230000003111 delayed effect Effects 0.000 description 3
- 101100243456 Arabidopsis thaliana PES2 gene Proteins 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 101001139126 Homo sapiens Krueppel-like factor 6 Proteins 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/136—Incoming video signal characteristics or properties
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/188—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a video data packet, e.g. a network abstraction layer [NAL] unit
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
- H04N19/31—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability in the temporal domain
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/46—Embedding additional information in the video signal during the compression process
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/23424—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234327—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by decomposing into layers, e.g. base layer and one or more enhancement layers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234381—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
- H04N21/2362—Generation or processing of Service Information [SI]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
- H04N21/2365—Multiplexing of several video streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/633—Control signals issued by server directed to the network components or client
- H04N21/6332—Control signals issued by server directed to the network components or client directed to client
- H04N21/6336—Control signals issued by server directed to the network components or client directed to client directed to decoder
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
- H04N21/845—Structuring of content, e.g. decomposing content into time segments
- H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/38—Transmitter circuitry for the transmission of television signals according to analogue transmission standards
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/44—Receiver circuitry for the reception of television signals according to analogue transmission standards
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
- H04N7/0127—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level by changing the field or frame frequency of the incoming video signal, e.g. frame rate converter
Definitions
- the present technology relates to a transmission device, a transmission method, and a reception device. Specifically, the present technology relates to a transmission apparatus that transmits image data of each picture constituting moving image data by hierarchical encoding.
- Receivers are expensive to support high frame frequency services, which is an impediment to early adoption. Only low-cost receivers dedicated to low frame frequency services are prevailing at the beginning, and if the service side starts high frame frequency services in the future, it will be impossible to view without a new receiver, and new services will spread. It becomes an obstruction factor.
- temporal scalability is proposed by hierarchically encoding image data of each picture constituting moving image data (see Non-Patent Document 1).
- the layer of each picture can be identified based on the temporal ID (temporal_id) inserted in the header of the NAL (Network Abstraction Layer) unit, and selective decoding up to the layer corresponding to the decoding capability becomes possible. .
- the purpose of this technique is to enable a good display on the receiving side even when the frame rate changes dynamically.
- the concept of this technology is Classifying the image data of each picture constituting the moving image data into a plurality of layers, encoding the image data of the classified pictures of each layer, and dividing the plurality of layers into a predetermined number of layer sets;
- An image encoding unit that generates the predetermined number of video streams each having encoded image data of each of the divided groups of pictures;
- the image encoding unit performs encoding so that at least a decoding interval of encoded image data of pictures in the lowest layer set is a predetermined interval,
- a transmission unit that transmits a container of a predetermined format including a video stream having encoded image data of at least the lowest hierarchical set of pictures among the predetermined number of generated video streams;
- the sequence of the video stream to be transmitted has a switching unit that switches from the first sequence to the second sequence having a frame rate different from that of the first sequence, at least the code of the picture corresponding to the switching unit
- the transmission apparatus further includes an information insertion unit that inserts display control information into the
- the image encoding unit encodes the image data of each picture constituting the moving image data to generate a predetermined number of video streams.
- the image data of each picture constituting the moving image data is classified into a plurality of layers and encoded.
- the plurality of hierarchies are divided into a predetermined number of hierarchies, and a predetermined number of video streams each having the encoded image data of the pictures of the divided hierarchies are generated.
- the image encoding unit encoding is performed so that at least the decoding interval of the encoded image data of the pictures in the lowest layer set is a predetermined interval, for example, an equal interval.
- a predetermined interval for example, an equal interval.
- the image encoding unit encodes all of the pictures in the hierarchical groups whose decoding timing of the encoded image data of the pictures in the hierarchical group positioned higher than the lowest hierarchical group is lower than the hierarchical group.
- the encoding may be performed so as to be an intermediate timing of the decoding timing of the image data.
- the intermediate timing mentioned here does not necessarily need to be the middle timing, and means a timing that does not overlap with the lower decoding timing.
- the receiving side has the ability to decode not only the lowest layer set but also the encoded image data of a picture of a layer set higher than that, the decoding process of each picture is performed. It becomes possible to proceed smoothly in sequence.
- the transmission unit transmits a container of a predetermined format including a video stream having encoded image data of at least the lowest hierarchical set of pictures among the predetermined number of video streams.
- the container may be a transport stream (MPEG-2 TS) adopted in the digital broadcasting standard.
- the container may be MP4 used for Internet distribution or the like, or a container of other formats.
- the switching unit corresponds to the switching unit.
- Display control information is inserted into the encoded image data of a picture to be processed or a packet for containerizing the encoded image data.
- the display control information controls to repeat the display of the last picture of the first sequence by a predetermined number, or the display period of the picture of the second sequence is displayed during the display period of the last picture of the first sequence.
- the information may be controlled to be skipped.
- the display control information may be display offset information for controlling the display timing of the second sequence of pictures.
- the container may be a transport stream, and the information insertion unit may insert the display control information into the extension field of the PES packet. Thereby, for example, using this display control information, it is possible to control the reading of the image data of each picture from the uncompressed data buffer.
- the sequence of the video stream to be transmitted has a switching unit that switches from the first sequence to the second sequence having a different frame rate from the first sequence, at least the switching is performed.
- the display control information is inserted into the encoded image data of the picture corresponding to the section, or the packet that containers the encoded image data, and is transmitted. Therefore, for example, the receiving side can control the reading of image data of each picture from the uncompressed data buffer based on this display control information, and even if the frame rate changes dynamically, a display gap is generated. A good display can be achieved without causing it to occur.
- the information insertion unit further inserts identification information that identifies the last picture of the first sequence into the encoded image data of the last picture of the first sequence. It may be made like. Thereby, when the sequence of the video stream to be transmitted has a switching unit that switches from the first sequence to the second sequence having a frame rate different from that of the first sequence, the receiving side is based on the identification information. Thus, it becomes possible to easily identify the last picture of the first sequence.
- identification information insertion that inserts identification information that identifies whether or not display control information is inserted into a container layer of encoded image data or a packet that containers the encoded image data It may be made to further comprise a part.
- the container is a transport stream
- the identification information insertion unit inserts the identification information as a descriptor in a video elementary stream loop arranged corresponding to a predetermined number of video streams under the program map table. You may be like.
- the receiving side can easily identify whether or not the display control information is inserted into the encoded image data based on the identification information without decoding the encoded image data.
- the picture data of each picture constituting the moving picture data is classified and encoded into a plurality of hierarchies, and the pictures of the respective hierarchies are obtained by dividing the plurality of hierarchies into a predetermined number of hierarchies.
- the sequence of the received video stream has a switching unit that switches from the first sequence to the second sequence having a different frame rate from the first sequence, at least the code of the picture corresponding to the switching unit Display control information is inserted in the packet for containerizing the encoded image data or the encoded image data.
- the image data of each picture constituting the moving image data is classified into a plurality of hierarchies and encoded by the receiving unit, and the plurality of hierarchies are divided into a predetermined number of hierarchies. Then, a video stream having encoded image data of at least the lowest hierarchical group of pictures among a predetermined number of video streams each having encoded image data of each hierarchical group of pictures is received. Then, the received video stream is processed by the processing unit.
- the video stream having the encoded image data of the lowest hierarchical set of pictures is encoded so that the decoding interval of each picture is a predetermined interval, for example, an equal interval. ing. Therefore, for example, when there is a decoding capability capable of processing the encoded image data of a plurality of hierarchies included in the lowest hierarchy set, the decoding process of the encoded image data of each picture should be performed continuously without difficulty. Is possible.
- the switching unit corresponds to this sequence.
- Display control information is inserted into the encoded image data of a picture to be processed or a packet for containerizing the encoded image data. Therefore, for example, reading of image data of each picture from the uncompressed data buffer can be controlled based on this display control information, and even when the frame rate changes dynamically, it is good without causing a display gap. Display is possible.
- the display control information controls to repeat the display of the last picture of the first sequence by a predetermined number, or the display period of the last picture of the first sequence is the display of the picture of the second sequence.
- the information may be controlled so as to be skipped.
- the display control information may be display offset information for controlling the display timing of the second sequence of pictures.
- the picture data of each picture constituting the moving picture data is classified and encoded into a plurality of hierarchies, and the pictures of the respective hierarchies are obtained by dividing the plurality of hierarchies into a predetermined number of hierarchies.
- a receiving unit that receives a container of a predetermined format including a video stream having encoded image data of at least the lowest hierarchical set of pictures among the predetermined number of video streams each having the encoded image data of Among the predetermined number of video streams, at least the video stream having the encoded image data of the picture of the lowest layer set is encoded so that the decoding interval of each picture becomes a predetermined interval,
- the sequence of the received video stream has a switching unit that switches from the first sequence to the second sequence having a different frame rate from the first sequence, at least the code of the picture corresponding to the switching unit Display control information is inserted in the converted image data, From the video stream included in the received container, the encoded image data of a picture of a layer below a predetermined layer corresponding to the decoding capability is selectively decoded, and the image data of each picture obtained by the decoding is stored in a buffer.
- An image decoding processing unit that captures, reads out and outputs image data of each picture at a predetermined timing;
- the receiving apparatus further includes a control unit that controls reading of an image of each picture from the buffer in the image decoding processing unit based on the display control information.
- a container of a predetermined format is received by the receiving unit.
- the image data of each picture constituting the moving image data is classified and encoded into a plurality of layers, and each layer is obtained by dividing the plurality of layers into a predetermined number of layer sets.
- a video stream having encoded image data of at least the lowest hierarchical set of pictures is included in a predetermined number of video streams each having encoded image data of a set of pictures.
- the video stream having the encoded image data of the lowest hierarchical set of pictures is encoded so that the decoding interval of each picture is a predetermined interval, for example, an equal interval. ing. Therefore, for example, when there is a decoding capability capable of processing the encoded image data of a plurality of hierarchies included in the lowest hierarchy set, the decoding process of the encoded image data of each picture should be performed continuously without difficulty. Is possible.
- the image decoding processing unit selectively decodes the encoded image data of a picture in a hierarchy lower than a predetermined hierarchy according to the decoding capability from the video stream included in the received container, and each picture obtained by the decoding is decoded. Image data is taken into the buffer, and the image data of each picture is read and output at a predetermined timing. This reading is controlled by the control unit based on the display control information.
- the display control information controls to repeat the display of the last picture of the first sequence by a predetermined number, or the display period of the last picture of the first sequence is the display of the picture of the second sequence.
- the information may be controlled so as to be skipped.
- the display control information may be display offset information for controlling the display timing of the second sequence of pictures.
- the image data of each picture obtained by decoding is taken into the buffer, and when the image data of each picture is read and output at a predetermined timing, the readout is displayed. It is controlled based on the control information. Therefore, for example, even when the frame rate changes dynamically, good display can be performed without generating a display gap.
- the picture data of each picture constituting the moving picture data is classified and encoded into a plurality of hierarchies, and the pictures of the respective hierarchies are obtained by dividing the plurality of hierarchies into a predetermined number of hierarchies.
- a receiving unit that receives a container of a predetermined format including a video stream having encoded image data of at least the lowest hierarchical set of pictures among the predetermined number of video streams each having the encoded image data of Among the predetermined number of video streams, at least the video stream having the encoded image data of the picture of the lowest layer set is encoded so that the decoding interval of each picture becomes a predetermined interval, Sequence switching for detecting a switching unit in which the sequence of the video stream is switched from the first sequence to the second sequence having a frame rate different from that of the first sequence based on the video stream included in the received container.
- a detection unit From the video stream included in the received container, the encoded image data of a picture of a layer below a predetermined layer corresponding to the decoding capability is selectively decoded, and the image data of each picture obtained by the decoding is stored in a buffer.
- An image decoding processing unit that captures, reads out and outputs image data of each picture at a predetermined timing;
- the receiving apparatus further includes a control unit that controls reading of an image of each picture from the buffer in the image decoding processing unit based on a detection output of the sequence switching detection unit.
- a container of a predetermined format is received by the receiving unit.
- the image data of each picture constituting the moving image data is classified and encoded into a plurality of layers, and each layer is obtained by dividing the plurality of layers into a predetermined number of layer sets.
- a video stream having encoded image data of at least the lowest hierarchical set of pictures is included in a predetermined number of video streams each having encoded image data of a set of pictures.
- the video stream having the encoded image data of the lowest hierarchical set of pictures is encoded so that the decoding interval of each picture is a predetermined interval, for example, an equal interval. ing. Therefore, for example, when there is a decoding capability capable of processing the encoded image data of a plurality of hierarchies included in the lowest hierarchy set, the decoding process of the encoded image data of each picture should be performed continuously without difficulty. Is possible.
- the sequence switching detection unit switches the sequence of the video stream from the first sequence to the second sequence having a different frame rate from the first sequence based on the video stream included in the received container. Parts are detected. Then, the image decoding processing unit selectively decodes the encoded image data of a picture of a layer below a predetermined layer corresponding to the decoding capability from a predetermined number of video streams included in the received container, and obtains the decoded image data. The image data of each picture is taken into the buffer, and the image data of each picture is read out at a predetermined timing and output. This reading is controlled by the control unit based on the detection output of the sequence switching detection unit.
- the image data of each picture obtained by decoding is taken into the buffer, and when the image data of each picture is read and output at a predetermined timing, this reading is performed in a sequence. It is controlled based on the switching detection output. Therefore, for example, even when the frame rate changes dynamically, good display can be performed without generating a display gap.
- FIG. It is a figure which shows an example of the processing flow of a demultiplexer.
- FIG. It is a figure which shows the structural example of a post process part.
- FIG. shows an example of the processing flow of a decoder and a post-processing part.
- FIG. 1 shows a configuration example of a transmission / reception system 10 as an embodiment.
- the transmission / reception system 10 includes a transmission device 100 and a reception device 200.
- the transmission device 100 transmits a transport stream TS as a container on a broadcast wave.
- the transport stream TS is obtained by classifying and encoding image data of each picture constituting moving image data into a plurality of hierarchies, and dividing the plurality of hierarchies into a predetermined number of hierarchies.
- a video stream having the encoded image data of at least the lowest hierarchical group of pictures is included.
- H.264 / AVC, H.H. Encoding such as H.265 / HEVC is performed, and the referenced picture is encoded so as to belong to a self-layer and / or a layer lower than the self-layer.
- the lowest hierarchy when dividing a plurality of hierarchies into a predetermined number of hierarchies, the lowest hierarchy includes a plurality of hierarchies, and the hierarchy set higher than the lowest hierarchy set has one hierarchy. To be included. Due to such division, on the receiving side, for example, when there is a decoding capability capable of processing encoded image data of pictures of a plurality of hierarchies included in the lowest hierarchy set, the encoding of the pictures of the lowest hierarchy set is performed. Only the video stream having the converted image data can be selected and taken into the buffer to be decoded.
- Hierarchy identification information for identifying the affiliation hierarchy is added to the encoded image data of the picture of each hierarchy for each picture.
- hierarchical identification information (“nuh_temporal_id_plus1” meaning temporal_id) is arranged in the header portion of the NAL unit (nal_unit) of each picture.
- At least the video stream having the encoded image data of the pictures in the lowest layer set is encoded so that the decoding interval of each picture is equal.
- the decoding timing of the encoded image data of the pictures of the hierarchical group positioned higher than the lowest hierarchical group is the encoded image data of the pictures of all the hierarchical groups positioned lower than this hierarchical group.
- Encoding is performed so as to be an intermediate timing of the decoding timing.
- the intermediate timing here does not need to be the middle timing of the decoding timings of the lower two pictures, and means that the timing does not overlap with the lower decoding timing. Therefore, in this case, for example, encoding in which the decoding timing of two or more pictures in the upper layer comes between the decoding timings of the two pictures on the lower side is included.
- the decoding process of each picture is performed on the receiving side. It becomes possible to proceed smoothly in sequence.
- the switching unit when the sequence of the video stream to be transmitted has a switching unit for switching from the first sequence to the second sequence having a frame rate different from that of the first sequence, at least the switching unit includes Display control information is inserted into the encoded picture data of the corresponding picture or the packet that containers the encoded picture data.
- Display control information is inserted into the encoded picture data of the corresponding picture or the packet that containers the encoded picture data.
- reading of image data of each picture from the uncompressed data buffer can be controlled based on this display control information, and even when the frame rate changes dynamically, no display gap is generated. Good display is possible.
- the display control information is, for example, controlled to repeat the display of the last picture of the first sequence by a predetermined number, or the display period of the last picture of the first sequence is the picture of the second sequence This is information for controlling to skip the display of.
- the display control information is display offset information for controlling the display timing of the second sequence of pictures.
- identification information for identifying the last picture of the first sequence is inserted into the encoded image data of the last picture of the first sequence. With this identification information, the receiving side can easily identify the last picture of the first sequence.
- identification information indicating whether or not the above-described display control information is inserted into the encoded image data or a packet that containers the encoded image data is inserted into the layer of the transport stream TS. Is done.
- This configuration information is inserted as a descriptor in a video elementary stream loop arranged corresponding to a predetermined number of video streams under the program map table.
- the receiving device 200 receives the above-described transport stream TS transmitted from the transmitting device 100 on a broadcast wave.
- the receiving apparatus 200 selectively decodes the encoded image data of a picture of a layer below a predetermined layer selected according to the decoding capability from the video stream included in the transport stream TS, and each obtained by decoding
- the image data of a picture is taken into a buffer (uncompressed data buffer).
- the receiving apparatus 200 reads out and outputs the image data of each picture from the buffer at a predetermined timing, and performs image reproduction.
- the switching unit corresponds to this sequence.
- Display control information is inserted into the encoded image data of a picture to be processed or a packet for containerizing the encoded image data. Based on this display control information, reading of the image of each picture from the buffer is controlled. With this control, even when the frame rate changes dynamically, good display is possible without generating a display gap.
- FIG. 2 shows a configuration example of the transmission device 100.
- the transmission apparatus 100 includes a CPU (Central Processing Unit) 101, an encoder 102, a compressed data buffer (cpb: coded picture buffer) 103, a multiplexer 104, and a transmission unit 105.
- the CPU 101 is a control unit and controls the operation of each unit of the transmission device 100.
- Encoder 102 inputs uncompressed moving image data and performs hierarchical encoding.
- the encoder 102 classifies the image data of each picture constituting the moving image data into a plurality of hierarchies. Then, the encoder 102 encodes the image data of the classified pictures of each layer, and generates a video stream having the encoded image data of the pictures of each layer.
- the encoder 102 is, for example, H.264. H.264 / AVC, H.H. Encoding such as H.265 / HEVC is performed. At this time, the encoder 102 performs encoding so that a picture to be referred to (referenced picture) belongs to the own hierarchy and / or a hierarchy lower than the own hierarchy.
- FIG. 3 shows an example of hierarchical encoding performed by the encoder 102.
- the image data is classified into four layers from 0 to 3, and image data of pictures in each layer is encoded.
- the vertical axis represents the hierarchy. 0 to 3 are set as temporal_id (hierarchy identification information) arranged in the header portion of the NAL unit (nal_unit) constituting the encoded image data of the pictures of layers 0 to 3, respectively.
- the horizontal axis indicates the display order (POC: picture order of composition), with the display time on the left and the display time on the right.
- POC picture order of composition
- FIG. 4 shows an example of hierarchical encoding performed by the encoder 102.
- the image data is classified into five layers from 0 to 4, and image data of pictures in each layer is encoded.
- the vertical axis represents the hierarchy. 0 to 4 are set as temporal_id (hierarchy identification information) arranged in the header portion of the NAL unit (nal_unit) constituting the encoded image data of the pictures of layers 0 to 4, respectively.
- the horizontal axis indicates the display order (POC: picture order of composition), with the display time on the left and the display time on the right.
- POC picture order of composition
- FIG. 5A shows a structure example (Syntax) of the NAL unit header
- FIG. 5B shows contents (Semantics) of main parameters in the structure example.
- the 6-bit field “Nal_unit_type” indicates the NAL unit type.
- the 6-bit field of “Nuh_layer_id” is assumed to be 0.
- a 3-bit field of “Nuh_temporal_id_plus1” indicates temporal_id and takes a value (1 to 7) obtained by adding 1.
- each of the rectangular frames indicates a picture, and the numbers indicate the order of the encoded pictures, that is, the encoding order (decoding order on the receiving side).
- a sub picture group (Sub group of pictures) is composed of eight pictures “2” to “9”, and “2” is the first picture of the sub picture group. Become. “1” is a picture of the previous sub-picture group.
- Several of these sub-picture groups are gathered to form a GOP (Group Of Pictures).
- a sub picture group (Sub group of pictures) is composed of 16 pictures “2” to “17”, and “2” is the head of the sub picture group. It becomes a picture. “1” is a picture of the previous sub-picture group. Several of these sub-picture groups are gathered to form a GOP (Group Of Pictures).
- the encoded image data of the first picture of the GOP is composed of NAL units of AUD, VPS, SPS, PPS, PSEI, SLICE, SSEI, and EOS as shown in FIG.
- pictures other than the first picture of the GOP are configured by NAL units of AUD, PPS, PSEI, SLICE, SSEI, and EOS.
- VPS and SPS can be transmitted once in a sequence (GOP), and PPS can be transmitted in each picture.
- the solid line arrows indicate the reference relationship of pictures in encoding.
- the picture “2” is a P picture and is encoded with reference to the picture “1”.
- the picture “3” is a B picture and is encoded with reference to the pictures “1” and “2”.
- other pictures are encoded with reference to nearby pictures in display order. Note that the picture in the highest hierarchy is not referenced from other pictures.
- the encoder 102 divides a plurality of hierarchies into two or more predetermined number of hierarchies, and generates a predetermined number of video streams each having encoded image data of pictures in each hierarchies.
- the encoder 102 includes a plurality of hierarchies in the lowest hierarchy set and divides the hierarchy set higher than the lowest hierarchy set so as to include one hierarchy.
- the encoder 102 has two hierarchies, with the hierarchies 0 to 2 being the lowest hierarchy set and the hierarchy 3 being the higher hierarchy set so as to be separated by an alternate long and short dash line. Divide into pairs.
- the encoder 102 generates two video streams (coded streams) each having the coded image data of each hierarchical set of pictures.
- the encoder 102 is configured such that the hierarchical levels 0 to 2 are the lowest hierarchical group and the hierarchical level 3 is positioned higher than the hierarchical level so as to be separated by the alternate long and short dashed lines. Further, the hierarchy 4 is further divided into three hierarchy groups as the hierarchy group positioned at the higher level. In this case, the encoder 102 generates three video streams (coded streams) each having the coded image data of each hierarchical set of pictures.
- the video stream having the encoded image data of the picture in the lowest hierarchical group is the base stream, and the stream type is “0x24”.
- a video stream including encoded image data of a picture of a hierarchical group positioned higher than the lowest hierarchical group is an enhanced stream, and the stream type is newly defined “0x25”.
- each enhanced stream can be identified instead of setting the stream types of all the enhanced streams to “0x25”. For example, when there are two enhanced streams, the stream type of the first enhanced stream is “0x25”, and the stream type of the second enhanced stream is “0x26”.
- This stream type constitutes identification information for identifying whether each of a predetermined number of video streams is a base stream or an enhanced stream.
- This stream type is inserted into the layer of the transport stream TS. That is, this stream type is inserted into a video elementary stream loop (Video
- the encoder 102 performs encoding so that at least the decoding intervals of the encoded image data of the pictures in the lowest hierarchical set are equal.
- FIG. 7A shows a case in which each picture is encoded at a full 120 Hz timing with temporal resolution in the example of hierarchical encoding in FIG. 3, and layers 0 to 2 constitute a base stream (B stream).
- the hierarchy is divided into two hierarchy sets, with the hierarchy set being the lowest hierarchy set and the hierarchy 3 being the hierarchy set constituting the enhancement stream (E stream) positioned above the hierarchy set.
- the temporal resolution of the pictures in the lowest layer set is 60 fps
- a decoder having a decoding capability of 60 fps has a continuous and stable decoding process. Is impossible.
- the encoding timing of the pictures in the lowest hierarchical group constituting the base stream is adjusted to be 60 Hz, and the encoded image data of the pictures in the lowest hierarchical group is set. Are encoded so that their decoding intervals are equal.
- a decoder having a decoding capability of 60 fps enables continuous and stable decoding processing on the encoded image data of the pictures in the lowest layer set constituting the base stream.
- the encoding timing of the hierarchical group of pictures constituting the enhancement stream (E stream), and hence the decoding timing of the encoded image data of the picture is the base stream (B stream).
- Encoding is performed so as to be an intermediate timing of the decoding timing of the encoded image data of the picture of the lowest layer group to be configured.
- FIG. 8 shows the encoding timing (decoding timing) of each picture in the case of generating two video streams of a base stream (B stream) and an enhancement stream (E stream) in the hierarchical encoding example of FIG. An example is shown. In this example, the decoding delay of the enhanced stream is minimized with respect to the base stream.
- the encoding timing of the picture of the base stream (B stream) is an even timing
- the encoding timing of the enhancement stream (E stream) is an odd timing
- the enhancement stream (E stream) is encoded immediately after the encoding order of the highest layer of the base stream (B stream). That is, the picture “7” of the enhanced stream (E stream) is encoded immediately after the picture “6” of the base stream (B stream).
- FIG. 9A shows a case in which each picture is encoded at a full 120 Hz timing with temporal resolution in the example of hierarchical encoding in FIG. 4, and layers 0 to 2 constitute a base stream (B stream).
- E stream1 constitutes an enhanced stream
- E stream2 constitutes an enhanced stream located in the upper hierarchy thereof.
- a case is shown in which a set is divided into three hierarchical sets.
- the temporal resolution of the pictures in the lowest layer set is 30 fps
- a decoder having a decoding capability of 30 fps has a continuous and stable decoding process. Is impossible.
- the encoding timing of the pictures in the lowest hierarchical group constituting the base stream is adjusted to be 30 Hz, and the encoded image data of the pictures in the lowest hierarchical group is set. Are encoded so that their decoding intervals are equal.
- a decoder having a decoding capability of 30 fps can continuously and stably decode the encoded image data of the pictures in the lowest layer set constituting the base stream.
- the encoding timing of the hierarchical group of pictures constituting the enhancement stream (E stream1), and hence the decoding timing of the encoded image data of the picture is the base stream (B stream). Encoding is performed so as to be an intermediate timing of the decoding timing of the encoded image data of the picture of the lowest layer group to be configured.
- the encoding timing of the hierarchical group of pictures constituting the enhancement stream (E stream2), and hence the decoding timing of the encoded image data of the picture is the base stream (B stream) and Encoding is performed so as to be an intermediate timing of the decoding timing of the encoded image data of the hierarchical set of pictures constituting the enhanced stream (E stream1).
- FIG. 10 shows a picture of each picture in the case of generating three video streams of a base stream (B ⁇ ⁇ stream), an enhanced stream (E ⁇ stream1), and an enhanced stream (E stream2) in the example of the hierarchical coding of FIG.
- An example of encoding timing (decoding timing) is shown. In this example, the decoding delay of the enhanced stream is minimized with respect to the base stream.
- the encoding timing of the picture of the base stream (B stream) is a multiple of 4
- the encoding timing of the enhancement stream (E stream1) is a multiple of 4
- the base stream (B stream) The timing is intermediate between the picture encoding timings.
- the encoding timing of the enhanced stream (E (stream1) is an odd timing.
- the enhancement stream (E stream1) is encoded immediately after the encoding order of the highest layer of the base stream (B stream). That is, the picture “10” of the enhanced stream (E stream1) is encoded immediately after the picture “8” of the base stream (B stream).
- the enhancement stream (E stream2) is encoded immediately after the encoding order of the enhancement stream (E stream1). That is, the picture “11” of the enhanced stream (E stream2) is encoded immediately after the picture “10” of the enhanced stream (E stream1).
- the encoder 102 encodes at least the code of the picture corresponding to the switching unit Display control information is inserted into the converted image data.
- the frame rate of the uncompressed moving image data input to the encoder 102 is switched, or the frame rate of the uncompressed moving image data input to the encoder 102 is the same.
- the frame rate is switched by frame decimation or interpolation. Note that when the frame rate is switched at the sequence switching unit, the number of video streams generated by the encoder 102 may be the same or different.
- FIG. 11 shows an example of the sequence switching unit.
- the first sequence 60 Hz image sequence
- the second sequence 120 H image sequence.
- the first sequence only the base stream (B stream) having the encoded image data of the pictures in the layers 0 to 2 in the example of the hierarchical encoding in FIG. 3 is generated.
- An enhancement stream (E stream) having encoded image data of a picture is generated.
- FIG. 12 shows an example of HRD (Hypothetical Reference Decoder) control of the encoder 102 in the sequence switching unit as shown in FIG.
- HRD Hypothetical Reference Decoder
- the step-like solid line a1 indicates the transition of the data amount of the substream 1 generated by encoding (encoding), and each stage corresponds to one picture unit.
- the height of the step indicates the amount of data generated by encoding.
- the step-like solid line b1 shows the transition of the data amount in cpb1 (coded picture buffer 1: compressed data buffer) consumed by decoding, and each stage corresponds to one picture unit. .
- the step height indicates the amount of data consumed by decoding.
- the slope R1 of the sloped solid line c1 indicates the bit rate of the encoded image data input to cpb1.
- Timing P01 is timing when the first byte data of the picture “2” which is the last display picture of the first sequence is input to cpb1.
- the decoder detects the SEI of the encoded image data of the picture “2”, recognizes that the picture “2” is the last picture (access unit) before switching, and also follows the picture. Detect the control method of (access unit).
- the display period of the picture before switching is detected from “clock_tick” and “time_scale” of VPS or SPS IV. Further, a change in the display period is detected based on a change in the same parameter element in the subsequent access unit (AU).
- an upper layer sub-stream having an access unit (AU) having a display timing later than the display timing of the current access unit (AU) is newly added to cpb (current frame If the rate is P, the change from P to N frame rate: P ⁇ N), or an upper layer having an access unit (AU) having a display timing later than the display timing of the current access unit (AU) Recognizing that a substream (sub-stream) is not newly input to cpb (if the current frame rate is N, a change from N to P frame rate: P ⁇ N), a subsequent access unit (AU ) Check parameters.
- Timing P02 is timing when the first byte data of the picture “10” that is the first decoded picture of the second sequence is input to cpb1.
- the decoder detects the SEI of the encoded image data “10” and detects the display period of the picture (access unit) after switching. In this case, the display period is detected from “clock_tick” and “time_scale” of VPS or SPS.
- the stair-like solid line a2 indicates the transition of the data amount of the substream 2 generated by encoding (encoding), and each stage corresponds to one picture unit.
- the height of the step indicates the amount of data generated by encoding.
- a stair-like solid line b2 indicates a change in the amount of data in cpb2 consumed by decoding (decoding), and each stage corresponds to one picture unit.
- the step height indicates the amount of data consumed by decoding.
- An inclination R2 of the inclined solid line c2 indicates the bit rate of the encoded image data input to cpb2.
- Timing P03 is the timing at which the first byte data of the picture “15”, which is the first display picture of the second sequence, is input to cpb2.
- the decoder detects the SEI of the encoded image data “15” and detects the time stamp offset.
- picture display is performed at a corrected display timing obtained by adding an offset value to the timing of “dpb_output_delay”.
- the picture is decoded at a modified decoding timing obtained by adding an offset value to the timing of “cpb_removal_delay”.
- both R1 and R2 are examples of a constant bit rate (constant_bit_rate).
- constant_bit_rate variable bit rate
- substream 1 is decoded in the order of pictures “0”, “2”, “4”, “6”, “8”, “10”, “12”. Are decoded in the order of pictures “15”, “17”, “19”. That is, only the picture of substream 1 is decoded in the first sequence, and the picture of substream 1 and the picture of substream 2 are decoded alternately in the second sequence.
- the decoded image data of each picture is input to dpb (decoded picture buffer: uncompressed data buffer), read out from dpb at the timing of “display image 1” shown in the figure, and output.
- the picture “2” is a picture of the first sequence (sequence of 60 Hz image), but is displayed at 120 Hz. Therefore, in this embodiment, as described above, by inserting the display control information into the encoded image data of the picture corresponding to at least the switching unit, on the receiving side, the “display image 2” or “display” shown in the figure. "Image 3" can be displayed.
- Display Image 2 the display timing of each picture in the second sequence is delayed by one cycle of 120 Hz.
- display picture 3 the display of the pictures in the second sequence is skipped for the display period of the last picture in the first sequence.
- the picture “2” is displayed at 60 Hz, and the occurrence of the display gap is eliminated.
- FIG. 13 shows another example of the sequence switching unit.
- the first sequence 120 Hz image sequence
- the second sequence 60H image sequence.
- a base stream B stream
- E stream an enhanced stream having encoded image data
- the second sequence only a base stream (B stream) having encoded image data of pictures in layers 0 to 2 in the example of hierarchical encoding in FIG. 3 is generated.
- FIG. 14 shows an example of HRD (Hypothetical Reference Decoder) control of the encoder 102 in the sequence switching unit as shown in FIG.
- HRD Hypothetical Reference Decoder
- a stair-like solid line a11 indicates a change in the data amount of the substream 1 generated by encoding (encoding), and each stage corresponds to one picture unit.
- the height of the step indicates the amount of data generated by encoding.
- the step-like solid line b11 shows the transition of the data amount in cpb1 (coded picture buffer 1: compressed data buffer) consumed by decoding (decoding), and each stage corresponds to one picture unit. .
- the step height indicates the amount of data consumed by decoding.
- the slope R11 of the sloped solid line c11 indicates the bit rate of the encoded image data input to cpb1.
- Timing P11 is the timing at which the first byte data of the picture “2” that is the last display picture of the first sequence is input to cpb1.
- the decoder detects the SEI of the encoded image data of the picture “2”, recognizes that the picture “2” is the last picture (access unit) before switching, and also follows the picture. Detect the control method of (access unit).
- Timing P12 is timing when the first byte data of the picture “10” that is the first decoded picture of the second sequence is input to cpb1. At this timing P12, the decoder detects the SEI of the encoded image data “10” and detects the display period of the picture (access unit) after switching.
- the timing P13 is a timing at which the first byte data of the picture “14”, which is the first display picture of the second sequence, is input to cpb1.
- the decoder detects the SEI of the encoded image data “14” and detects the time stamp offset.
- both R11 and R21 are examples of a constant bit rate (constant_bit_rate), but the concept is the same for a variable bit rate (variable_bit_rate) without being limited thereto.
- the picture “2” is a picture of the first sequence (sequence of 120 Hz image), but is displayed at 60 Hz. Therefore, in this embodiment, as described above, by inserting the display control information into the encoded image data of the picture corresponding to at least the switching unit, on the receiving side, the “display image 2” or “display” shown in the figure. "Image 3" can be displayed.
- Display Image 2 the display timing of each picture in the second sequence is advanced by one cycle of 120 Hz.
- display picture 3 the display of the last picture of the first sequence is repeated only once.
- the picture “2” is displayed at 120 Hz, and the occurrence of the display gap is eliminated.
- the encoder 102 inserts newly defined AU timing control information SEI (au_timing_control_information SEI) including the display control information described above as one of the prefix SEI (Prefix_SEI).
- FIG. 15A shows a structural example (Syntax) of an interface (I / F) for inserting AU timing control information SEI.
- the field of “uuid_iso_iec — 11578” has a UUID value indicated by “ISO / IEC 11578: 1996 Annex A”.
- “Au_timing_control_information ()” is inserted in the field of “user_data_payload_byte”.
- FIG. 16 shows a structural example (Syntax) of “au_timing_control_information ()”.
- FIG. 17 shows the contents (Semantics) of main information in the structural example.
- a 16-bit field of “userdata_id” is assigned a predetermined user data ID.
- the 8-bit field of “au_timing_control_information_length” indicates the number of bytes of “au_timing_control_information” (counted from the next element of this element).
- the 1-bit field of “last_au_flag” indicates that the current access unit (picture) is the last access unit of CVD (coded video sequence). “1” indicates that it is the last access unit, and “0” indicates that it is not the last access unit.
- “1-bit field of“ next_au_presentation_skip_flag ” indicates that the display of the subsequent access unit is skipped. “1” indicates that the display of the subsequent access unit is skipped until the end of the display period of the current access unit. “0” indicates that the display of the subsequent access unit is not skipped. In this case, the decoded data is displayed at the display timing specified by the stream.
- 1-bit field of “current_au_repeat_flag” indicates that the current access unit is displayed and repeated. “1” indicates that the access unit to be displayed in the next timing grid (timing grid) is replaced by display repeat of the current access unit. “0” indicates that the next access unit is displayed at the display timing specified by the stream.
- this “current_au_repeat_flag” is “1”
- This 4-bit field specifies the repeat count of the current access unit. For example, “0001” indicates that the number of repeats is 1, “0010” indicates that the number of repeats is 2, and “0011” indicates that the number of repeats is 3.
- Offset_timing_control_flag 1-bit field of “offset_timing_control_flag” indicates that offset information is transmitted. “1” indicates that “offset_to_cpb_removal_delay” and “offset_to_dpb_output_delay” are transmitted, and there is a 24-bit field in which each is arranged.
- a 24-bit field of “offset_to_cpb_removal_delay” indicates a difference value from “cpb_removal_delay” of the picture.
- Offset_to_dpb_output_delay indicates a difference value from “dpb_output_delay” of the picture.
- FIG. 18 shows the transition of the value of each flag when the receiving side performs change control from the display of “display image 1” to the display of “display image 2”.
- “last_au_flag” of the picture “2” that is the last picture of the first sequence is “1”, which indicates that this picture is the last picture of the first sequence.
- “offset_timing_control_flag” of the second sequence picture is “1”, and “offset_to_cpb_removal_delay” and “offset_to_dpb_output_delay” are transmitted.
- the display timing of each picture of the second sequence is controlled to be delayed by one cycle of the second sequence, and the display of “display image 2” is realized.
- FIG. 19 shows the transition of the value of each flag when the receiving side controls to change from the display of “display screen 1” to the display of “display screen 3”.
- “last_au_flag” of the picture “2” that is the last picture of the first sequence is “1”, which indicates that this picture is the last picture of the first sequence.
- “next_au_presentation_skip_flag” of the picture “2” is “1”, which indicates that the display of the subsequent access unit is skipped until the end of the display period of the current access unit.
- the display of the second sequence of pictures is skipped for the display period of the picture “2”, and the display of “display image 3” is realized.
- FIGS. 20 and 21 show the values of the flags of the AU timing control information SEI (see FIGS. 15 to 17) in the sequence switching (switching from 1 ⁇ speed to 2 ⁇ speed) in FIGS. 13 and 14 described above. It shows the transition.
- FIG. 20 shows the transition of the value of each flag when the receiving side performs change control from the display of “display image 1” to the display of “display image 2”.
- “last_au_flag” of the picture “2” that is the last picture of the first sequence is “1”, which indicates that this picture is the last picture of the first sequence.
- “offset_timing_control_flag” of the second sequence picture is “1”, and “offset_to_cpb_removal_delay” and “offset_to_dpb_output_delay” are transmitted.
- the display timing of each picture of the second sequence is controlled to be advanced by one cycle of the first sequence, and the display of “display image 2” is realized.
- FIG. 21 shows the transition of the value of each flag when the receiving side performs change control from the display of “display image 1” to the display of “display image 3”.
- “last_au_flag” of the picture “2” that is the last picture of the first sequence is “1”, which indicates that this picture is the last picture of the first sequence.
- “current_au_repeat_flag” of the picture “2” is “1”, which indicates that the access unit to be displayed in the next timing grid (timing grid) is replaced by display repeat of the current access unit.
- “repeat_type” of the picture “2” is set to “0001”, which indicates that the number of repeats is 1. Thereby, on the receiving side, the display of the picture “2” is repeated only once, and the display of “display picture 3” is realized.
- a first sequence for example, a sequence of 30 Hz images
- a second sequence for example, a sequence of 120 Hz images
- FIG. 22 shows the transition of the value of each flag when the receiving side performs change control from the display of “display image 1” to the display of “display image 2”.
- “last_au_flag” of the picture “0” that is the last picture of the first sequence is “1”, which indicates that this picture is the last picture of the first sequence.
- “offset_timing_control_flag” of the second sequence picture is “1”, and “offset_to_cpb_removal_delay” and “offset_to_dpb_output_delay” are transmitted.
- the display timing of each picture in the second sequence is controlled to be delayed by three periods of the second sequence, and the display of “display image 2” is realized.
- FIG. 23 shows the transition of the value of each flag when the receiving side performs change control from the display of “display image 1” to the display of “display image 3”.
- “last_au_flag” of the picture “0” that is the last picture of the first sequence is “1”, which indicates that this picture is the last picture of the first sequence.
- “next_au_presentation_skip_flag” of the picture “0” is “1”, which indicates that the display of the subsequent access unit is skipped until the end of the display period of the current access unit.
- FIGS. 24 and 25 show transitions of the values of the flags of the AU timing control information SEI (see FIGS. 15 to 17) in the sequence switching from the 4 ⁇ speed to the 1 ⁇ speed.
- This example is an example of switching from a first sequence (for example, a sequence of 120 Hz images) to a second sequence (for example, a sequence of 30 Hz images).
- An enhanced stream (E stream 1) having the encoded image data and an enhanced stream (E stream 2) having the encoded image data of the layer 4 picture in the example of the hierarchical encoding shown in FIG. 4 are generated.
- the second sequence only the base stream (B stream) having the encoded image data of the pictures of layers 0 to 2 in the example of the hierarchical encoding of FIG. 4 is generated.
- FIG. 24 shows the transition of the value of each flag when the receiving side performs change control from the display of “display image 1” to the display of “display image 2”.
- “last_au_flag” of the picture “0” that is the last picture of the first sequence is “1”, which indicates that this picture is the last picture of the first sequence.
- “offset_timing_control_flag” of the second sequence picture is “1”, and “offset_to_cpb_removal_delay” and “offset_to_dpb_output_delay” are transmitted.
- the display timing of each picture of the second sequence is controlled to be advanced by three cycles of the first sequence, and the display of “display image 2” is realized.
- FIG. 25 shows the transition of the value of each flag when the receiving side performs change control from the display of “display image 1” to the display of “display image 3”.
- “last_au_flag” of the picture “0” that is the last picture of the first sequence is “1”, which indicates that this picture is the last picture of the first sequence.
- “current_au_repeat_flag” of the picture “2” is “1”, which indicates that the access unit to be displayed in the next timing grid (timing grid) is replaced by display repeat of the current access unit.
- “repeat_type” of the picture “2” is “00011”, which indicates that the number of repeats is three.
- the AU timing control information SEI is similarly applied to the switching at other magnifications such as the sequence switching from the 1 ⁇ speed to the 3 ⁇ speed and the sequence switching from the 3 ⁇ speed to the 1 ⁇ speed.
- display of “display image 2” or “display image 3” is realized on the receiving side.
- FIG. 26 shows a configuration example of the encoder 102.
- the encoder 102 includes a temporal ID generation unit 121, a buffer delay control unit 122, an HRD (Hypothetical Reference Decoder) setting unit 123, a parameter set / SEI encoding unit 124, a slice encoding unit 125, and a NAL packetizing unit 126. have.
- HRD Hypothetical Reference Decoder
- the temporal ID generation unit 121 is supplied with information on the number of layers (Number of layers) from the CPU 101.
- the buffer delay control unit 122 is supplied with information of minimum decoding capability (Target ⁇ ⁇ ⁇ ⁇ minimum decoder capability) from the CPU 101 and temporal_id generated by the temporal ID generation unit 121.
- the buffer delay control unit 122 calculates “initial_cpb_removal_delay”, which is an initial value of cpb buffering (buffering), and “cpb_removal_delay” and “dpb_output_delay” for each picture, for each video stream.
- the buffer delay control unit 122 controls “Cpb_removal_delay” in the cpb buffer for each substream.
- the buffer delay control unit 122 controls the dpb buffer so that there is no buffer failure between the decode timing and the display timing of the decoder.
- “cpb_removal_delay” is controlled so that the decoding timings of the pictures in the lowest layer set are equally spaced.
- the encoding timing of the encoded image data of the pictures of the hierarchical group positioned higher than the lowest hierarchical group is the same as the encoded image data of the pictures of all the hierarchical groups positioned lower than this hierarchical group.
- “Cpb_removal_delay” is controlled so as to be an intermediate timing of the encoding timing.
- dpb_output_delay is controlled so that the cpb buffer is not broken.
- the encoding timing has the same meaning as the decoding timing read from the compressed data buffer (cpb: coded picture buffer) on the receiving side.
- An HRD (Hypothetical Reference Decoder) setting unit 123 is supplied with “cpb_removal_delay” and “dpb_output_delay” of the picture of each video stream calculated by the buffer delay control unit 122 and the number of streams (Number of streams) from the CPU 101. Information is supplied. The HRD setting unit 123 performs HRD setting based on these pieces of information.
- the temporary_id is supplied to the parameter set / SEI encoding unit 124 together with the HRD setting information.
- the parameter set / SEI encoding unit 124 generates parameter sets such as VPS, SPS, and PPS of pictures in each layer and various types of SEI according to the number of streams to be encoded.
- the above-mentioned AU timing control SEI (au_timing_control SEI) is generated.
- picture timing SEI Picture timing SEI
- dpb_output_delay is generated.
- a buffering period SEI BuffereePerifod SEI
- the buffering period SEI is generated corresponding to the first picture (access unit) of the GOP.
- “Initial cpb removal time” indicates a time (initial time) taken out when decoding the encoded image data of the first picture of GOP (Group Of Picture) from the compressed data buffer (cpb).
- “Cpb_removal_delay” is a time for extracting the encoded image data of each picture from the compressed data buffer (cpb), and the time is determined together with “initial_cpb_removal_time”.
- “Dpb_output_delay” indicates the time taken to decode and enter the uncompressed data buffer (dpb).
- the slice encoding unit 125 encodes the image data of the pictures in each layer to obtain slice data (slice segment header, slice segment data).
- the slice encoding unit 125 inserts “ref_idx_l0_active (ref_idx_l1_active)” indicating the index of the prediction destination picture of “PredictionPreUnit” into “slice segment header” as information indicating the prediction state in the temporal direction by the frame buffer.
- the referenced picture is determined together with the hierarchical level indicated by temporal_id.
- the slice encoding unit 125 inserts the index of the current slice (slice) into “slicessegment header” as “short_term_ref_pic_set_idx”, “” or “it_idx_sps”.
- the NAL packetizing unit 126 generates encoded image data of pictures in each layer based on the parameter set and SEI generated by the parameter set / SEI encoding unit 124 and the slice data generated by the slice encoding unit 125.
- the number of video streams (encoded streams) corresponding to the number of streams is output.
- temporal_id indicating the hierarchy is added to the NAL unit header for each picture (see FIG. 5). Also, pictures belonging to the layer indicated by temporal_id are bundled as a sublayer (sub_layer), and the bit rate level designation value “Level_idc” for each sublayer is set to “sublayer_level_idc” and inserted into the VPS or SPS.
- FIG. 27 shows a processing flow of the encoder 102.
- the encoder 102 starts processing, and then proceeds to processing in step ST2.
- the encoder 102 checks the parameters of the moving image sequence to be encoded. This parameter includes a frame rate (frame frequency).
- step ST3 the encoder 102 determines whether or not there is a change in the frame rate between sequences. When there is a change in the frame rate, the encoder 102 proceeds to the process of step ST4. In step ST4, buffer input / output timing management before and after the change point is performed.
- step ST5 the encoder 102 proceeds to the process of step ST5.
- the encoder 102 When there is no change in the frame rate in step ST3 described above, the encoder 102 immediately moves to the process in step 5.
- step ST5 the encoder 102 performs HRD management on all the pictures (Slice), encodes a slice (Slice), a parameter set (parameter set), SEI, and the like, and then converts the stream into a NAL packet ( NAL packet). Then, the encoder 102 complete
- the compressed data buffer (cpb) 103 temporarily stores the video stream generated by the encoder 102 and including the encoded data of the pictures of each layer.
- the multiplexer 104 reads the video stream stored in the compressed data buffer 103, converts it into a PES packet, further converts it into a transport packet, multiplexes it, and obtains a transport stream TS as a multiplexed stream.
- the transport stream TS includes at least the lowest layer of the pictures in the predetermined number of video streams having the encoded image data of the pictures of each layer set obtained by dividing a plurality of layers.
- a video stream having the encoded image data is included.
- the transport stream TS normally includes all of the predetermined number of video streams generated by the encoder 102, but excludes video streams in which the upper layer set has encoded image data of a picture. It is also possible.
- the multiplexer 104 inserts display control information into a packet for containerizing a video stream, for example, a PES packet.
- This display control information is the same as the display control information inserted into the encoded image data by the encoder 102 as described above.
- offset type stamp information offset_timestamp_information
- AU presentation control au_presentation_control
- PES extension PES extension
- FIG. 28A shows a structural example (Syntax) of PES extension field data (pes_extension_field_data).
- FIG. 28B shows the contents (Semantics) of main information in the structural example. Note that “PES_extensionextfield length” is given outside of this syntax structure.
- An 8-bit field of “start_sync_byte” indicates a code value indicating the start of an extension field (extension field).
- extension_field_type indicates the type of the extension field. “0x02” indicates that “offset_timestamp_information ()” and “au_presentation_control ()” are supplied.
- the 4-bit field of “maximum_temporal_layer_minus1” indicates the total number of layers of a plurality of substreams (sub-streams) constituting the service, and is a value obtained by subtracting “1” from the maximum value of “temporal_layer_id”.
- FIG. 29A shows a structural example (Syntax) of “offset_timestamp_information ()”.
- FIG. 29B shows the contents (Semantics) of main information in the structural example.
- a 24-bit field of “offset_to_DTS” indicates an offset difference value (with 90 KHz unit sign) from the DTS attached to the PES header (PES header).
- a 24-bit field of “offset_to_PTS” indicates an offset difference value (with 90 KHz unit sign) from the PTS attached to the PES header (PES header).
- FIG. 30 shows a structural example (Syntax) of “au_presentation_control ()”.
- FIG. 31 shows the contents (Semantics) of main information in the structural example.
- a 1-bit field of “last_au_flag” indicates that the current access unit (picture) is the last access unit of CVD (coded video sequence). “1” indicates that it is the last access unit, and “0” indicates that it is not the last access unit.
- “1-bit field of“ next_au_presentation_skip_flag ” indicates that the display of the subsequent access unit is skipped. “1” indicates that the display of the subsequent access unit is skipped until the end of the display period of the current access unit. “0” indicates that the display of the subsequent access unit is not skipped. In this case, the decoded data is displayed at the display timing specified by the stream.
- 1-bit field of “current_au_repeat_flag” indicates that the current access unit is displayed and repeated. “1” indicates that the access unit to be displayed in the next timing grid (timing grid) is replaced by display repeat of the current access unit. “0” indicates that the next access unit is displayed at the display timing specified by the stream.
- this “current_au_repeat_flag” is “1”
- This 4-bit field specifies the repeat count of the current access unit. For example, “0001” indicates that the number of repeats is 1, “0010” indicates that the number of repeats is 2, and “0011” indicates that the number of repeats is 3.
- the multiplexer 104 inserts identification information indicating whether or not AU timing control SEI (au_timing_control SEI) is inserted into the encoded image data in the layer of the transport stream TS.
- This identification information is inserted as a descriptor in a video elementary stream loop arranged corresponding to a predetermined number of video streams under the program map table.
- the multiplexer 104 inserts a newly defined temporal control descriptor (Temporal_control_descriptor) together with the HEVC descriptor (HEVC_descriptor).
- Temporal_control_descriptor a newly defined temporal control descriptor
- HEVC_descriptor a newly defined temporal control descriptor
- FIG. 32 shows a structural example (Syntax) of the temporal control descriptor.
- Temporal_control_descriptor_tag indicates the descriptor type. Here, it shows that it is a temporal control descriptor.
- the 8-bit field of “Temporal_control_descriptor_length” indicates the length (size) of the descriptor, and indicates the number of subsequent bytes as the length of the descriptor. Here, 1 byte is shown.
- a 1 bit field of “au_timing_control_SEI_existed” indicates whether or not AU timing control SEI exists, and “1” indicates that it exists.
- FIG. 33 shows a structural example (Syntax) of the HEVC descriptor (HEVC_descriptor).
- the 8-bit field of “descriptor_tag” indicates a descriptor type, and here indicates that it is a HEVC descriptor.
- the 8-bit field of “descriptor_length” indicates the length (size) of the descriptor, and indicates the number of subsequent bytes as the length of the descriptor.
- the 8-bit field of “level_idc” indicates the bit rate level specification value.
- temporary_layer_subset_flag_ 1
- temporary_id_min indicates the value of temporal_id of the lowest hierarchy of the hierarchically encoded data included in the corresponding video stream.
- Temporal_id_max indicates the value of temporal_id of the highest hierarchy of the hierarchically encoded data included in the corresponding video stream.
- the multiplexer 104 inserts identification information for identifying whether each video stream is a base stream or an enhanced stream.
- the identification information is inserted as a stream type in a video elementary stream loop (Video ES loop) arranged corresponding to a predetermined number of video streams under the program map table.
- the stream type of the base stream is “0x24”.
- the stream type of the enhanced stream is newly defined, for example, “0x25”. If there are multiple enhanced streams, the stream types of all enhanced streams are not the same, but multiple stream types are newly defined as enhanced stream types so that each enhanced stream can be identified. Also good. For example, when there are two enhanced streams, the stream type of the first enhanced stream is “0x25”, and the stream type of the second enhanced stream is “0x26”.
- FIG. 34 shows a configuration example of the multiplexer 104.
- the multiplexer 104 includes a section coding unit 142, PES packetization units 143-1 to 143-N, a switch unit 144, and a transport packetization unit 145.
- the PES packetizers 143-1 to 143-N read the video streams 1 to N stored in the compressed data buffer 103, respectively, and generate PES packets.
- the video streams 1 to N include at least one base stream.
- N is 2 or more, one base stream and one or more enhanced streams are included.
- the PES packetizers 143-1 to 143-N add a DTS (Decoding Time Stamp) and a PTS (Presentation Time Stamp) time stamp to the PES header based on the HRD information of the video streams 1 to N.
- DTS Decoding Time Stamp
- PTS Presentation Time Stamp
- “cpu_removal_delay” and “dpb_output_delay” of each picture are referred to, and DTS and PTS are generated with accuracy synchronized with STC (System Time Clock) time, and are arranged at predetermined positions of the PES header.
- the PES packetizers 143-1 to 143-N read the video stream (Elementary Stream) stored in the compressed data buffer 103 and generate a PES packet. At this time, the PES packetizers 143-1 to 143-N add DTS (Decoding Time Stamp) and PTS (Presentation Time Stamp) time stamps to the PES header based on the HRD information of the video stream.
- DTS Decoding Time Stamp
- PTS Presentation Time Stamp
- “cpu_removal_delay” and “dpb_output_delay” of each picture are referred to, converted into DTS and PTS, respectively, with accuracy synchronized with STC (System Time Clock) time, and arranged at predetermined positions of the PES header.
- the PES packetization units 143-1 to 143-N determine that the time stamps are discontinuous, the SES, or the HRD information of the video stream, and the like.
- the above-described offset type stamp information (offset_timestamp_information) and AU presentation control (au_presentation_control) are created. Then, the PES packetization units 143-1 to 143-N insert this information into the field of the PES extension (PES extension) of the PES packet.
- the switch unit 144 selectively extracts the PES packets generated by the PES packetizing units 143-1 to 143-N based on the packet identifier (PID), and sends the PES packets to the transport packetizing unit 145.
- the transport packetization unit 145 generates a TS packet including a PES packet in the payload, and obtains a transport stream TS.
- the section coding unit 142 generates various section data to be inserted into the transport stream TS.
- the section coding unit 142 is supplied with information such as the number of layers (Number ⁇ of layers) and the number of streams (Number of streams) from the CPU 101.
- the section coding unit 142 generates the above-described HEVC descriptor (HEVC_descriptor) based on these pieces of information.
- the section coding unit 142 is supplied with information on the number of layers (Number of layers) from the CPU 101 and whether the AU timing control SEI (au_timing_control SEI) is inserted into the encoded image data from the CPU 101.
- the section coding unit 142 generates the above-described temporal control descriptor (Temporal_control_descriptor) based on this information.
- the section coding unit 142 sends various section data to the transport packetizing unit 145.
- the transport packetization unit 145 generates a TS packet including this section data and inserts it in the transport stream TS.
- the stream type is also inserted into the video elementary stream loop (Video ES loop) arranged corresponding to each video stream.
- the stream type of the base stream is “0x24”
- the stream type of the enhanced stream is, for example, “0x25” that is newly defined.
- FIG. 35 shows a processing flow of the multiplexer 104.
- the multiplexer 104 starts processing, and then proceeds to processing in step ST12.
- the multiplexer 104 calculates a time stamp from the picture timing SEI, buffering period SEI, AU timing control SEI, or HRD information of the video stream (Elementary Stream).
- step ST13 the multiplexer 104 creates offset type stamp information (offset_timestamp_information) and AU presentation control (au_presentation_control), and inserts them in the field of the PES extension (PES extension).
- step ST14 the multiplexer 104 inserts a video stream (ElementaryPEStream) into the PES payload.
- step ST15 the multiplexer 104 encodes the temporal control descriptor (Temporal_control_descriptor) into the section area.
- step ST16 the multiplexer 104 outputs the TS packet.
- the multiplexer 104 ends the process in step ST17 after the process of step ST16.
- FIG. 36 shows a configuration example of the transport stream TS when a certain service is distributed in two streams.
- the transport stream TS includes two video streams, a base stream and an enhanced stream. That is, in this configuration example, there is a base stream PES packet “video PES1” and an enhanced stream PES packet “video PES2”.
- offset type stamp information offset_timestamp_information
- AU presentation control au_presentation_control
- a buffering period SEI a picture timing SEI, an AU timing control SEI, and the like are inserted into the encoded image data of each picture.
- DTS and PTS are also arranged in the PES header.
- the transport stream TS includes a PMT (Program Map Table) as one of PSI (Program Specific Information).
- This PSI is information describing to which program each elementary stream included in the transport stream belongs.
- PMT has a program loop that describes information related to the entire program.
- the PMT has an elementary stream loop having information related to each video stream.
- video ES1 loop corresponding to the base stream
- video elementary stream loop video ES2 loop
- video ES1 loop information such as the stream type and packet identifier (PID) is arranged corresponding to the base stream (video PES1), and a descriptor describing information related to the video stream is also arranged.
- the This stream type is “0x24” indicating the base stream.
- the HEVC descriptor and the temporal control descriptor described above are inserted as one of the descriptors.
- video ES2 loop information such as a stream type and a packet identifier (PID) is arranged corresponding to the enhanced stream (video PES2), and a descriptor describing information related to the video stream is also provided. Be placed.
- This stream type indicates an enhanced stream, for example, “0x25” that is newly defined.
- the HEVC descriptor and the temporal control descriptor described above are inserted as one of the descriptors.
- the transmission unit 105 modulates the transport stream TS with a modulation scheme suitable for broadcasting such as QPSK / OFDM, and transmits an RF modulation signal from the transmission antenna.
- a modulation scheme suitable for broadcasting such as QPSK / OFDM
- Uncompressed moving image data is input to the encoder 102.
- the encoder 102 performs hierarchical encoding on the moving image data. That is, in the encoder 102, the image data of each picture constituting the moving image data is classified into a plurality of layers and encoded, and a video stream having the encoded image data of the pictures of each layer is generated. At this time, encoding is performed so that the picture to be referenced belongs to the self hierarchy and / or a hierarchy lower than the self hierarchy.
- the encoder 102 divides a plurality of hierarchies into a predetermined number of hierarchies, and generates a predetermined number of video streams each having encoded image data of pictures in each hierarchies.
- a base stream having encoded image data of pictures of the lowest hierarchical group is generated, and a predetermined number of encoded image data of pictures of hierarchical groups positioned higher than the lowest hierarchical group are included.
- An enhanced stream is generated.
- the encoding is performed so that the decoding intervals of the encoded image data of the pictures in the lowest hierarchy set are equal.
- the encoder 102 when a sequence of a predetermined number of video streams to be generated has a switching unit for switching from the first sequence to a second sequence having a frame rate different from that of the first sequence, at least the switching unit corresponds.
- Display control information (AU timing control information SEI: see FIG. 16) is inserted into the encoded image data of the picture to be played.
- the predetermined number of video streams generated by the encoder 102 is supplied to the compressed data buffer (cpb) 103 and temporarily accumulated.
- each video stream stored in the compressed data buffer 103 is read out, converted into a PES packet, further converted into a transport packet, and multiplexed to obtain a transport stream TS as a multiplexed stream.
- display control information (offset type stamp information, AU presentation control: see FIGS. 29 and 30) is displayed in the field of a packet for containerizing a video stream, for example, a PES extension (PES extension) of a PES packet. Is inserted.
- identification information temporary control descriptor: refer to FIG. 32
- AU timing control SEI au_timing_control SEI
- identification information for identifying whether each of a predetermined number of video streams is a base stream or an enhanced stream is inserted into the transport stream TS layer.
- This identification information is, for example, a stream type inserted into a video elementary stream loop (Video (ES loop) arranged corresponding to each video stream.
- the stream type of the base stream is “0x24”
- the stream type of the enhanced stream is, for example, “0x25” that is newly defined.
- the transport stream TS generated by the multiplexer 104 is sent to the transmission unit 105.
- the transport stream TS is modulated by a modulation method suitable for broadcasting such as QPSK / OFDM, and an RF modulation signal is transmitted from the transmission antenna.
- FIG. 37 shows a configuration example of the receiving device 200.
- the receiving apparatus 200 includes a CPU (Central Processing Unit) 201, a receiving unit 202, a demultiplexer 203, and a compressed data buffer (cpb: coded picture buffer) 204.
- the receiving apparatus 200 includes a decoder 205, an uncompressed data buffer (dpb: decoded picture buffer) 206, and a post processing unit 207.
- the CPU 201 constitutes a control unit and controls the operation of each unit of the receiving device 200.
- the receiving unit 202 demodulates the RF modulation signal received by the receiving antenna, and acquires the transport stream TS.
- the demultiplexer 203 selectively extracts, from the transport stream TS, encoded image data of a picture of a layer set corresponding to a decoding capability (Decoder temporal layer capability) and sends it to a compressed data buffer (cpb: coded picture buffer) 204. .
- FIG. 38 shows a configuration example of the demultiplexer 203.
- the demultiplexer 203 includes a PID processing unit 231, N stream processing units 232-1 to 232-N, and a stream integration unit 233.
- the PID processing unit 231 performs filtering based on the packet identifier (PID) by the decoding capability, and extracts a predetermined number of video streams including at least the base stream.
- Each video stream extracted by the PID processing unit 231 is sent to a corresponding stream processing unit.
- the stream processing unit 232-1 processes the base stream
- the stream processing units 232-2 to 232-N process the enhanced stream.
- the stream processing unit 232-1 includes a section parsing unit 241, a PES packet parsing unit 242, a PES header extracting unit 243, and a PES payload extracting unit 244.
- the section analysis unit 241 analyzes the section data of the target video stream, acquires the presence information of the AU timing control SEI in the encoded image data based on, for example, the temporal control descriptor, and sends it to the CPU 201.
- the PES packet analysis unit 242 analyzes the PES packet. Based on the analysis of the PES header analysis unit 243 and the PES packet analysis unit 242, the PES header extraction unit acquires the PTS and DTS inserted in the header, and further the offset time stamp inserted in the PES extension, and sends it to the CPU 201. The CPU 201 corrects the PTS and DTS with the offset time stamp, and determines the decoder timing and display timing.
- the PES payload extraction unit 244 extracts encoded image data of a picture included in the PES payload based on the analysis of the PES packet analysis unit 242, and sends the encoded image data to the stream integration unit 233.
- the stream processing units 232-2 to 232-N also perform the same processing as the above-described stream processing unit 232-1 to extract the encoded image data of the pictures included in the ES payload, and send them to the stream integration unit 233. send.
- the stream integration unit 233 integrates the encoded image data of each picture sent from the stream processing units 232-1 to 232-N and forms a compressed data buffer (cpb) as a single video stream (elementary video stream). ) 204.
- the stream integration unit 233 combines the video streams (encoded streams) output from the target layer selection unit 252 into one.
- the encoded image data of each picture is made into one stream based on the decoding timing information.
- FIG. 39 shows an example of stream combination. In this example, a base stream picture at 60 Hz intervals and an enhanced stream picture at 60 Hz intervals are combined. In this case, each picture is a stream with a time stamp of 120 Hz.
- FIG. 40 shows an example of the processing flow of the demultiplexer 203.
- the demultiplexer 203 starts processing, and then extracts a TS payload in step ST32.
- the demultiplexer 203 performs PID analysis to determine whether the section is a section.
- the demultiplexer 203 demultiplexes the packet that passes through the corresponding PID filter and performs section parsing.
- the demultiplexer 203 analyzes the temporal control descriptor (Temporal_control_descriptor) in step ST35.
- the demultiplexer 203 determines whether there is an AU timing control SEI (au_timing_control_SEI).
- the demultiplexer 203 checks the PES extension (PES extension) of the PES header (PES header) in step ST37.
- the demultiplexer 203 proceeds to step ST38 after the process of step ST37 and ends the process.
- the demultiplexer 203 immediately proceeds to step ST38 and ends the process.
- the demultiplexer 203 extracts a PES packet in step ST39.
- the demultiplexer 203 determines whether it is a PES header. When it is a PES header, the demultiplexer 203 determines in step ST42 whether the PES extension includes offset time stamp information (offset_timestamp_information).
- the demultiplexer 203 When there is offset type stamp information, the demultiplexer 203 notifies the system, that is, the CPU 201 of the offset time stamp (offset_to_DTS, offset_to_PTS) in step ST43. Thereafter, the demultiplexer 203 proceeds to the process of step ST44. When the offset / time stamp / information is found in step ST42, the demultiplexer 203 immediately moves to the process in step ST44.
- step ST44 the demultiplexer 203 determines whether there is a DTS or PTS. When there are DTS and PTS, the demultiplexer 203 proceeds to the process of step ST46. On the other hand, when there is no DTS and PTS, the demultiplexer 203 interpolates and generates the DTS and PTS in step ST45, and then proceeds to the processing of step ST46. In step ST46, the demultiplexer 203 notifies the system, that is, the CPU 201 of DTS and PTS. Thereafter, the demultiplexer 203 moves to step ST38 and ends the process.
- the demultiplexer 203 extracts the PES payload (PES payload) in step ST47.
- PES payload PES payload
- step ST ⁇ b> 48 the demultiplexer 203 combines the encoded streams to be subjected to PID in the order of DTS and transfers the combined streams to the compressed data buffer (cpb) 204. Thereafter, the demultiplexer 203 moves to step ST38 and ends the process.
- the compressed data buffer (cpb) 204 temporarily stores the video stream (encoded stream) extracted by the demultiplexer 203.
- the decoder 205 extracts from the video stream stored in the compressed data buffer 204 encoded image data of a picture of a hierarchy designated as a hierarchy to be decoded. Then, the decoder 205 decodes the encoded image data of each extracted picture at the decoding timing of the picture, and sends the decoded picture data to the uncompressed data buffer (dpb) 206.
- the hierarchy to be decoded from the CPU 201 is designated by temporal_id.
- This designated layer is the entire layer included in the video stream (encoded stream) extracted by the demultiplexer 203 or a part of the lower layer, and is set automatically by the CPU 201 or in response to a user operation.
- the decoder 205 is given a decoding timing from the CPU 201 based on DTS (Decoding Time stamp). Note that the decoder 205 reads the image data of the referenced picture from the uncompressed data buffer 206 and uses it as necessary when decoding the encoded image data of each picture.
- DTS Decoding Time stamp
- FIG. 41 shows a configuration example of the decoder 205.
- the decoder 205 includes a temporal ID analysis unit 251, a target hierarchy selection unit 252, and a decoding unit 254.
- the temporal ID analysis unit 251 reads the video stream (encoded stream) stored in the compressed data buffer 204 and analyzes temporal_id inserted in the NAL unit header of the encoded image data of each picture.
- the target hierarchy selection unit 252 extracts, from the video stream read from the compressed data buffer 204, encoded image data of a picture of a hierarchy designated as a hierarchy to be decoded based on the analysis result of the temporal ID analysis unit 251. .
- the encoded image data of each picture extracted by the target hierarchy selection unit 252 is sent to the decoding unit 254.
- the decoding unit 254 sequentially decodes the encoded image data of each picture at the decoding timing and sends the decoded image data to the uncompressed data buffer (dpb) 206.
- the decoding unit 254 analyzes the VPS and SPS, for example, grasps the bit rate level designation value “sublayer_level_idc” for each sublayer, and confirms whether or not the decoding can be performed within the decoding capability.
- the decoding unit 254 analyzes the SEI, grasps, for example, “initial_cpb_removal_time” and “cpb_removal_delay”, and confirms whether the decoding timing from the CPU 201 is appropriate.
- the decoding unit 254 analyzes the AU timing control SEI (au_timing_control SEI) and sends the included display control information to the CPU 201.
- the CPU 201 controls reading of each picture from the compressed data buffer (dpb) 206 based on the display control information, and as described above, the “switching display image 2” or “display image 3” is performed in the sequence switching unit. (See FIGS. 12, 14, 18 to 25).
- the decoding unit 254 acquires “ref_idx_l0_active (ref_idx_l1_active) as information indicating a prediction destination in the time direction from the slice header (Slice header), and performs prediction in the time direction. Note that the decoded picture is processed as a reference by another picture, using “short_term_ref_pic_set_idx” or “it_idx_sps” obtained from the slice header (slice header) as an index.
- the uncompressed data buffer (dpb) 206 temporarily stores the image data of each picture decoded by the decoder 205.
- the post processing unit 207 performs processing for matching the frame rate with the display capability for the image data of each picture sequentially read from the uncompressed data buffer (dpb) 206 at the display timing. In this case, display timing is given from the CPU 201 based on PTS (Presentation Time stamp).
- the post processing unit 207 sends the image data of each decoded picture to the display as it is.
- the post processing unit 207 has a temporal resolution for the image data of each picture after decoding. Sub-sample processing is performed so as to be 1/2, and the image data is sent to the display as 60 fps image data.
- the post processing unit 207 has a temporal resolution for the image data of each picture after decoding. Interpolation processing is performed so as to be doubled, and it is sent to the display as 120 fps image data. For example, when the frame rate of the image data of each picture after decoding is 60 fps and the display capability is 60 fps, the post processing unit 207 sends the image data of each decoded picture to the display as it is.
- FIG. 42 shows a configuration example of the post processing unit 207.
- the frame rate of the image data of each picture after decoding is 120 fps or 60 fps and the display capability is 120 fps or 60 fps.
- the post processing unit 207 includes an interpolation unit 271, a subsample unit 272, and a switch unit 273.
- the decoded image data of each picture from the uncompressed data buffer 206 is directly input to the switch unit 273, or is input to the switch unit 273 after the frame rate is doubled by the interpolation unit 271, or the subsample unit After the frame rate is set to 1/2 the frame rate at 272, the frame rate is input to the switch unit 273.
- Selection information is supplied from the CPU 201 to the switch unit 273.
- the selection information is generated automatically by the CPU 201 with reference to the display capability or in response to a user operation.
- the switch unit 273 selectively outputs any of the inputs based on the selection information.
- the frame rate of the image data of each picture sequentially read from the uncompressed data buffer (dpb) 206 at the display timing is adapted to the display capability.
- FIG. 43 shows an example of the processing flow of the decoder 205 and post processing unit 207.
- the decoder 205 and the post processing unit 207 start the processing, and then move to the processing in step ST52.
- the decoder 205 reads out the decoding target video stream stored in the compressed data buffer (cpb) 204, and selects a picture in a hierarchy designated as a decoding target from the CPU 201 based on temporal_id.
- step ST53 the decoder 205 sequentially decodes the encoded image data of each selected picture at the decode timing, and transfers the decoded image data of each picture to the uncompressed data buffer (dpb) 206. , Temporarily accumulate.
- step ST54 the post processing unit 207 reads the image data of each picture from the uncompressed data buffer (dpb) 206 at the display timing.
- reading is controlled based on the display control information.
- the read control as described above, there are a repeat control and a skip control for realizing the “display image 3”, and a read timing control for realizing the “display image 2”.
- AU timing control SEI au_timing_control SEI
- offset type stamp information offset_timestamp_information
- AU presentation control au_presentation_control
- step ST55 the post processing unit 207 determines whether or not the frame rate of the read image data of each picture matches the display capability.
- the post processing unit 207 sends the frame rate to the display in accordance with the display capability in step ST56, and then ends the processing in step ST57.
- the post processing unit 207 sends the frame rate as it is to the display in step ST58, and then ends the process in step ST57.
- the RF modulation signal received by the reception antenna is demodulated, and the transport stream TS is acquired.
- This transport stream TS is sent to the demultiplexer 203.
- the demultiplexer 203 all or a part of the video stream is subjected to PID filtering from the transport stream TS according to the decoding capability (Decoder temporal layer capability).
- the decoding capability when the decoding capability is high, all video streams of the base stream and the enhanced stream are selected. For example, when the decoding capability is low, a predetermined number of video streams including a base stream including a hierarchy capable of decoding are selected. Then, from the demultiplexer 203, the encoded image data of the picture of the selected video stream is sent to the compressed data buffer (cpb) 204 and temporarily stored.
- cpb compressed data buffer
- encoded image data of a picture of a hierarchy designated as a hierarchy to be decoded is extracted from the video stream stored in the compressed data buffer 204. Then, the decoder 205 decodes the extracted encoded image data of each picture at the decoding timing of the picture, sends it to the uncompressed data buffer (dpb) 206, and temporarily accumulates it. In this case, when the encoded image data of each picture is decoded, the image data of the referenced picture is read from the uncompressed data buffer 206 and used as necessary.
- Image data of each picture sequentially read from the uncompressed data buffer (dpb) 206 at the display timing is sent to the post processing unit 207.
- the post processing unit 207 interpolation or sub-sampling is performed on the image data of each picture so that the frame rate matches the display capability.
- the image data of each picture processed by the post processing unit 207 is supplied to a display, and a moving image is displayed using the image data of each picture.
- the sequence of the video stream to be transmitted is changed from the first sequence to the second sequence in which the frame rate is different from the first sequence.
- display control information is inserted and transmitted at least in the encoded image data and / or PES extension of the picture corresponding to the switching unit. Therefore, for example, the receiving side can control the reading of image data of each picture from the uncompressed data buffer based on this display control information, and even if the frame rate changes dynamically, a display gap is generated. A good display can be achieved without causing it to occur.
- the transmitting side identifies the last picture of the first sequence in the encoded image data and / or PES extension of the last picture of the first sequence. Identification information to be inserted is inserted. Therefore, for example, when the sequence of the video stream to be transmitted has a switching unit that switches from the first sequence to the second sequence having a frame rate different from that of the first sequence, the receiving side includes this identification information. Based on this, the last picture of the first sequence can be easily identified.
- identification information for identifying whether or not display control information is inserted into the encoded image data is inserted into the container layer on the transmission side. Therefore, for example, on the receiving side, it is possible to easily identify whether or not the display control information is inserted into the encoded image data based on the identification information without decoding the encoded image data.
- image data of each picture obtained by decoding is taken into a buffer, and the image data of each picture is read and output at a predetermined timing. This reading is controlled based on the encoded image data and / or display control information inserted in the PES extension. Therefore, for example, even when the frame rate changes dynamically, good display can be performed without generating a display gap.
- display control information is inserted into the encoded image data and / or PES extension on the transmission side, and display control is performed on the reception side based on the display control information.
- the receiving side detects the switching part of the sequence of the video stream to be transmitted and performs similar display control.
- a display period can be obtained from “clock_tick” and “time_scale” of VPS or SPS, and a sequence switching portion can be detected by the change in the display period.
- the transmission / reception system 10 including the transmission device 100 and the reception device 200 is shown.
- the configuration of the transmission / reception system to which the present technology can be applied is not limited to this.
- the receiving device 200 may have a configuration of a set-top box and a monitor connected by a digital interface such as (High-Definition Multimedia Interface (HDMI)). Trademark.
- HDMI High-Definition Multimedia Interface
- the container is a transport stream (MPEG-2 TS)
- MPEG-2 TS transport stream
- the present technology can be similarly applied to a system configured to be distributed to receiving terminals using a network such as the Internet.
- the Internet distribution it is often distributed in a container of MP4 or other formats.
- containers of various formats such as transport stream (MPEG-2 TS) adopted in the digital broadcasting standard and MP4 used in Internet distribution correspond to the container.
- the encoding is performed so that the decoding interval of the encoded image data of the pictures in the lowest layer set is equal.
- the present technology is not necessarily limited to this. It is not necessary that the encoding is performed at a predetermined interval.
- this technique can also take the following structures.
- (1) The image data of each picture constituting the moving image data is classified into a plurality of hierarchies, the image data of the classified pictures of each hierarchy is encoded, and the plurality of hierarchies are grouped into a predetermined number of hierarchies.
- An image encoding unit that divides and generates the predetermined number of video streams each having the encoded image data of each of the divided groups of pictures, The image encoding unit performs encoding so that at least a decoding interval of encoded image data of pictures in the lowest layer set is a predetermined interval,
- a transmission unit that transmits a container of a predetermined format including a video stream having encoded image data of at least the lowest hierarchical set of pictures among the predetermined number of generated video streams;
- the sequence of the video stream to be transmitted has a switching unit that switches from the first sequence to the second sequence having a frame rate different from that of the first sequence, at least the code of the picture corresponding to the switching unit
- a transmission apparatus further comprising: an encoded image data or an information insertion unit that inserts display control information into a packet that containers the encoded image data.
- the display control information is The display of the last picture of the first sequence is controlled to be repeated by a predetermined number, or the display of the picture of the second sequence is skipped during the display period of the last picture of the first sequence.
- the transmission device according to (1) which is information to be controlled.
- the display control information is The transmission apparatus according to (1), wherein the transmission offset information controls display timing of the second sequence of pictures.
- the container is a transport stream, The information insertion part The transmission apparatus according to (1), wherein the display control information is inserted into an extension field of a PES packet.
- the information insertion unit The identification information for identifying that it is the last picture of the first sequence is further inserted into the encoded image data of the last picture of the first sequence. Any one of (1) to (4) Transmitter.
- the identification information insertion unit for inserting identification information for identifying whether or not the display control information is inserted into the encoded image data is further provided in the container layer.
- the transmission device according to any one of the above.
- the container is a transport stream
- the identification information insertion unit The transmitting apparatus according to (6), wherein the identification information is inserted as a descriptor in a video elementary stream loop arranged corresponding to the predetermined number of video streams under a program map table.
- the image encoding unit The decoding timing of the encoded image data of the pictures of the hierarchical group positioned higher than the lowest hierarchical group is intermediate between the decoding timings of the encoded image data of the pictures of all the hierarchical groups positioned lower than the hierarchical group.
- the transmission device according to any one of (1) to (7), wherein encoding is performed so that timing is reached. (9)
- the image data of each picture constituting the moving image data is classified into a plurality of hierarchies, the image data of the classified pictures of each hierarchy is encoded, and the plurality of hierarchies are grouped into a predetermined number of hierarchies.
- encoding is performed so that at least the decoding interval of the encoded image data of the pictures in the lowest layer set is a predetermined interval
- the sequence of the video stream to be transmitted has a switching unit that switches from the first sequence to the second sequence having a frame rate different from that of the first sequence, at least the code of the picture corresponding to the switching unit
- An information insertion step of inserting display control information into encoded image data or a packet for containerizing the encoded image data.
- Image data of each picture constituting the moving image data is classified and encoded into a plurality of layers, and each layer obtained by dividing the plurality of layers into a predetermined number of layer sets
- a processing unit for processing the received video stream Among the predetermined number of video streams, at least the video stream having the encoded image data of the picture of the lowest layer set is encoded so that the decoding interval of each picture becomes a predetermined interval,
- the sequence of the received video stream has a switching unit that switches from the first sequence to the second sequence having a different frame rate from the first sequence, at least the code of the picture corresponding to the switching unit
- the display control information is The display of the last picture of the first sequence is controlled to be repeated by a predetermined number, or the display of the picture of the second sequence is skipped during the display period of the last picture of the first sequence.
- the display control information is The receiving apparatus according to (10), wherein the receiving apparatus is display offset information for controlling display timing of the picture of the second sequence.
- the image data of each picture constituting the moving image data is classified and encoded into a plurality of layers, and each layer obtained by dividing the plurality of layers into a predetermined number of layer sets
- the predetermined number of video streams at least the video stream having the encoded image data of the picture of the lowest layer set is encoded so that the decoding interval of each picture becomes a predetermined interval,
- the sequence of the video stream included in the received container has a switching unit that switches from the first sequence to the second sequence having a different frame rate from the first sequence, at least the switching unit corresponds to Display control information is inserted into encoded image data of a picture to be encoded or a packet for containerizing the encoded image data, From the video stream included in the received container, the encoded image data of a picture of a layer below
- An image decoding processing unit that captures, reads out and outputs image data of each picture at a predetermined timing;
- a receiving device further comprising: a control unit that controls reading of an image of each picture from the buffer in the image decoding processing unit based on the display control information.
- the display control information is The display of the last picture of the first sequence is controlled to be repeated by a predetermined number, or the display of the picture of the second sequence is skipped during the display period of the last picture of the first sequence.
- the receiving device according to (13), which is information to be controlled.
- the display control information is The receiving apparatus according to (13), which is display offset information for controlling display timing of the picture of the second sequence.
- Image data of each picture constituting moving image data is classified and encoded into a plurality of layers, and each layer obtained by dividing the plurality of layers into a predetermined number of layer sets
- a switching detection unit From the video stream included in the received container, the encoded image data of a picture of a layer below a predetermined layer corresponding to the decoding capability is selectively decoded, and the image data of each picture obtained by the decoding is stored in a buffer.
- An image decoding processing unit that captures, reads out and outputs image data of each picture at a predetermined timing;
- a receiving device further comprising: a control unit that controls reading of an image of each picture from the buffer in the image decoding processing unit based on a detection output of the sequence switching detection unit.
- the main feature of the present technology is that when the sequence of the transmitted video stream is switched from the first sequence to the second sequence having a frame rate different from that of the first sequence, at least the switching unit corresponds to the switching unit.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Business, Economics & Management (AREA)
- Marketing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Television Systems (AREA)
Abstract
Description
動画像データを構成する各ピクチャの画像データを複数の階層に分類し、該分類された各階層のピクチャの画像データを符号化すると共に、上記複数の階層を所定数の階層組に分割し、該分割された各階層組のピクチャの符号化画像データをそれぞれ持つ上記所定数のビデオストリームを生成する画像符号化部を備え、
上記画像符号化部は、少なくとも、最下位の階層組のピクチャの符号化画像データのデコード間隔が所定間隔となるように符号化し、
上記生成された所定数のビデオストリームのうち少なくとも最下位の階層組のピクチャの符号化画像データを持つビデオストリームを含む所定フォーマットのコンテナを送信する送信部と、
上記送信されるビデオストリームのシーケンスが第1のシーケンスから該第1のシーケンスとはフレームレートを異にする第2のシーケンスに切り替わる切り替わり部を持つ場合、少なくとも、該切り替わり部に対応するピクチャの符号化画像データ、または該符号化画像データをコンテナするパケットに表示制御情報を挿入する情報挿入部とをさらに備える
送信装置にある。
動画像データを構成する各ピクチャの画像データが複数の階層に分類されて符号化されると共に、上記複数の階層が所定数の階層組に分割されることで得られた、各階層組のピクチャの符号化画像データをそれぞれ持つ上記所定数のビデオストリームのうち少なくとも最下位の階層組のピクチャの符号化画像データを持つビデオストリームを受信する受信部と、
上記受信されたビデオストリームを処理する処理部とを備え、
上記所定数のビデオストリームのうち、少なくとも、最下位の階層組のピクチャの符号化画像データを持つビデオストリームは、各ピクチャのデコード間隔が所定間隔となるように符号化されており、
上記受信されたビデオストリームのシーケンスが第1のシーケンスから該第1のシーケンスとはフレームレートを異にする第2のシーケンスに切り替わる切り替わり部を持つ場合、少なくとも、該切り替え部に対応するピクチャの符号化画像データ、または該符号化画像データをコンテナするパケットに表示制御情報が挿入されている
受信装置にある。
動画像データを構成する各ピクチャの画像データが複数の階層に分類されて符号化されると共に、上記複数の階層が所定数の階層組に分割されることで得られた、各階層組のピクチャの符号化画像データをそれぞれ持つ上記所定数のビデオストリームのうち少なくとも最下位の階層組のピクチャの符号化画像データを持つビデオストリームを含む所定フォーマットのコンテナを受信する受信部を備え、
上記所定数のビデオストリームのうち、少なくとも、最下位の階層組のピクチャの符号化画像データを持つビデオストリームは、各ピクチャのデコード間隔が所定間隔となるように符号化されており、
上記受信されたビデオストリームのシーケンスが第1のシーケンスから該第1のシーケンスとはフレームレートを異にする第2のシーケンスに切り替わる切り替わり部を持つ場合、少なくとも、該切り替え部に対応するピクチャの符号化画像データに表示制御情報が挿入されており、
上記受信されたコンテナに含まれるビデオストリームからデコード能力に応じた所定階層以下の階層のピクチャの符号化画像データを選択的にデコードし、該デコードされて得られた各ピクチャの画像データをバッファに取り込み、該各ピクチャの画像データを所定のタイミングで読み出して出力する画像復号処理部と、
上記画像復号処理部における上記バッファからの各ピクチャの画像の読み出しを上記表示制御情報に基づいて制御する制御部とをさらに備える
受信装置にある。
動画像データを構成する各ピクチャの画像データが複数の階層に分類されて符号化されると共に、上記複数の階層が所定数の階層組に分割されることで得られた、各階層組のピクチャの符号化画像データをそれぞれ持つ上記所定数のビデオストリームのうち少なくとも最下位の階層組のピクチャの符号化画像データを持つビデオストリームを含む所定フォーマットのコンテナを受信する受信部を備え、
上記所定数のビデオストリームのうち、少なくとも、最下位の階層組のピクチャの符号化画像データを持つビデオストリームは、各ピクチャのデコード間隔が所定間隔となるように符号化されており、
上記受信されたコンテナに含まれるビデオストリームに基づいて該ビデオストリームのシーケンスが第1のシーケンスから該第1のシーケンスとはフレームレートを異にする第2のシーケンスに切り替わる切り替わり部を検出するシーケンス切り替わり検出部と、
上記受信されたコンテナに含まれるビデオストリームからデコード能力に応じた所定階層以下の階層のピクチャの符号化画像データを選択的にデコードし、該デコードされて得られた各ピクチャの画像データをバッファに取り込み、該各ピクチャの画像データを所定のタイミングで読み出して出力する画像復号処理部と、
上記画像復号処理部における上記バッファからの各ピクチャの画像の読み出しを上記シーケンス切り替わり検出部の検出出力に基づいて制御する制御部とをさらに備える
受信装置にある。
1.実施の形態
2.変形例
[送受信システム]
図1は、実施の形態としての送受信システム10の構成例を示している。この送受信システム10は、送信装置100と、受信装置200とを有する構成となっている。
図2は、送信装置100の構成例を示している。この送信装置100は、CPU(Central Processing Unit)101と、エンコーダ102と、圧縮データバッファ(cpb:coded picture buffer)103と、マルチプレクサ104と、送信部105を有している。CPU101は、制御部であり、送信装置100の各部の動作を制御する。
図37は、受信装置200の構成例を示している。この受信装置200は、CPU(Central Processing Unit)201と、受信部202と、デマルチプレクサ203と、圧縮データバッファ(cpb:coded picture buffer)204を有している。また、この受信装置200は、デコーダ205と、非圧縮データバッファ(dpb:decoded picture buffer)206と、ポスト処理部207を有している。CPU201は、制御部を構成し、受信装置200の各部の動作を制御する。
なお、上述実施の形態においては、送信側において符号化画像データおよび/またはPESエクステンションに表示制御情報が挿入され、受信側では、その表示制御情報に基づいて表示制御を行う構成となっている。しかし、符号化画像データおよび/またはPESエクステンションに表示制御情報が挿入されていない場合であっても、受信側において、送信されてくるビデオストリームのシーケンスの切り替わり部を検出して、同様の表示制御を行うことが考えられる。この場合、例えば、VPSあるいはSPSの「clock_tick」と「time_scale」から表示期間を求め、この表示期間の変化によりシーケンスの切り替わり部を検出することが可能となる。
(1)動画像データを構成する各ピクチャの画像データを複数の階層に分類し、該分類された各階層のピクチャの画像データを符号化すると共に、上記複数の階層を所定数の階層組に分割し、該分割された各階層組のピクチャの符号化画像データをそれぞれ持つ上記所定数のビデオストリームを生成する画像符号化部を備え、
上記画像符号化部は、少なくとも、最下位の階層組のピクチャの符号化画像データのデコード間隔が所定間隔となるように符号化し、
上記生成された所定数のビデオストリームのうち少なくとも最下位の階層組のピクチャの符号化画像データを持つビデオストリームを含む所定フォーマットのコンテナを送信する送信部と、
上記送信されるビデオストリームのシーケンスが第1のシーケンスから該第1のシーケンスとはフレームレートを異にする第2のシーケンスに切り替わる切り替わり部を持つ場合、少なくとも、該切り替わり部に対応するピクチャの符号化画像データ、または該符号化画像データをコンテナするパケットに表示制御情報を挿入する情報挿入部とをさらに備える
送信装置。
(2)上記表示制御情報は、
上記第1のシーケンスの最後のピクチャの表示を所定数だけリピートするように制御する、あるいは上記第1のシーケンスの最後のピクチャの表示期間は上記第2のシーケンスのピクチャの表示をスキップするように制御する情報である
前記(1)に記載の送信装置。
(3)上記表示制御情報は、
上記第2のシーケンスのピクチャの表示タイミングを制御する表示オフセット情報である
前記(1)に記載の送信装置。
(4)上記コンテナはトランスポートストリームであり、
上記情報挿入部は、
上記表示制御情報をPESパケットのエクステンションフィールドに挿入する
前記(1)に記載の送信装置。
(5)上記情報挿入部は、
上記第1のシーケンスの最後のピクチャの符号化画像データに、該第1のシーケンスの最後のピクチャであることを識別する識別情報をさらに挿入する
前記(1)から(4)のいずれかに記載の送信装置。
(6)上記コンテナのレイヤに、上記符号化画像データに上記表示制御情報が挿入されているか否かを識別する識別情報を挿入する識別情報挿入部をさらに備える
前記(1)から(5)のいずれかに記載の送信装置。
(7)上記コンテナはトランスポートストリームであり、
上記識別情報挿入部は、
上記識別情報を、プログラムマップテーブルの配下に上記所定数のビデオストリームにそれぞれ対応して配置されたビデオエレメンタリストリームループの中にデスクリプタとして挿入する
前記(6)に記載の送信装置。
(8)上記画像符号化部は、
上記最下位の階層組より上位に位置する階層組のピクチャの符号化画像データのデコードタイミングが、該階層組より下位側に位置するすべての階層組のピクチャの符号化画像データのデコードタイミングの中間タイミングとなるように符号化する
前記(1)から(7)のいずれかに記載の送信装置。
(9)動画像データを構成する各ピクチャの画像データを複数の階層に分類し、該分類された各階層のピクチャの画像データを符号化すると共に、上記複数の階層を所定数の階層組に分割し、該分割された各階層組のピクチャの符号化画像データをそれぞれ持つ上記所定数のビデオストリームを生成する画像符号化ステップを有し、
上記画像符号化ステップでは、少なくとも、最下位の階層組のピクチャの符号化画像データのデコード間隔が所定間隔となるように符号化し、
送信部により、上記画像符号化ステップで生成された所定数のビデオストリームのうち少なくとも最下位の階層組のピクチャの符号化画像データを持つビデオストリームを含む所定フォーマットのコンテナを送信する送信ステップと、
上記送信されるビデオストリームのシーケンスが第1のシーケンスから該第1のシーケンスとはフレームレートを異にする第2のシーケンスに切り替わる切り替わり部を持つ場合、少なくとも、該切り替わり部に対応するピクチャの符号化画像データ、または該符号化画像データをコンテナするパケットに表示制御情報を挿入する情報挿入ステップとをさらに有する
送信方法。
(10)動画像データを構成する各ピクチャの画像データが複数の階層に分類されて符号化されると共に、上記複数の階層が所定数の階層組に分割されることで得られた、各階層組のピクチャの符号化画像データをそれぞれ持つ上記所定数のビデオストリームのうち少なくとも最下位の階層組のピクチャの符号化画像データを持つビデオストリームを受信する受信部と、
上記受信されたビデオストリームを処理する処理部とを備え、
上記所定数のビデオストリームのうち、少なくとも、最下位の階層組のピクチャの符号化画像データを持つビデオストリームは、各ピクチャのデコード間隔が所定間隔となるように符号化されており、
上記受信されたビデオストリームのシーケンスが第1のシーケンスから該第1のシーケンスとはフレームレートを異にする第2のシーケンスに切り替わる切り替わり部を持つ場合、少なくとも、該切り替え部に対応するピクチャの符号化画像データ、または該符号化画像データをコンテナするパケットに表示制御情報が挿入されている
受信装置。
(11)上記表示制御情報は、
上記第1のシーケンスの最後のピクチャの表示を所定数だけリピートするように制御する、あるいは上記第1のシーケンスの最後のピクチャの表示期間は上記第2のシーケンスのピクチャの表示をスキップするように制御する情報である
前記(10)に記載の受信装置。
(12)上記表示制御情報は、
上記第2のシーケンスのピクチャの表示タイミングを制御する表示オフセット情報である
前記(10)に記載の受信装置。
(13)動画像データを構成する各ピクチャの画像データが複数の階層に分類されて符号化されると共に、上記複数の階層が所定数の階層組に分割されることで得られた、各階層組のピクチャの符号化画像データをそれぞれ持つ上記所定数のビデオストリームのうち少なくとも最下位の階層組のピクチャの符号化画像データを持つビデオストリームを含む所定フォーマットのコンテナを受信する受信部を備え、
上記所定数のビデオストリームのうち、少なくとも、最下位の階層組のピクチャの符号化画像データを持つビデオストリームは、各ピクチャのデコード間隔が所定間隔となるように符号化されており、
上記受信されたコンテナに含まれるビデオストリームのシーケンスが第1のシーケンスから該第1のシーケンスとはフレームレートを異にする第2のシーケンスに切り替わる切り替わり部を持つ場合、少なくとも、該切り替え部に対応するピクチャの符号化画像データ、または該符号化画像データをコンテナするパケットに表示制御情報が挿入されており、
上記受信されたコンテナに含まれるビデオストリームからデコード能力に応じた所定階層以下の階層のピクチャの符号化画像データを選択的にデコードし、該デコードされて得られた各ピクチャの画像データをバッファに取り込み、該各ピクチャの画像データを所定のタイミングで読み出して出力する画像復号処理部と、
上記画像復号処理部における上記バッファからの各ピクチャの画像の読み出しを上記表示制御情報に基づいて制御する制御部とをさらに備える
受信装置。
(14)上記表示制御情報は、
上記第1のシーケンスの最後のピクチャの表示を所定数だけリピートするように制御する、あるいは上記第1のシーケンスの最後のピクチャの表示期間は上記第2のシーケンスのピクチャの表示をスキップするように制御する情報である
前記(13)に記載の受信装置。
(15)上記表示制御情報は、
上記第2のシーケンスのピクチャの表示タイミングを制御する表示オフセット情報である
前記(13)に記載の受信装置。
(16)動画像データを構成する各ピクチャの画像データが複数の階層に分類されて符号化されると共に、上記複数の階層が所定数の階層組に分割されることで得られた、各階層組のピクチャの符号化画像データをそれぞれ持つ上記所定数のビデオストリームのうち少なくとも最下位の階層組のピクチャの符号化画像データを持つビデオストリームを含む所定フォーマットのコンテナを受信する受信部を備え、
上記所定数のビデオストリームのうち、少なくとも、最下位の階層組のピクチャの符号化画像データを持つビデオストリームは、各ピクチャのデコード間隔が所定間隔となるように符号化されており、
上記受信されたコンテナに含まれるビデオストリームに基づいて、該ビデオストリームのシーケンスが第1のシーケンスから該第1のシーケンスとはフレームレートを異にする第2のシーケンスに切り替わる切り替わり部を検出するシーケンス切り替わり検出部と、
上記受信されたコンテナに含まれるビデオストリームからデコード能力に応じた所定階層以下の階層のピクチャの符号化画像データを選択的にデコードし、該デコードされて得られた各ピクチャの画像データをバッファに取り込み、該各ピクチャの画像データを所定のタイミングで読み出して出力する画像復号処理部と、
上記画像復号処理部における上記バッファからの各ピクチャの画像の読み出しを上記シーケンス切り替わり検出部の検出出力に基づいて制御する制御部とをさらに備える
受信装置。
100・・・送信装置
101・・・CPU
102・・・エンコーダ
103・・・圧縮データバッファ(cpb)
104・・・マルチプレクサ
105・・・送信部
121・・・テンポラルID発生部
122・・・バッファ遅延制御部
123・・・HRD設定部
124・・・パラメータセット/SEIエンコード部
125・・・スライスエンコード部
126・・・NALパケット化部
142・・・セクションコーディング部
143-1~143-N・・・PESパケット化部
144・・・スイッチ部
145・・・トランスポートパケット化部
200・・・受信装置
201・・・CPU
202・・・受信部
203・・・デマルチプレクサ
204・・・圧縮データバッファ(cpb)
205・・・デコーダ
206・・・非圧縮データバッファ(dpb)
207・・・ポスト処理部
231・・・PID処理部
232-1~232-N・・・ストリーム処理部
233・・・ストリーム統合部
241・・・セクション・パース部
242・・・PESパケット・パース部
243・・・PESヘッダ抽出部
244・・・PESペイロード抽出部
251・・・テンポラルID解析部
252・・・対象階層選択部
254・・・デコード部
271・・・補間部
272・・・サブサンプル部
273・・・スイッチ部
Claims (16)
- 動画像データを構成する各ピクチャの画像データを複数の階層に分類し、該分類された各階層のピクチャの画像データを符号化すると共に、上記複数の階層を所定数の階層組に分割し、該分割された各階層組のピクチャの符号化画像データをそれぞれ持つ上記所定数のビデオストリームを生成する画像符号化部を備え、
上記画像符号化部は、少なくとも、最下位の階層組のピクチャの符号化画像データのデコード間隔が所定間隔となるように符号化し、
上記生成された所定数のビデオストリームのうち少なくとも最下位の階層組のピクチャの符号化画像データを持つビデオストリームを含む所定フォーマットのコンテナを送信する送信部と、
上記送信されるビデオストリームのシーケンスが第1のシーケンスから該第1のシーケンスとはフレームレートを異にする第2のシーケンスに切り替わる切り替わり部を持つ場合、少なくとも、該切り替わり部に対応するピクチャの符号化画像データ、または該符号化画像データをコンテナするパケットに表示制御情報を挿入する情報挿入部とをさらに備える
送信装置。 - 上記表示制御情報は、
上記第1のシーケンスの最後のピクチャの表示を所定数だけリピートするように制御する、あるいは上記第1のシーケンスの最後のピクチャの表示期間は上記第2のシーケンスのピクチャの表示をスキップするように制御する情報である
請求項1に記載の送信装置。 - 上記表示制御情報は、
上記第2のシーケンスのピクチャの表示タイミングを制御する表示オフセット情報である
請求項1に記載の送信装置。 - 上記コンテナはトランスポートストリームであり、
上記情報挿入部は、
上記表示制御情報をPESパケットのエクステンションフィールドに挿入する
請求項1に記載の送信装置。 - 上記情報挿入部は、
上記第1のシーケンスの最後のピクチャの符号化画像データに、該第1のシーケンスの最後のピクチャであることを識別する識別情報をさらに挿入する
請求項1に記載の送信装置。 - 上記コンテナのレイヤに、上記符号化画像データに上記表示制御情報が挿入されているか否かを識別する識別情報を挿入する識別情報挿入部をさらに備える
請求項1に記載の送信装置。 - 上記コンテナはトランスポートストリームであり、
上記識別情報挿入部は、
上記識別情報を、プログラムマップテーブルの配下に上記所定数のビデオストリームにそれぞれ対応して配置されたビデオエレメンタリストリームループの中にデスクリプタとして挿入する
請求項6に記載の送信装置。 - 上記画像符号化部は、
上記最下位の階層組より上位に位置する階層組のピクチャの符号化画像データのデコードタイミングが、該階層組より下位側に位置するすべての階層組のピクチャの符号化画像データのデコードタイミングの中間タイミングとなるように符号化する
請求項1に記載の送信装置。 - 動画像データを構成する各ピクチャの画像データを複数の階層に分類し、該分類された各階層のピクチャの画像データを符号化すると共に、上記複数の階層を所定数の階層組に分割し、該分割された各階層組のピクチャの符号化画像データをそれぞれ持つ上記所定数のビデオストリームを生成する画像符号化ステップを有し、
上記画像符号化ステップでは、少なくとも、最下位の階層組のピクチャの符号化画像データのデコード間隔が所定間隔となるように符号化し、
送信部により、上記画像符号化ステップで生成された所定数のビデオストリームのうち少なくとも最下位の階層組のピクチャの符号化画像データを持つビデオストリームを含む所定フォーマットのコンテナを送信する送信ステップと、
上記送信されるビデオストリームのシーケンスが第1のシーケンスから該第1のシーケンスとはフレームレートを異にする第2のシーケンスに切り替わる切り替わり部を持つ場合、少なくとも、該切り替わり部に対応するピクチャの符号化画像データ、または該符号化画像データをコンテナするパケットに表示制御情報を挿入する情報挿入ステップとをさらに有する
送信方法。 - 動画像データを構成する各ピクチャの画像データが複数の階層に分類されて符号化されると共に、上記複数の階層が所定数の階層組に分割されることで得られた、各階層組のピクチャの符号化画像データをそれぞれ持つ上記所定数のビデオストリームのうち少なくとも最下位の階層組のピクチャの符号化画像データを持つビデオストリームを受信する受信部と、
上記受信されたビデオストリームを処理する処理部とを備え、
上記所定数のビデオストリームのうち、少なくとも、最下位の階層組のピクチャの符号化画像データを持つビデオストリームは、各ピクチャのデコード間隔が所定間隔となるように符号化されており、
上記受信されたビデオストリームのシーケンスが第1のシーケンスから該第1のシーケンスとはフレームレートを異にする第2のシーケンスに切り替わる切り替わり部を持つ場合、少なくとも、該切り替え部に対応するピクチャの符号化画像データ、または該符号化画像データをコンテナするパケットに表示制御情報が挿入されている
受信装置。 - 上記表示制御情報は、
上記第1のシーケンスの最後のピクチャの表示を所定数だけリピートするように制御する、あるいは上記第1のシーケンスの最後のピクチャの表示期間は上記第2のシーケンスのピクチャの表示をスキップするように制御する情報である
請求項10に記載の受信装置。 - 上記表示制御情報は、
上記第2のシーケンスのピクチャの表示タイミングを制御する表示オフセット情報である
請求項10に記載の受信装置。 - 動画像データを構成する各ピクチャの画像データが複数の階層に分類されて符号化されると共に、上記複数の階層が所定数の階層組に分割されることで得られた、各階層組のピクチャの符号化画像データをそれぞれ持つ上記所定数のビデオストリームのうち少なくとも最下位の階層組のピクチャの符号化画像データを持つビデオストリームを含む所定フォーマットのコンテナを受信する受信部を備え、
上記所定数のビデオストリームのうち、少なくとも、最下位の階層組のピクチャの符号化画像データを持つビデオストリームは、各ピクチャのデコード間隔が所定間隔となるように符号化されており、
上記受信されたコンテナに含まれるビデオストリームのシーケンスが第1のシーケンスから該第1のシーケンスとはフレームレートを異にする第2のシーケンスに切り替わる切り替わり部を持つ場合、少なくとも、該切り替え部に対応するピクチャの符号化画像データ、または該符号化画像データをコンテナするパケットに表示制御情報が挿入されており、
上記受信されたコンテナに含まれるビデオストリームからデコード能力に応じた所定階層以下の階層のピクチャの符号化画像データを選択的にデコードし、該デコードされて得られた各ピクチャの画像データをバッファに取り込み、該各ピクチャの画像データを所定のタイミングで読み出して出力する画像復号処理部と、
上記画像復号処理部における上記バッファからの各ピクチャの画像の読み出しを上記表示制御情報に基づいて制御する制御部とをさらに備える
受信装置。 - 上記表示制御情報は、
上記第1のシーケンスの最後のピクチャの表示を所定数だけリピートするように制御する、あるいは上記第1のシーケンスの最後のピクチャの表示期間は上記第2のシーケンスのピクチャの表示をスキップするように制御する情報である
請求項13に記載の受信装置。 - 上記表示制御情報は、
上記第2のシーケンスのピクチャの表示タイミングを制御する表示オフセット情報である
請求項13に記載の受信装置。 - 動画像データを構成する各ピクチャの画像データが複数の階層に分類されて符号化されると共に、上記複数の階層が所定数の階層組に分割されることで得られた、各階層組のピクチャの符号化画像データをそれぞれ持つ上記所定数のビデオストリームのうち少なくとも最下位の階層組のピクチャの符号化画像データを持つビデオストリームを含む所定フォーマットのコンテナを受信する受信部を備え、
上記所定数のビデオストリームのうち、少なくとも、最下位の階層組のピクチャの符号化画像データを持つビデオストリームは、各ピクチャのデコード間隔が所定間隔となるように符号化されており、
上記受信されたコンテナに含まれるビデオストリームに基づいて、該ビデオストリームのシーケンスが第1のシーケンスから該第1のシーケンスとはフレームレートを異にする第2のシーケンスに切り替わる切り替わり部を検出するシーケンス切り替わり検出部と、
上記受信されたコンテナに含まれるビデオストリームからデコード能力に応じた所定階層以下の階層のピクチャの符号化画像データを選択的にデコードし、該デコードされて得られた各ピクチャの画像データをバッファに取り込み、該各ピクチャの画像データを所定のタイミングで読み出して出力する画像復号処理部と、
上記画像復号処理部における上記バッファからの各ピクチャの画像の読み出しを上記シーケンス切り替わり検出部の検出出力に基づいて制御する制御部とをさらに備える
受信装置。
Priority Applications (8)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
RU2016112623A RU2660957C2 (ru) | 2013-10-11 | 2014-10-01 | Устройство передачи, способ передачи и устройство приема |
JP2015541544A JP6558246B2 (ja) | 2013-10-11 | 2014-10-01 | 送信装置、送信方法および受信装置 |
US14/909,592 US10547857B2 (en) | 2013-10-11 | 2014-10-01 | Transmission device, transmission method and reception device |
BR112016007448-3A BR112016007448B1 (pt) | 2013-10-11 | 2014-10-01 | Dispositivos de transmissão e recepção, e, método de transmissão |
EP19190982.9A EP3591980A1 (en) | 2013-10-11 | 2014-10-01 | Reception device and reception method of video streams with changing frame rates |
EP14851673.5A EP3057330B1 (en) | 2013-10-11 | 2014-10-01 | Transmission device, transmission method, and reception device |
US16/707,085 US11025930B2 (en) | 2013-10-11 | 2019-12-09 | Transmission device, transmission method and reception device |
US17/234,226 US11589061B2 (en) | 2013-10-11 | 2021-04-19 | Transmission device, transmission method and reception device |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2013-214235 | 2013-10-11 | ||
JP2013214235 | 2013-10-11 |
Related Child Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/909,592 A-371-Of-International US10547857B2 (en) | 2013-10-11 | 2014-10-01 | Transmission device, transmission method and reception device |
US16/707,085 Continuation US11025930B2 (en) | 2013-10-11 | 2019-12-09 | Transmission device, transmission method and reception device |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2015053157A1 true WO2015053157A1 (ja) | 2015-04-16 |
Family
ID=52812973
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2014/076334 WO2015053157A1 (ja) | 2013-10-11 | 2014-10-01 | 送信装置、送信方法および受信装置 |
Country Status (6)
Country | Link |
---|---|
US (3) | US10547857B2 (ja) |
EP (2) | EP3057330B1 (ja) |
JP (5) | JP6558246B2 (ja) |
BR (1) | BR112016007448B1 (ja) |
RU (1) | RU2660957C2 (ja) |
WO (1) | WO2015053157A1 (ja) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2016054548A (ja) * | 2013-10-18 | 2016-04-14 | パナソニック株式会社 | 画像復号方法及び画像復号装置 |
WO2016190089A1 (ja) * | 2015-05-22 | 2016-12-01 | ソニー株式会社 | 送信装置、送信方法、画像処理装置、画像処理方法、受信装置および受信方法 |
JPWO2017164087A1 (ja) * | 2016-03-24 | 2019-02-14 | ソニー株式会社 | 送信装置、送信方法、受信装置および受信方法 |
WO2019069987A1 (ja) * | 2017-10-06 | 2019-04-11 | パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ | 符号化装置、復号装置、符号化方法および復号方法 |
JP2022009528A (ja) * | 2016-08-22 | 2022-01-14 | ソニーグループ株式会社 | 送信方法 |
JP2023526955A (ja) * | 2020-05-22 | 2023-06-26 | ジーイー ビデオ コンプレッション エルエルシー | ビデオエンコーダ、ビデオデコーダ、符号化及び復号のための方法、並びに高度なビデオ符号化概念を実現するためのビデオデータストリーム |
Families Citing this family (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CA3083172C (en) * | 2014-05-21 | 2022-01-25 | Arris Enterprises Llc | Individual buffer management in transport of scalable video |
US10257528B2 (en) * | 2015-10-08 | 2019-04-09 | Electronics And Telecommunications Research Institute | Method and apparatus for adaptive encoding and decoding based on image quality |
US10609356B1 (en) * | 2017-01-23 | 2020-03-31 | Amazon Technologies, Inc. | Using a temporal enhancement layer to encode and decode stereoscopic video content |
CN111183651A (zh) | 2017-10-12 | 2020-05-19 | 索尼公司 | 图像处理设备、图像处理方法、发送设备、发送方法和接收设备 |
EP3700198B1 (en) * | 2017-10-16 | 2024-04-03 | Sony Group Corporation | Imaging device, image processing apparatus, and image processing method |
JPWO2021261536A1 (ja) * | 2020-06-25 | 2021-12-30 | ||
US20230117444A1 (en) * | 2021-10-19 | 2023-04-20 | Microsoft Technology Licensing, Llc | Ultra-low latency streaming of real-time media |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2004180290A (ja) * | 2002-11-15 | 2004-06-24 | Sony Corp | 伝送装置と伝送方法と再生装置と再生方法およびプログラムと記録媒体 |
JP2008245325A (ja) * | 1997-10-31 | 2008-10-09 | Matsushita Electric Ind Co Ltd | 動画像符号化方法 |
JP2012124896A (ja) * | 2010-12-07 | 2012-06-28 | Mediatek Inc | 低遅延階層のbグループを有するピクチャ構造を用いた動画像符号化の方法及び装置 |
Family Cites Families (26)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6111610A (en) | 1997-12-11 | 2000-08-29 | Faroudja Laboratories, Inc. | Displaying film-originated video on high frame rate monitors without motions discontinuities |
CN100380958C (zh) | 2002-11-15 | 2008-04-09 | 索尼株式会社 | 发送装置、发送方法、再现装置、再现方法 |
US20130136189A9 (en) * | 2003-01-13 | 2013-05-30 | Jiang Fu | Method and system for converting a dss stream to an encrypted mpeg stream |
EP1604518A1 (en) * | 2003-03-03 | 2005-12-14 | Nokia Corporation | Method, system and network entity for indicating hierarchical mode for transport streams carried in broadband transmission |
US7852919B2 (en) * | 2003-09-07 | 2010-12-14 | Microsoft Corporation | Field start code for entry point frames with predicted first field |
US9258519B2 (en) * | 2005-09-27 | 2016-02-09 | Qualcomm Incorporated | Encoder assisted frame rate up conversion using various motion models |
CA2849697A1 (en) * | 2006-10-16 | 2008-04-24 | Alexandros Eleftheriadis | Systems and methods for signaling and performing temporal level switching in scalable video coding |
KR101455161B1 (ko) | 2007-01-08 | 2014-10-28 | 톰슨 라이센싱 | 비디오 스트림 스플라이싱을 위한 방법 및 장치 |
CN101690229A (zh) * | 2007-06-26 | 2010-03-31 | 诺基亚公司 | 用于指示时间层切换点的系统和方法 |
JP2009027598A (ja) * | 2007-07-23 | 2009-02-05 | Hitachi Ltd | 映像配信サーバおよび映像配信方法 |
US20110110436A1 (en) * | 2008-04-25 | 2011-05-12 | Thomas Schierl | Flexible Sub-Stream Referencing Within a Transport Data Stream |
JP5460006B2 (ja) * | 2008-09-18 | 2014-04-02 | キヤノン株式会社 | 動画検索装置及び動画検索方法 |
CA2758237C (en) * | 2009-04-09 | 2017-08-15 | Telefonaktiebolaget Lm Ericsson (Publ) | Media container file management |
WO2012023281A1 (ja) * | 2010-08-17 | 2012-02-23 | パナソニック株式会社 | 動画像復号方法、動画像符号化方法、動画像復号装置、及び、動画像符号化装置 |
EP2664151A4 (en) | 2011-01-14 | 2016-01-20 | Vidyo Inc | HIGH-COATED SYNTAX FOR TIMELY SCALABILITY |
MX2013008895A (es) * | 2011-02-16 | 2013-08-21 | Panasonic Corp | Metodo de codificacion de video y metodo de decodificacion de video. |
US20120287987A1 (en) * | 2011-05-13 | 2012-11-15 | Madhukar Budagavi | Coding of Scene Changes Using Picture Dropping |
JP2012244411A (ja) * | 2011-05-19 | 2012-12-10 | Sony Corp | 画像データ送信装置、画像データ送信方法および画像データ受信装置 |
US20130170561A1 (en) * | 2011-07-05 | 2013-07-04 | Nokia Corporation | Method and apparatus for video coding and decoding |
US9264717B2 (en) | 2011-10-31 | 2016-02-16 | Qualcomm Incorporated | Random access with advanced decoded picture buffer (DPB) management in video coding |
CN108322744B (zh) * | 2012-01-31 | 2022-08-30 | Vid拓展公司 | 用于可缩放的高效视频译码(hevc)的参考图片集(rps)信令 |
US20140092953A1 (en) * | 2012-10-02 | 2014-04-03 | Sharp Laboratories Of America, Inc. | Method for signaling a step-wise temporal sub-layer access sample |
US20140098868A1 (en) * | 2012-10-04 | 2014-04-10 | Qualcomm Incorporated | File format for video data |
JP2014116046A (ja) | 2012-12-10 | 2014-06-26 | Hoya Corp | 磁気ディスク用ガラス基板の製造方法 |
JP6571314B2 (ja) | 2013-06-18 | 2019-09-04 | パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカPanasonic Intellectual Property Corporation of America | 送信方法 |
JP6538324B2 (ja) | 2013-10-18 | 2019-07-03 | パナソニック株式会社 | 画像符号化方法および画像符号化装置 |
-
2014
- 2014-10-01 US US14/909,592 patent/US10547857B2/en active Active
- 2014-10-01 EP EP14851673.5A patent/EP3057330B1/en active Active
- 2014-10-01 BR BR112016007448-3A patent/BR112016007448B1/pt not_active IP Right Cessation
- 2014-10-01 RU RU2016112623A patent/RU2660957C2/ru active
- 2014-10-01 WO PCT/JP2014/076334 patent/WO2015053157A1/ja active Application Filing
- 2014-10-01 JP JP2015541544A patent/JP6558246B2/ja active Active
- 2014-10-01 EP EP19190982.9A patent/EP3591980A1/en active Pending
-
2019
- 2019-07-17 JP JP2019132221A patent/JP6699790B2/ja active Active
- 2019-12-09 US US16/707,085 patent/US11025930B2/en active Active
-
2020
- 2020-04-07 JP JP2020069188A patent/JP6876241B2/ja active Active
-
2021
- 2021-04-19 US US17/234,226 patent/US11589061B2/en active Active
- 2021-04-20 JP JP2021070958A patent/JP7192910B2/ja active Active
-
2022
- 2022-12-07 JP JP2022195929A patent/JP7396447B2/ja active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2008245325A (ja) * | 1997-10-31 | 2008-10-09 | Matsushita Electric Ind Co Ltd | 動画像符号化方法 |
JP2004180290A (ja) * | 2002-11-15 | 2004-06-24 | Sony Corp | 伝送装置と伝送方法と再生装置と再生方法およびプログラムと記録媒体 |
JP2012124896A (ja) * | 2010-12-07 | 2012-06-28 | Mediatek Inc | 低遅延階層のbグループを有するピクチャ構造を用いた動画像符号化の方法及び装置 |
Non-Patent Citations (1)
Title |
---|
GARY J. SULLIVAN; JENS-RAINER OHM, WOO-JIN HAN; THOMAS WIEGAND: "Overview of the High Efficiency Video Coding (HEVC) Standard", IEEE TRANSACTIONS ON CIRCUITS ANDSYSTEMS FOR VIDEO TECNOROGY, vol. 22, no. 12, December 2012 (2012-12-01), pages 1649 - 1668 |
Cited By (21)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2020005275A (ja) * | 2013-10-18 | 2020-01-09 | パナソニック株式会社 | 画像符号化方法、画像復号方法、画像符号化装置及び画像復号装置 |
JP5909026B2 (ja) * | 2013-10-18 | 2016-04-26 | パナソニック株式会社 | 画像符号化方法及び画像符号化装置 |
US12063379B2 (en) | 2013-10-18 | 2024-08-13 | Sun Patent Trust | Image coding method, image decoding method, image coding apparatus, receiving apparatus, and transmitting apparatus |
JPWO2015056409A1 (ja) * | 2013-10-18 | 2017-03-09 | パナソニック株式会社 | 画像符号化方法及び画像符号化装置 |
US11785231B2 (en) | 2013-10-18 | 2023-10-10 | Sun Patent Trust | Image coding method, image decoding method, image coding apparatus, receiving apparatus, and transmitting apparatus |
JP2016054548A (ja) * | 2013-10-18 | 2016-04-14 | パナソニック株式会社 | 画像復号方法及び画像復号装置 |
JP2018186560A (ja) * | 2013-10-18 | 2018-11-22 | パナソニック株式会社 | 画像符号化方法、画像復号方法、画像符号化装置及び画像復号装置 |
US11778208B2 (en) | 2013-10-18 | 2023-10-03 | Sun Patent Trust | Image coding method, image decoding method, image coding apparatus, receiving apparatus, and transmitting apparatus |
US11647211B2 (en) | 2013-10-18 | 2023-05-09 | Sun Patent Trust | Image coding method, image decoding method, image coding apparatus, receiving apparatus, and transmitting apparatus |
JPWO2016190089A1 (ja) * | 2015-05-22 | 2018-03-08 | ソニー株式会社 | 送信装置、送信方法、画像処理装置、画像処理方法、受信装置および受信方法 |
CN107534786B (zh) * | 2015-05-22 | 2020-10-27 | 索尼公司 | 传输装置、传输方法、图像处理装置、图像处理方法、接收装置、及接收方法 |
US10904592B2 (en) | 2015-05-22 | 2021-01-26 | Sony Corporation | Transmission apparatus, transmission method, image processing apparatus, image processing method, reception apparatus, and reception method |
CN107534786A (zh) * | 2015-05-22 | 2018-01-02 | 索尼公司 | 传输装置、传输方法、图像处理装置、图像处理方法、接收装置、及接收方法 |
WO2016190089A1 (ja) * | 2015-05-22 | 2016-12-01 | ソニー株式会社 | 送信装置、送信方法、画像処理装置、画像処理方法、受信装置および受信方法 |
JP6996490B2 (ja) | 2016-03-24 | 2022-01-17 | ソニーグループ株式会社 | 送信装置、送信方法、受信装置および受信方法 |
JPWO2017164087A1 (ja) * | 2016-03-24 | 2019-02-14 | ソニー株式会社 | 送信装置、送信方法、受信装置および受信方法 |
JP2022009528A (ja) * | 2016-08-22 | 2022-01-14 | ソニーグループ株式会社 | 送信方法 |
JP7184140B2 (ja) | 2016-08-22 | 2022-12-06 | ソニーグループ株式会社 | 送信方法 |
JP7548285B2 (ja) | 2016-08-22 | 2024-09-10 | ソニーグループ株式会社 | 送信方法および受信装置 |
WO2019069987A1 (ja) * | 2017-10-06 | 2019-04-11 | パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ | 符号化装置、復号装置、符号化方法および復号方法 |
JP2023526955A (ja) * | 2020-05-22 | 2023-06-26 | ジーイー ビデオ コンプレッション エルエルシー | ビデオエンコーダ、ビデオデコーダ、符号化及び復号のための方法、並びに高度なビデオ符号化概念を実現するためのビデオデータストリーム |
Also Published As
Publication number | Publication date |
---|---|
JP7396447B2 (ja) | 2023-12-12 |
US20210258589A1 (en) | 2021-08-19 |
JP7192910B2 (ja) | 2022-12-20 |
RU2016112623A (ru) | 2017-10-09 |
JP2019176528A (ja) | 2019-10-10 |
US11025930B2 (en) | 2021-06-01 |
US10547857B2 (en) | 2020-01-28 |
US11589061B2 (en) | 2023-02-21 |
BR112016007448A2 (pt) | 2017-08-01 |
US20200120348A1 (en) | 2020-04-16 |
JPWO2015053157A1 (ja) | 2017-03-09 |
JP6558246B2 (ja) | 2019-08-14 |
BR112016007448B1 (pt) | 2023-04-11 |
EP3591980A1 (en) | 2020-01-08 |
RU2660957C2 (ru) | 2018-07-11 |
JP2023016995A (ja) | 2023-02-02 |
JP2021114785A (ja) | 2021-08-05 |
JP6876241B2 (ja) | 2021-05-26 |
EP3057330A1 (en) | 2016-08-17 |
EP3057330B1 (en) | 2020-04-01 |
JP6699790B2 (ja) | 2020-05-27 |
US20160212434A1 (en) | 2016-07-21 |
JP2020115673A (ja) | 2020-07-30 |
EP3057330A4 (en) | 2017-08-09 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP6876241B2 (ja) | 送受信システムおよびその処理方法 | |
JP5789004B2 (ja) | 送信装置、送信方法、受信装置、受信方法、符号化装置および符号化方法 | |
JP5947269B2 (ja) | 符号化装置、符号化方法、送信装置および受信装置 | |
JP6795076B2 (ja) | 送受信システムおよび送受信方法 | |
JP7338745B2 (ja) | 受信装置 | |
JP5954508B2 (ja) | 符号化装置、符号化方法、送信装置および受信装置 | |
JP6614275B2 (ja) | 受信装置、受信方法、送信装置および送信方法 | |
JP6341228B2 (ja) | 符号化装置、符号化方法、送信装置、送信方法、受信装置および受信方法 | |
JP5954509B2 (ja) | 符号化装置、符号化方法、送信装置および受信装置 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 14851673 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2015541544 Country of ref document: JP Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 14909592 Country of ref document: US |
|
ENP | Entry into the national phase |
Ref document number: 2016112623 Country of ref document: RU Kind code of ref document: A |
|
REEP | Request for entry into the european phase |
Ref document number: 2014851673 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2014851673 Country of ref document: EP |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
REG | Reference to national code |
Ref country code: BR Ref legal event code: B01A Ref document number: 112016007448 Country of ref document: BR |
|
ENP | Entry into the national phase |
Ref document number: 112016007448 Country of ref document: BR Kind code of ref document: A2 Effective date: 20160404 |