JP2018129782A - Video recording/reproducing apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize a video recording system capable of recording a program of MMT system by a simple system.SOLUTION: A video recording/reproducing apparatus (100) includes a tuner/demodulation section (11), a multiplexing/demultiplexing section (21), decode sections (31, 32, 33, 34) and a recording/reproducing control section (41). The tuner/demodulation section (11)receives broadcast waves (Ba). The multiplexing/demultiplexing section (21) demultiplexes a multiplexed stream (Sm) of the broadcast waves (Ba). The decode sections (31, 32, 33, 34) decompress the compressed data of the demultiplexed multiplexed stream (Sm). The recording/reproducing control section (41)collects video data or control information for recording from the data decompressed in the decode sections (31, 32, 33, 34). When recording the video data contained in the broadcast waves (Ba) using MMT system, the video recording/reproducing apparatus (100) records as a video data coupling the packet data.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a video recording / reproducing apparatus.

  In recent years, with the progress of video compression technology or broadcast wave modulation technology, introduction of a new television broadcasting system is being studied in each country. The new scheme improves transmission efficiency. For this reason, it becomes possible to broadcast higher quality video using the same radio wave band as before. It is also possible to broadcast video of the same quality as before in a narrower radio band. Along with the introduction of a new method, cooperation with the network is being studied. In addition, an application or content that is stored in a receiver and operates is also being studied.

  The system for recording TV broadcasts is closely related to the normal TV broadcast system. When a new television broadcasting system is adopted, a new television broadcasting system program cannot be recorded as it is. Therefore, when the television broadcasting system is changed, it is necessary to change the recording system to a system corresponding to the new broadcasting system.

  In particular, when a replaceable recording medium is used, it is necessary to consider compatibility with other playback devices. The replaceable recording medium is, for example, a DVD (registered trademark: Digital Versatile Disc) or a Blu-ray disc (registered trademark: BLU-RAY DISC).

  It is difficult to record a new television broadcasting system as it is with the currently used recording system. For this reason, a recording system corresponding to a new television broadcasting system is required.

  For example, Patent Document 1 describes a television broadcast recording method using the BDAV method, a method for dividing and recording large-volume recording data, and a method for accessing large-volume recording data. In addition, a method is described in which compressed video data and audio data are stored and recorded in MPEG2TS. In addition, a method is described in which a television broadcast transmitted in MPEG2TS is received and recorded in the MPEG2TS format.

WO 2006/030767 (0021 to 0105, FIGS. 1 to 21)

  However, the recording system described in Patent Document 1 supports only the MPEG2TS format container format and cannot record the MMT format broadcast that is planned to be adopted in the new television broadcasting format.

  In view of these problems, the present invention provides a method for recording a broadcast in MMT format.

  A video recording / reproducing apparatus according to the present invention includes a tuner / demodulator that receives broadcast waves, a demultiplexer that demultiplexes the multiplexed stream of broadcast waves, and the multiplexed stream is demultiplexed. A decoding unit that decompresses compressed data; and a recording / playback control unit that collects video data or control information for recording from the data decompressed by the decoding unit, and is included in the broadcast wave using the MMT format When the video data is recorded, it is recorded as video data obtained by combining packet data.

  The video recording method according to the present invention can record an MMT program by a simple method.

1 is a configuration diagram of a video recording / reproducing apparatus 100 according to Embodiment 1. FIG. It is a figure explaining the demultiplexing procedure. It is the figure which showed the logical structure of data. It is a schematic diagram at the time of combining and recording a TLV packet. It is a conceptual diagram explaining packet selection and time synchronization. It is a figure explaining the synchronous system using the MPU time stamp descriptor in a MMTP system. It is a flowchart which shows the procedure which produces a time table. It is a figure which shows the time table which can be searched by the time of a frame unit. It is a figure explaining the synchronous system using the MPU time stamp descriptor in a MMTP system. It is explanatory drawing at the time of recording stream data as BMFF. 10 is an explanatory diagram of demultiplexing by MPT in the MMTP scheme according to Embodiment 2. FIG. It is explanatory drawing of the relationship between a time table and video data. It is an example of the MPT data structure in a MMTP system. It is explanatory drawing of the alignment at the time of thinking centering on an image | video. 10 is a schematic diagram of a video stream according to Embodiment 3. FIG. It is the figure which showed the division | segmentation of an image. It is the figure which showed the division | segmentation of an image. It is a figure of the time table corresponding to a division | segmentation slice segment. It is a figure which shows a response | compatibility with each item of a time table, and data. It is explanatory drawing of fast-forward reproduction | regeneration. It is explanatory drawing of the fast-forward reproduction | regeneration which concerns on the modification 1. FIG. It is explanatory drawing of the order of the update of an image. It is explanatory drawing of the order of the update of an image. It is a figure of the time table concerning the modification 2. It is explanatory drawing of fast-forward reproduction | regeneration.

  As one of the ways to enjoy television broadcasting, it has been conventionally performed to record (record) a television broadcast program and to reproduce and view it later. There are two main ways to enjoy recording. One is called “time shift”. In the time shift, when the program cannot be viewed at the time when broadcasting is performed, the program is recorded and the program is reproduced later. Others are called “archives”. An archive is a program that is recorded and stored. Therefore, the recorded program can be viewed at any time.

  In the case of using time shift, after recording a program, the use is often completed in a short period of time. On the other hand, when an archive is used, it is often used after recording a program for a long period of time.

  In addition to television broadcasts, video taken by users themselves is also recorded. Video data shot by a home video camera or the like is recorded and stored on an optical disk or a hard disk.

  Recently, video collected on a network or the like may be recorded. In addition to recording at home, recorded images are also used in art museums, museums, and digital signage. “Digital signage” is an advertising medium that uses a digital technology for display and communication to display video or information on a flat display or projector.

  Commercial content such as movies includes a read-only DVD or Blu-ray disc on which the content is recorded. Recently, in addition to optical discs with packages, content services that are downloaded via a network are increasing.

  The format for recording broadcast content may differ from the recording format for commercially available content. For example, in the case of a Blu-ray disc, two types of formats are defined. First, BDAV is a format for recording broadcast content. BDAV can record broadcast waves as they are. Second, BDMV is a commercially available content format. BDMV has an advanced playback control function. AVCHD is used when recording video on an HDD or a memory card used in a camera or the like. AVCHD has been modified based on BDMV.

  In television broadcasting, the introduction of an ultra high-definition system that supports 4K video with a resolution four times that of the conventional system is planned. In addition, higher quality 8K video broadcasting is also being studied. Similarly, introduction of a new television broadcasting system is being studied in the North American region or the European region in a manner that expands the current broadcasting method.

  Although there are differences depending on regions, the conventional television broadcasting system adopts MPEG2 or AVC (h.264) as a video compression system. Further, the conventional television broadcasting system employs MPEG2TS as a multiplexing system. MPEG2TS is premised on a single transmission path of broadcasting, and is configured such that a broadcast station sends video or audio together. MPEG2TS employs a fixed-length packet system with a synchronization mark as a transmission unit. For this reason, a system for recording MPEG2TS is often adopted as a recording system.

  On the other hand, in the new television broadcasting system, there are many standards that incorporate MPEG2, AVC, or HEVC (h.265) as a video compression system. In HEVC (High Efficiency Video Coding), when compared with the same image quality, the compression efficiency is four times that of MPEG2 and twice that of AVC. HEVC is a compression technique necessary for a new television broadcasting system aiming at high image quality or narrow band.

  Also, in the new television broadcasting system, importance is attached to consistency with network technology. Therefore, in a new television broadcasting system, adoption of MMT (MPEG Media Transport) is considered as a multiplexing system. MMT is a method that can provide information through a plurality of transmission paths, and separately transmits video or audio, and the receiver can select and receive them. The MMT employs variable length packets. In addition, a method of changing a video stream to be reproduced according to the reception status is also being studied.

  As described above, it is difficult to record a new television broadcasting system as it is in the currently employed recording system. For this reason, a recording system corresponding to a new television broadcasting system is required.

  Here, when creating a new recording method, several approaches can be considered.

  First, a new broadcasting system is completely converted into a conventional video and recorded. In this case, it is not necessary to change the recording method, and compatibility with a conventional reproducing apparatus can be obtained. However, it cannot take advantage of new methods such as high image quality.

  Second, the conventional recording system is followed as much as possible, and video and audio, which are the center of information of television broadcasting, are converted into a new system and recorded. In this case, the merit of the new method can be received for video and audio. However, subtitles or data broadcasting cannot be enjoyed.

  Thirdly, a new broadcast system program data is recorded as it is. Except for externally dependent parts such as network services, many of the benefits of the new method can be used.

Embodiment 1 FIG.
FIG. 1 is a configuration diagram of a video recording / reproducing apparatus 100 according to the first embodiment.

  The recording / playback apparatus 100 includes a tuner / demodulation unit 11, a demultiplexing unit 21, and a recording / playback control unit 41. In the following embodiments, a recording / playback apparatus will be described. However, a recording apparatus that does not include a playback portion can be used.

  The recording / playback apparatus 100 includes a video decoding unit 32, an audio decoding unit 31, a caption decoding / rendering unit 33, or a data broadcast / EPG processing unit 34. The decoding unit includes a video decoding unit 32, an audio decoding unit 31, a caption decoding / rendering unit 33, or a data broadcasting / EPG processing unit 34.

  Further, the recording / reproducing apparatus 100 can include a built-in recording apparatus 51 or an optical disc drive 52.

  In addition, the recording / playback apparatus 100 can include an external input unit 12 or a network unit 13. The external input unit 12 and the network unit 13 have a function of receiving an external signal input. The tuner / demodulator 11 also has a function of receiving an external signal input.

<Configuration of recording / playback apparatus 100>
The tuner / demodulator 11 receives the broadcast wave Ba. Then, the tuner / demodulator 11 demodulates the received broadcast wave Ba. The external input unit 12 receives data from an external device Ei. The external device is a recording device such as a video camera, for example. The “data” includes video data, audio data, caption data, data broadcast data, control information, and the like.

  The network unit 13 receives data from the network Ne. The network Ne is a computer network or a communication network that can exchange signals, data, or information by connecting a plurality of computers or electronic devices, for example.

  The demultiplexing unit 21 demultiplexes the multiplexed stream Sm. That is, the demultiplexing unit 21 separately extracts various data from the multiplexed stream Sm.

  The audio decoding unit 31 expands the compressed audio data. The audio decoding unit 31 expands the compressed audio data included in the elementary stream Se.

  The video decoding unit 32 expands the compressed video data. The video decoding unit 32 expands the compressed video data included in the elementary stream Se.

  The caption decoding / rendering unit 33 decompresses the compressed caption data. The caption decoding / rendering unit 33 decompresses the compressed caption data included in the elementary stream Se.

  The data broadcast / EPG processing unit 34 decompresses the compressed data broadcast data. The data broadcast / EPG processing unit 34 decompresses the compressed data broadcast data included in the elementary stream Se.

  The recording / playback control unit 41 collects video data, audio data, or control information for recording. The recording / playback control unit 41 converts the collected data into a data format for recording.

  The built-in recording device 51 is a recording device provided in the recording / playback device 100. The built-in recording device 51 is, for example, a hard disk drive, a volatile memory, or a nonvolatile memory.

  The optical disc drive 52 records data on the optical disc 53. The optical disk drive 52 reads data from the optical disk 53. The optical disc 53 is, for example, a Blu-ray disc or a DVD.

<Data processing flow when no data is recorded>
First, the flow when displaying a broadcast program on a television without recording it will be described. That is, a broadcast wave is taken as an example to describe the process from receiving a broadcast wave to outputting video and audio. In FIG. 1, the television is the display device Dd and the acoustic device Es.

  As an input signal from the outside, there are a broadcast wave Ba received by an antenna, a video signal from a video camera (external device Ei), a video signal from a video playback device, or video data from a network Ne. A user selects an input unit using a remote controller (remote controller) or operation buttons. The input unit is a tuner / demodulator 11, an external input unit 12, a network unit 13, or the like. A remote control is a remote control device operated by a user.

  In the case of the broadcast wave Ba, a broadcast station or a program is set. In the case of the network Ne, data acquisition destination or data access information is set. It is assumed that the video service that the user wants to watch or the video service that the user wants to record is specified. In addition, it is assumed that the video service that the user wants to view or the video service that the user wants to record is ready to be received.

  A broadcast wave Ba received by an antenna or the like is input to the tuner / demodulator 11. The tuner / demodulator 11 extracts the radio wave of the designated broadcast station from the broadcast wave Ba. Then, the tuner / demodulator 11 demodulates the broadcast wave Ba using a specified demodulation method. The tuner / demodulator 11 extracts digital data from the broadcast wave Ba.

  Here, the extracted digital data is a multiplexed stream Sm obtained by multiplexing video data, audio data, caption data, control information, or the like. In addition, a plurality of programs may be stored together in one multiplexed stream Sm.

  The “stream” represents data having a time flow or data transmitted in a form having a time flow. For example, in the case of video data, the stream is a video stream, and in the case of audio data, the stream is an audio stream. Other examples include a caption stream, a multiplexed stream, a received stream, a data stream, and the like.

  The multiplexed stream Sm is taken out by the tuner / demodulator 11. The multiplexed stream Sm extracted by the tuner / demodulator 11 is sent to the demultiplexer 21. The demultiplexer 21 separately extracts various data or various control information from the multiplexed stream Sm. The data is, for example, data that directly configures a program. The data is, for example, a video stream, an audio stream, or a caption stream. The data is, for example, a data broadcast program or data stored in the multiplexed stream Sm.

  Various types of data (elementary stream Se) that have been demultiplexed are compressed data.

  The video stream is compressed video data. The video decoding unit 32 expands the compressed video data. The expanded video data is output as video from the display device Dd or the like. The display device Dd is, for example, a television.

  Similarly, the audio stream is compressed audio data. The audio decoding unit 31 expands the compressed audio data. The expanded audio data is output as audio from the acoustic device Es. The acoustic device Es is, for example, a television.

When output from the display device Dd or acoustic device Es, and the video data Di ₁ and audio data Ds, the buffering and synchronization is performed (not shown). As a result, there is no deviation between video and audio. The output timing of the video data and the audio data is a timing designated or calculated from control information or a system clock.

  The subtitle stream is compressed subtitle data. The caption decoding / rendering unit 33 decompresses the compressed caption data. The caption decoding / rendering unit 33 interprets the decompressed caption data. The caption decoding / rendering unit 33 visualizes the interpreted caption data. The visualized caption data is synthesized with video data or audio data at a designated timing. The combined caption data is output as a video from the display device Dd or the like.

  A data broadcast stream is compressed data. The data broadcast / EPG processing unit 34 decompresses the compressed data broadcast data. The data broadcast / EPG processing unit 34 synthesizes the expanded video data and the expanded audio data at a designated timing. The synthesized video data and audio data are output as video and audio from a television or the like.

  The recording / reproducing apparatus 100 outputs the broadcast wave Ba thus received as synchronized video and audio.

<Recording data>
Next, consider recording this video data.

  The recording / playback control unit 41 collects recording video data, audio data, control information, or the like in the flow from the reception of the broadcast wave Ba to the display of the video. The collected data is converted into a data format for recording. The data converted into the recording data format is recorded on the optical disk 53 or the like through the built-in recording device 51 or the optical disk drive 52.

There are various possible combinations of positions at which the recording / playback control unit 41 extracts video data or audio data. However, for ease of explanation, the following three paths (A) to (C) will be simplified for explanation. The three paths are (A) extraction in the state of the multiplexed stream Sm, (B) extraction in the state of the elementary stream Se that has been demultiplexed, and (C) a state in which the video and audio are decompressed (video data Di ₁ ).

  In the following, (A) and (B) will be described.

<< (A-1) Data recording in multiplexed stream Sm state (1) >>
Consider data extraction in a multiplexed stream state and recording on a Blu-ray disc (optical disc 53). The data is retrieved using the route (A) in FIG.

  For example, in the case of a conventional Japanese broadcasting system, the multiplexed stream Sm extracted from the tuner / demodulator 11 is data in a form in which a video signal compressed in the MPEG2 format is stored in the MPEG2TS multiplexed stream. It is.

  The MPEG2TS broadcast stream employs fixed-length packets of 192 byte size. Further, a data file for recording format can be created by combining the packets in the order of reception. The data file created in this way is called a stream file.

  A plurality of programs are multiplexed in a stream that is actually broadcast. The process of taking out the target program from among them is necessary. However, the description thereof is omitted here.

  First, the case of the BDAV format of the Blu-ray disc will be described.

  In the case of the BDAV format of the Blu-ray disc, a clip file, a playlist file, an info file, or the like is required in addition to the stream file. The clip file is a file in which detailed information for accessing data in the stream file is recorded. The playlist file manages time-series information. The time series information is, for example, information such as the start point, end point, or playback stream switching of one program. The info file manages information on the entire disc such as a reproducible program list.

  These pieces of information are created from management information in the stream, reservation information set by the user, or management information obtained from the tuner / demodulator 11. These are described in detail in Patent Document 1, for example.

  Next, the case of a new broadcasting system in Japan will be described.

  This is, for example, “Advanced High-Bandwidth Satellite Digital Broadcasting Operation Manual Version 1.1 NEXTVF TR-004” (published March 30, 2016, Part 1, Part 2, Chapter 5) It is described in pages 2-31 to 2-40 and FIG.

  This method is compatible with 4K and 8K high resolution video. In addition, this method corresponds to expansion of the color gamut and luminance range. This method employs HEVC as a video compression method. This method employs a combination of MMT and TLV as a multiplexing method.

  MMT and TLV are designed in consideration of consistency with IP packets used in the network. TLV is a transmission method using a broadcast wave of an IP packet. MMT defines a method and data format for transferring video data using IP packets. The IP packet employs a variable length packet. For this reason, MMT and TLV also employ variable length packets.

  The demultiplexing procedure in this method will be described with reference to FIG. FIG. 2 is a diagram for explaining the demultiplexing procedure.

In FIG. 2, the horizontal direction indicates the order of data reception. That is, data is arranged in the order received in the horizontal axis Ha direction. It is written from left to right in the order received. The vertical axis Va direction indicates the flow of data processing. That is, in the direction of the vertical axis Va, the data extraction order by packet analysis is shown. In the range of the vertical axis Va _1, which releases the multiplexing of the multiplexed stream Sm. In the range of the vertical axis Va _2, which releases the multiplexing of the elementary stream Se.

  The data of the TLV packet is obtained by demodulating the broadcast wave Ba. The TLV packet includes information related to broadcasting. Information related to broadcasting is, for example, broadcasting identifiers, channels, broadcasting station names, IP addresses, port numbers, or information on radio waves to be used. “Radio waves to be used” are terrestrial waves, BS broadcasts, CS broadcasts, and the like. “Information on radio waves to be used” refers to the form of broadcasting, modulation method, frequency, deflection direction or turning direction.

  The payload refers to the data part in the data transmission. That is, the payload corresponds to the entire transmitted data excluding management information for transmission processing. The management information is, for example, a header or metadata.

  TLV (Type-Length-Value) is a format that collectively represents the type, length, and value of information. UDP (User Datagram Protocol) is a protocol that operates in the transport layer of the upper protocol of IP. UDP operates by bridging the network layer IP and the protocol above the session layer. MMTP (registered trademark) is a multimedia multiplexing transmission protocol.

  The tuner / demodulator 11 extracts broadcast station information, IP address information or port number information from the TLV packet. The tuner / demodulator 11 extracts a necessary IP packet by using the IP address information. Next, the tuner / demodulator 11 extracts the UDP packet using the port number information.

  At this point, the tuner / demodulator 11 demultiplexes the broadcast wave Ba sent from the broadcast station. Then, the tuner / demodulator 11 extracts the data of the UDP packet. The tuner / demodulator 11 removes the UDP header from the UDP packet. Then, the tuner / demodulator 11 extracts the UDP payload from the UDP packet. Thereby, the tuner / demodulator 11 can extract the MMTP packet from the UDP packet.

  In the new broadcasting standard in Japan, one MMTP packet is stored in one UDP / IP packet. Furthermore, one UDP / IP packet is stored in one TLV packet. Therefore, in a packet transmitting MMTP after the control data is separated, the MMTP packet is extracted by simply removing the TLV header, IP header, and UDP header from the TLV packet.

  When the MMTP packet is extracted, the multiplexing of the broadcast wave Ba sent from the broadcast station is released. However, the broadcast wave Ba may be multiplexed as a series of MMTP packets in which a plurality of programs are collected. In that case, in order to extract only the target program, first, control information is extracted, and an MMTP packet is selected and extracted according to the description of the control information.

  One of the control signals sent as an MMTP packet is a PLT (Package List Table). When all the information is transmitted by broadcast waves, “packet_id” specified by “MMT_general_location_info” in the PLT is referred to. This “packet_id” is used to filter the MMTP packet. As a result, it is possible to select an MMTP packet including management information of a target program. The “target program” is a program scheduled to be viewed.

  When acquiring program management information from the network, an IP address and a port number described in “MMT_general_location_info” are specified. Alternatively, “MMT_general_location_info” designates a program acquisition destination using a URL.

  Next, data including MPT (MMT_Package_Table) is extracted from the selected MMTP packet. In the MPT, a combination of assets such as video, audio, or subtitles constituting a target program and an acquisition destination are described. The acquisition destination of each asset is indicated by “packet_id” or network information by “MMT_general_location_info”. Here, the network information is an IP address, a port number, or a URL.

  In this way, the control data indicated by PLT and the assets constituting the program indicated by MPT are filtered by, for example, “packet_id”. As a result, the MMTP packet of the target program can be extracted.

  In the process of extracting the MMTP packet or MMTP payload from the TLV packet, attention is paid to one packet. The packet header is removed. However, the data in the packet does not change. However, in practice, packet selection and classification are performed from the control information and the contents of the packet header at each stage.

[When recording without using standard format]
As one method of recording a program, for example, a method of recording a received broadcast packet as it is will be described.

  FIG. 4 is a schematic diagram when TLV packets are combined and recorded.

  In the example shown in FIG. 4, the multiplexing stream Sm sent from the broadcasting station is demultiplexed at the time of the TLV packet. The TLV packet of the program is combined as it is. The combined TLV packet constitutes a file.

  In FIG. 4, TLV packets TP-0, TP-1, TP-2, and TP-3 are shown. “−0” or the like represents the order of reception of the TLV packets TP. For example, the TLV packet TP-0 is the first received TLV packet TP. The horizontal axis Ha represents the order of reception.

  The TLV packets TP-1, TP-2, and TP-3 received after the TLV packet TP-0 are combined after the TLV packet TP-0 received first. Then, the combined TLV packets TP-0, TP-1, TP-2, and TP-3 are recorded. The TLV packet TP employs a variable length packet.

  Since data is received in packet units, the beginning of the packet is clear at the time of reception. However, when one file is created by combining with other packets, it is necessary to determine the head position of each packet.

  The first method is a method of creating a list of packet head positions as a management file Af at the time of recording.

  This is because the start position of the packet in the file is known at the time of recording the packet. In this case, a list of the head position of the packet may be created in association with the packet number or the like.

  In the second method, data is read from the head of the file, and if there is head data Hm, it is determined that the head of the TLV packet TP is present. Then, the data combined with the TLV packet TP is read.

  In the TLV packet TP, a fixed value “0x7F” is stored in the first 1 byte in order to identify the head of the packet. Then, 1-byte data representing the packet type is stored after the first byte. Then, 2 bytes of data indicating the data length are stored behind it.

  Here, the top data Hm of the TLV packet TP is a fixed value “0x7F”.

  If a fixed value “0x7F” (first data Hm) is read, the data read immediately before is interpreted as a TLV packet TP and processed.

  The fixed value “0x7F” is not a special value. The fixed value “0x7F” is also present in the data. Therefore, there is a possibility of reading from the wrong position. Even in this case, the presence or absence of contradiction as data of the TLV packet TP is confirmed. Alternatively, after reading data of the data length, it is confirmed whether or not the beginning of the next data is a fixed value “0x7F” every time. As a result, packets can be read at the correct data delimiters.

  In this example, a method of recording a TLV packet is shown. However, other methods for recording UDP / IP packets can also be employed. Also, a method of recording the MMTP packet as it is can be adopted.

  However, the reason why the TLV packet is selected is that a fixed byte for identification is prepared at the head of the TLV packet, and the packet identification is relatively easy.

  In other methods of recording packets, there is a case where a mark for identification is not inserted. Therefore, it is possible to adopt a method of inserting an identifier uniquely. In the case of a UDP / IP packet, for example, a method of using an IP address information as a mark for identification can be employed. The IP address information does not change during the same program.

<< (A-2) Recording of data in multiplexed stream Sm state (2) >>
The MPT is stored in an MMTP packet and transmitted. The MPT includes control information. The demultiplexing unit 21 distributes the packets based on the information described in the MPT. The information described in the MPT is, for example, packet ID information included in the MMTP header. As a result, the demultiplexing unit 21 can individually extract video information or audio information. Then, the demultiplexing unit 21 can demultiplex the multiplexed stream Sm.

  The MFU is stored in the MMTP packet. A plurality of MFUs may be stored in one MMTP packet. One MFU may be stored in one MMTP packet. One MFU may be stored in a plurality of MMTP packets.

  The MFU of video data or the MFU of audio data included in the MMTP is a processing unit called an access unit (hereinafter referred to as AU) or NAL unit. A NAL (Network Abstraction Layer) unit is data obtained by further dividing AU.

  When video data is directly stored in the MFU, a case where an AU is stored and a case where a NAL unit is stored are defined. However, a new broadcasting system in Japan adopts a system that stores data in a NAL unit. Therefore, in the following description, it will be described as being stored as a NAL unit.

  FIG. 2 shows a case where the NAL unit is directly stored in the MFU. The NAL unit includes a VCL-NAL unit that includes only video data and a non-VCL-NAL unit that does not include video data and stores management information (VCL: Video Coding Layer).

  The non-VCL-NAL unit can obtain control information by removing the NAL header.

  When the VCL-NAL unit removes the NAL header, the divided video data is taken out. By combining these divided video data, compressed video data for one frame is obtained.

  Usually, a NAL unit handles a plurality of NAL units including management information for one frame and video data as a set, and is called AU. In the case of video data, the demultiplexer 21 includes video data. Then, the demultiplexing unit 21 can reconstruct an ES (elementary stream).

  The AU of video data is basically a video unit for one frame. The video unit for one frame is a unit for representing a picture. In the video apparatus, this picture is displayed as a moving image by sequentially switching the pictures in time series. However, there are several types of dependency relationships with the previous and next frames.

  One is called an I picture. An I picture can reproduce one picture with this data alone.

  Others are called P pictures or B pictures. These data depend on other pictures. Therefore, a P picture and a B picture cannot reproduce a single picture by itself. P pictures and B pictures can be reproduced by referring to other pictures. A P picture refers to one other picture. The B picture refers to two other pictures.

  A television broadcast or a video recording / playback apparatus is required to be able to view from the middle of a program. In addition, a television broadcast or a video recording / playback apparatus is required to be able to access randomly. For this reason, it is inconvenient if the picture to be referenced is a wide range.

  Therefore, a certain amount of time or the number of frames is treated as a set. In this set of data, it is determined that a picture to be referred to is completed.

  This set of data is called GOP (Group Of Pictures). The GOP includes at least one I picture. When viewing from the middle of a program, even if the acquired stream data starts from an incomplete position where the video cannot be reproduced, the video can be reproduced and displayed from the beginning of the next GOP.

  For example, a new broadcasting system in Japan requires a GOP to be created with a target of 0.5 seconds for 2K broadcasts and 1 second for 4K broadcasts. As a result, even when the TV is turned on or when the channel is switched, an image can be displayed in 1 to 2 seconds.

  When the video is switched over the entire screen, it is more efficient to divide the GOP before and after the switching. For example, the efficiency of compression ratio or image reproduction is improved. In the case where the entire screen is switched, for example, the scene is switched. For this reason, the GOP length is not a fixed value but is flexibly operated. That is, the GOP length is changed according to the situation.

  In communication or net distribution, a unit of GOP that is longer than a new broadcasting system in Japan may be used.

  When reproducing the recorded video, the video is accessed from the head of the GOP including the frame to be displayed. This enables smooth random access. In addition, when fast-forwarding or the like, only an I picture can be displayed for each GOP. A GOP is a unit of video data that can be reproduced as a moving image. GOP is a unit indicating a position where reproduction is possible. The GOP is a unit indicating a position where random access is possible.

  When the broadcast wave Ba is received, a PLT (Package List Table) is received. PLT is one of control signals sent as MMTP packets.

  When all the information is transmitted by the broadcast wave Ba, “packet_id” specified by “MMT_general_location_info” in the PLT is referred to. Then, the MMTP packet is filtered by this “packet_id”. As a result, it is possible to select an MMTP packet including management information of a target program.

  When program management information is acquired from the network Ne, the program acquisition destination is specified by the IP address and port number described in “MMT_general_location_info”. Alternatively, the program acquisition destination is specified by the URL described in “MMT_general_location_info”.

  Next, data including MPT (MMT_Package_Table) is selected from the selected MMTP packets. The MPT describes a combination of assets such as video, audio, or subtitles constituting a program and an acquisition source. The acquisition source of each asset is indicated by “packet_id” or network information described in “MMT_general_location_info”. The network information is an IP address and a port number. Alternatively, the network information is a URL.

  In addition, regarding assets that need to be synchronized in time such as video, audio, or subtitles, an MPU time stamp descriptor and an MPU extended time stamp descriptor are defined for each asset.

  As acquisition destinations of assets and the like, a case where they are included in the broadcast wave Ba and a case where they are acquired from the network Ne are determined. However, for simplification of description, a case where the broadcast wave Ba is included and transmitted will be described below as an example.

  In the new Japanese broadcasting system, video data and audio data are transmitted by storing the NAL unit directly in the MFU. Also, in the new Japanese broadcasting system, the MMTP packet with “RAP_flag” added to the immediately preceding MMTP packet with “RAP_flag” added is handled as a collection of data. The one data collection is treated as an MPU. “RAP_flag” indicates a starting point of randomly accessible data.

  These can be identified as belonging to the same MPU by examining the presence or absence of “RAP_flag” in the MMTP packet. Alternatively, these can be identified as belonging to the same MPU by “MPU_sequence_number” of the MMTP packet. Here, the MPU in this usage is temporarily referred to as a “stream transmission unit MPU”.

  This “stream transmission unit MPU” starts with randomly accessible data. For this reason, in terms of video, it is in GOP units. That is, the “stream transmission unit MPU” can be considered as a collection of MMTP packets constituting one GOP from the viewpoint of MMTP.

  The MPU time stamp descriptor and the MPU extended time stamp descriptor described above are associated with the “stream transmission unit MPU” and provide information on the time to be synchronized.

  The MPU time stamp descriptor indicates the timing at which the MPU such as video or audio is first reproduced in each MPU in the time of NTP (Network Time Protocol) format. NTP is a protocol for correctly synchronizing the clocks of computers via a network with a system clock built in the computers.

  In the MPU extended time stamp descriptor, the reproduction timing is described as a relative time in the MPU for each AU (frame in the case of video) in each MPU. The relative time in the MPU is a difference from the head in the AU or a difference from the immediately preceding AU. “Difference” is the difference between two values. For example, here, the two values are times.

  With these descriptions, a combination of video, audio, or subtitles can be specified. And it can reproduce | regenerate, taking time synchronization, such as an image | video, an audio | voice, or a subtitle.

  In television broadcasting, since PLT and MPT can display a program in a short time after the television is turned on, they are retransmitted at a relatively short cycle. In the case of a new Japanese broadcasting scheme, PLT and MPT are sent every 100 ms.

  So far, a method for recording a file in which TLV packets, TCP packets, and MMTP packets are combined has been described. Alternatively, a method of recording a file that combines a TLV packet, a UDP packet, and an MMTP packet has been described.

  However, as it is, information on the reproduction time or information indicating a randomly accessible position is distributed and recorded in various places in the file. This is not suitable for access to information. In addition, the same information is recorded many times, which is redundant.

  This information is recorded in a distributed manner because, in broadcasting, it is necessary to arrange and display information in a short time, regardless of the point in time when program reception starts. For this reason, information is distributed and provided in various places in the stream. This is because it is not necessary to consider recording and random access in broadcasting.

  When viewing a recorded program, a search is performed, connection between videos by scene search or editing, and the like are performed. Therefore, random access is required. Therefore, information necessary for random access is created independently at the time of recording information or after recording information.

  Data generation for random access will be described with reference to FIGS. 5, 6, and 7.

  Here, an example will be described in which video data including video or audio is created as a file combined with MMTP packets. However, the same applies when using TLV or UDP / IP packets.

  FIG. 5 is a conceptual diagram illustrating packet selection and time synchronization. What is represented by a square is an MMTP packet.

  In FIG. 5, the horizontal direction indicates the order of data reception. That is, data is arranged in the order received in the horizontal axis Ha direction. It is written from left to right in the order received.

  The packet described as “R” in the video MMTP packet is set to “RAP_flag”. It includes the beginning of the GOP.

  In FIG. 5, the arrows drawn from the MPT represent the relationship in which the playback time of each MPU is determined by the MPU time stamp descriptor of this MPT.

  For example, MPT-0 that appears first designates “packet_ID” as a video asset. In FIG. 5, as an example, MPT-0 designates playback time stamps of MPU-v0 and MPU-v1. The MPU is composed of a plurality of MMTP packets.

  The same MPT-0 designates “packet_ID” as an audio asset. In FIG. 5, as an example, MPT-0 designates playback time stamps of MPU-a0 and MPU-a1. Note that the number of audio MPUs is not limited to one per MMTP packet. Also, the audio MPU does not always appear at the same frequency as the video MPU. However, in the drawing, one voice MPU is used for one MMTP packet. In addition, the appearance frequency of the video MPU and the appearance frequency of the audio MPU are drawn to the same level.

  In FIG. 5, three flows of audio, video, and control information are written. However, actually, the recording / reproducing apparatus 100 receives the data in a mixed state as one data flow.

  A state where such different types of information are mixed as one data is said to be multiplexed. From the multiplexed state, the target data flow (stream) can be extracted and separated using “packet_ID”, various flags, various identifiers, or the like. “Packet_ID”, various flags, various identifiers, and the like are added to each MMTP packet. Extracting the target data from the multiplexed data is called demultiplexing.

  An MMTP packet including MPT is extracted from the multiplexed data of the target program. Then, MPT is extracted from the MMTP packet. The MPT stores a list of various assets constituting the program and a method for acquiring the same.

  For example, the type of video and “packet_ID” storing the data can be known. In addition, the type of voice and “packet_ID” storing the data can be known. With these “packet_ID”, the received MMTP packet is selected or classified. By these, video data or audio data constituting a program can be individually taken out. That is, multiplexing can be released.

  One of the purposes of the container format is to make it easy to handle different data in a single set. The container format is, for example, MMT or MPEG2TS.

  In FIG. 5, the received stream data is divided into three streams: an MPT stream, a video stream, and an audio stream.

  Another purpose of the container format is to replay these data in time. In other words, another purpose of the container format is to reproduce these data synchronously.

  In FIG. 5, when MPT-0 is received, asset information such as video or audio constituting the program can be obtained. Further, time stamp information is described for each of these assets. The time stamp information is information indicating timing such as video display or audio playback. The time stamp information is described in the MPT. The time stamp information is a reproduction time for the number of “stream transmission unit MPU” for each asset.

  For example, for video, MPT-0 describes the playback time of the first frame of MPU-v0 and the playback time of the first frame of MPU-v1. Also, MPT-1 describes the playback time of the first frame of each of MPU-v1 and MPU-v2.

  Similarly, for MP3-0, the playback time of MPU-a0 and the playback time of MPU-a1 are described in MPT-0. In addition, MPT-a1 playback time and MPU-a2 presentation time are described in MPT-1.

  Here, for the sake of explanation, it is assumed that each asset has two time stamps of “stream transmission unit MPU”. In practice, however, it is possible to have more time stamps.

  In this way, the playback time of the “stream transmission unit MPU” can be specified by the MPU. The video and audio can be reproduced in synchronization. Although not described here, the same applies to subtitles.

  A case where such information is recorded will be described with reference to FIG. FIG. 6 is a diagram for explaining a synchronization method using an MPU time stamp descriptor in the MMTP method. FIG. 6 shows an asset table, a time table, and a data file. The upper side of the data file is the top of the data.

  In the data file, MMTP packets constituting a program are sequentially recorded. Simply, when MMTP packets are recorded in the order of reception, the video data, audio data, control information, etc. are recorded in a mixed state. Here, for the sake of explanation, the “stream transmission unit MPU” is described in a summarized form.

  For example, “MPT-0” is MPT. And the order of receipt is given. Since the order of “MPT-0” is “0”, it indicates that it was received first.

  For example, “MPU-v0” is an MMTP packet selected as a video stream. The MMTP packet is handled as an MPU in which a plurality of MMTP packets are collected. And the order of receipt is given. Since the order of “MPU-v0” is “0”, it indicates that it was received first.

  For example, “MPU-a0” is an MMTP packet selected as an audio stream. The number of voice MPUs is not necessarily one per MMTP packet. Also, the audio MPU does not always appear at the same frequency as the video MPU. However, in order to simplify the explanation, it is assumed that the audio MPU occurs at the same frequency as the video MPU. Also, one audio MPU is provided for each MMTP packet.

  The packet described as “R” in the data file has “RAP_flag” set. It includes the beginning of the GOP. Therefore, video can be efficiently reproduced by starting to read data from this packet.

  Also, discarding incomplete data is reduced. The video data constitutes a GOP. For this reason, if the data of the head I picture portion is missed, the following dozens of pictures cannot be reproduced as video. If the head data of the GOP is missed, the read data is discarded and the next GOP head is awaited. Such discarding of data can be reduced. Such discarding of data causes a display delay at the time of cueing or random access. For this reason, smooth reproduction becomes possible by reducing the discard of data.

  As described above, the data file recorded in this way is not suitable for random access.

  First, the video data is variable length data. For this reason, it is impossible to specify where the data for reproducing the target video exists. Second, a packet for which “RAP_flag” is set cannot be directly called.

  Therefore, a search table for random access is prepared.

  In the asset table of FIG. 6, “packet_id” is stored. Using “packet_id”, the required asset can be extracted from the MMTP packet in the data file.

  In the time table, the recording position in the file of the packet including “RAP_flag” is stored. Then, the designated reproduction time of the “stream transmission unit MPU” including this packet is stored. These pieces of information are arranged in order of reproduction time.

  The contents of the asset table and the time table can be created from, for example, information described in the MPT and information recorded in the recording / reproducing apparatus 100.

  A case where reproduction is performed at a determined time will be described.

  For example, in FIG. 6, when displaying a video from time “0: 0: 1.00”, first, the time in the time table is searched. Then, “25000000”, which is the position on the file, is read out from the column where the times coincide.

  Therefore, data is read from the position “25000000” of the data file. Then, data reproduction processing is performed. As a result, data can be reproduced from the designated position.

  The data file is a state in which video data, audio data, or other data is mixed in units of packets. However, by referring to the asset table or MPT and performing packet classification, video data, audio data, or the like can be separated and reproduced.

  In some cases, the time table does not have the same time as the time at which playback is to be started. In this case, a time close to the time at which reproduction is to be started is selected from the times described in the time table, and data is reproduced therefrom.

  For example, when “0: 0: 1.70” is designated as the reproduction start time, “0: 0: 1.50” and “0: 0: 2.00” described in the time table are used. Then, “0: 0: 1.50” close to the designated time is selected. Then, the data is reproduced from the position “33000000” on the data file.

  The position on the data file is, for example, a position in byte units from the beginning of the file. Alternatively, the position on the data file is, for example, a position in block units. Alternatively, the position on the data file is, for example, a position in units of sectors.

  The case of special playback will be described. Special playback is, for example, fast forward or rewind. For example, in the case of fast-forwarding, the time table is sequentially read, and the file is read from the designated position. Then, when the data for one frame is reproduced, it moves to the position of the next time. As a result, data can be reproduced by fast-forwarding.

  FIG. 7 is a flowchart showing a procedure for creating a time table.

  In the Japanese broadcasting system, the MPT is retransmitted every 100 ms. If 1 GOP is 0.5 seconds, MPT is received 5 times during that time. One GOP is one “stream transmission unit MPU”.

  Further, it is allowed to store 15 time stamps in one asset of one MPT. That is, the time stamp is duplicated.

  In step S7001, an MMTP packet is extracted when stream data is received. When the MMTP packet includes MPT, time stamp processing is performed.

  In step S7002, MPT is extracted from the MMTP packet. Then, a combination of the sequence number of the “stream transmission unit MPU” for each asset and the time stamp information is extracted.

  In step S7003, duplication of time stamp information is removed. This is because the time stamp information is duplicated as described above. At this point, a sequence number of “stream transmission unit MPU” and a list of time stamps indicating the reproduction time are obtained.

  Although not included in this flowchart, MMTP packets including video data or audio data in parallel are sequentially recorded on the internal storage device 51 or the optical disc 53 as a data file. The position on the data file is determined at the time of recording.

  In step S7004, when the MMTP packet in which “RAP_flag” is set is recorded, the position of the packet on the file and the sequence number of the “stream transmission unit MPU” to which the MMTP packet belongs are extracted. Information on the position recorded on the file is added to the time stamp information having the same “stream transmission unit MPU” sequence number in the time stamp list.

  In step S7005, it is confirmed whether or not these processes are completed. If the process has not ended, “no” is selected and the process proceeds to step S7001. If the processing is completed, “yes” is selected, and the process proceeds to step S7006.

  In step S7006, the created data is written to the internal storage device 51 or the optical disc 53.

  In this way, a time table can be created.

  Here, random access is realized using time stamp information of an MPU time stamp descriptor set for each GOP. GOP is “stream transmission unit MPU”. Therefore, the position where data reproduction can be started is in GOP units. That is, the playback position can be specified only in units of 0.5 seconds or 1 second. For example, in the case of cueing by time designation, fast-forwarding or rewinding, this degree of accuracy is sufficient.

  However, when editing is performed after recording in the internal storage device 51 or the optical disk 53, it is not sufficient to specify the position in units of GOPs. For example, when different positions of the same program are combined and reproduced continuously, or when different programs are combined and reproduced continuously. With such a combination, a favorite scene collection or the like can be created.

  Therefore, the MPU extended time stamp descriptor in the MPT is used.

  FIG. 8 is a diagram showing a time table that can be searched by time in frame units.

  The time table shown in FIG. 8 is a time table in which a normal time table is extended so that searching can be performed by time in frame units. The time table shown in FIG. 8 has information of reproduction time, data position on the file, and AU number.

  The AU number indicates the AU number in the data belonging to the same “stream transmission unit MPU”. In the case of video data, AU corresponds to a picture. However, for the sake of decoding efficiency, the order of arrangement of AUs in a GOP does not necessarily match the display order of pictures.

  In this time table, they are arranged in the order of playback times. For this reason, the AU numbers are mixed. That is, the AU numbers are not arranged in order.

  In the MPU extended time stamp descriptor, the reproduction time is given to the AU in each “stream transmission unit MPU” as a difference from the AU displayed first. Alternatively, in the MPU extended time stamp descriptor, a reproduction time interval is given to each AU in each “stream transmission unit MPU”.

  Therefore, the reproduction time of each AU can be calculated from the reproduction time of the MPU time stamp descriptor and the difference time of the MPU extended time stamp descriptor. Alternatively, the playback time of each AU can be calculated from the playback time of the MPU time stamp descriptor and the playback time interval of the MPU extended time stamp descriptor.

  In FIG. 8, for the sake of explanation, it is described in units of 1/100 second as an example.

  In the MPU extended time stamp descriptor, a number that divides 1 second is defined as a timescale. The reproduction time of each AU is expressed using this time scale. The frame rate used in the video is 60 frames per second or 24 frames per second. If one frame is expressed in decimal units, it is not divisible and an error occurs. For this reason, a time scale is used. Therefore, a value using this time scale can be adopted in the time column of each frame in the time table.

  Even when data reproduction is to be started in the middle of a GOP, the decoding is always performed from the beginning of the GOP. Therefore, the data reading start position is the same within the same “stream transmission unit MPU”.

  An example of data reproduction using this time table will be described.

  When the search is performed by designating the time “00: 00: 01.04”, there is no picture corresponding to this time. For this reason, reproduction is performed from the picture of “00: 00: 01.03” which is the previous picture.

  Referring to the time table, this picture is an AU of AU number 2 of “stream transmission unit MPU” starting from a position “25000000” on the file. That is, the AU of this picture is the third AU.

  Therefore, reading is started from the position “25000000” on the file, and decoding is started. Decoding of the first AU is completed, and picture data is created. After this, the created picture is not displayed. Then, subsequent AUs having only difference information are decoded. Then, after the decoding of the third AU is completed, reproduction is started from the video of the third AU. In this way, data (video) can be reproduced from the middle of the GOP.

  In this example, reproduction is performed using one time table having detailed information. However, for example, in the case of the above-mentioned fast-forward playback, it is not necessarily an efficient method. Therefore, it is possible to take a method of performing a search in two stages of a standard time table and a detailed time table.

  In the new broadcasting system in Japan, the MPU extended time stamp descriptor specifies the playback interval between AUs. The interval is 1/60 second or 1/120 second. This interval is variable. However, the same frame rate is used in the program. Therefore, the reproduction time of each frame can be obtained by calculation without using a detailed time table.

  In this case, the search is performed using the time table for each “stream transmission unit MPU” shown in FIG. Then, the “stream transmission unit MPU” including the time at which playback is to be started is specified. Then, the difference between the reproduction time and the time at which reproduction is to be started is obtained. From the time difference and the frame rate, it is possible to determine the number of pictures in the display order in the “stream transmission unit MPU”.

  If there is no frame at the specified time, the frame immediately before the specified time or the frame immediately after the specified time in the display order. That is, the frame immediately before or after the specified time is adopted.

  As described above, the AU arrangement order and the picture playback time order are different. However, decoding is started from the head of the “stream transmission unit MPU”. Then, the decoding of the picture having the calculated reproduction order is completed. Playback starts from a picture that has been decoded. As a result, cueing can be performed with accuracy in units of pictures.

  Here, when the time table described above is created as a separate file and recorded, it is not necessary to record a time stamp in the data file for recording the program contents.

  The MPT includes control information other than the time stamp. For example, MPT includes asset information and the like. The time stamp is, for example, an MPU time stamp descriptor or an MPU extended time stamp descriptor. That is, the MPT includes control information other than the time stamp such as asset information in addition to the MPU time stamp descriptor or the MPU extended time stamp descriptor.

  However, these pieces of information are not of a nature that changes during the program. For this reason, the MPT itself can be omitted by recording it in one place as a separate file. For example, in FIG. 6, management information (Packet_id) is held as an asset table.

  When the MPT is stored alone in the MMTP packet, it is not necessary to record the MMTP packet storing the MPT when recording the MMTP packet in the data file.

  When the MPT is stored in the MMTP packet together with other management information, the MMTP packet is reconstructed with the management information excluding the MPT. Then, the reconstructed MMTP packet can be recorded.

  The MPU time stamp descriptor or the MPU extended time stamp descriptor is highly redundant data during broadcasting. For this reason, if it can be omitted, the data size to be recorded can be reduced.

  In the above description, it is not necessary to record the MPT. However, it may be convenient to record the MPT together with an MMTP packet including a stream such as a video stream or an audio stream.

  For example, when decoding processing is performed using an LSI in which the demultiplexing unit 21 and the decoding units 31, 32, 33, and 34 are integrated, data including MPT is input to the demultiplexing unit 21. Thus, demultiplexing, decoding, and synchronization processing can be performed collectively.

  Also, when performing cueing in units of frames, if there is an MPT before or at the beginning of the “stream transmission unit MPU”, the playback time in units of frames is set before starting playback with reference to this MPT. You can cue for it. The MPT may be near the head of the “stream transmission unit MPU”.

  Also in this case, it is not necessary to record all the MPTs transmitted from the television station. Processing can also be performed by recording only a part of the MPT.

  FIG. 9 is a diagram for explaining a synchronization method using an MPU time stamp descriptor in the MMTP method. In FIG. 9, the MPT is stored immediately before the “stream transmission unit MPU”. The time table indicates the position of the MPT, not the position of the head of the “stream transmission unit MPU”.

  In the conventional Blu-ray disc recording method, “EP_map” is stored in the clip file as information corresponding to these time tables. However, “EP_map” has a structure based on MPEG2TS having a fixed-length packet and a time stamp for each packet. Therefore, it cannot be applied to MMT data as it is.

  Therefore, the above time table is used instead of “EP_map”. As a result, data access can be facilitated when MMT data is recorded on a Blu-ray disc.

  In this example, description has been given of combining and recording MMTP packets. However, packets can be combined and recorded in each state of a TLV packet, an IP packet, or a UDP packet. It is also possible to expand the time table and record each asset. That is, it is possible to search for the reading start position of video data or audio data. Then, the data is combined and recorded as a list of MFUs.

[When recording using the standard format]
In MMT, as described above, a storage format is defined separately from the transmission format. In the MMT accumulation format, data is stored based on the BMFF (ISO / IEC 14496-12 ISO Base Media File Format) format. In this case, the data layout has the logical structure shown in FIG. FIG. 3 is a diagram showing a logical structure of data.

  This data chunk is called an MPU. This MPU is different from the aforementioned “stream transmission unit MPU”. Here, it is temporarily called “program storage MPU”. The “program storage MPU” is stored and managed as a normal file.

  The structure of “program storage MPU” will be described using the logical structure of FIG.

  The MPU metadata includes file management data, asset information, various parameters, hint information, and the like. Asset information is information for managing a combination of video data and audio data. Various parameters are parameters for setting the operation mode of the decoder. The hint information is information for reconstructing the MMTP packet from the MFU stored in the MPU file.

  Movie fragment metadata is information for accessing video data, audio data, or the like divided by playback time. Movie fragment metadata is information for reproducing video data or audio data.

  Actual video data and audio data are stored as a list of MFUs. One movie fragment metadata and a group of MFUs managed by the metadata are collectively called a movie fragment. In general, a plurality of movie fragments constitute one content. In network streaming or the like, generally, one fragment is composed of 10 to 15 seconds.

  When MPU metadata and movie fragment metadata are sent by broadcast wave Ba or the like, the MPU can be configured using these. Various header information until PLT, MPT, other control information or MFU is taken out is redundant and need not be recorded.

  On the other hand, in the new Japanese broadcasting scheme proposal, MPU metadata and movie fragment metadata are not transmitted during broadcasting. Therefore, the MPT metadata and the like are uniquely created by combining the PLT information, the MPT information, and various types of header information.

  Consider using the above-mentioned storage format for recording on the optical disk 53 as well. The optical disk 53 can read data stored physically continuously at a relatively high speed. However, when reading data at a position distant in the diametrical direction on the optical disc 53, it takes time to read the data because it involves a head seek. That is, in the optical disk 53, data that is highly likely to be used before and after is arranged in a continuous area or in a close position on the disk 53, so that data can be read efficiently.

  When data is recorded with the data arrangement as it is in the logical structure shown in FIG. 3, MPU metadata, movie fragment metadata, and MFU are distributed and stored as a physical arrangement. MPU metadata is data necessary for reproduction. The MFU stores video data. For this reason, due to the characteristics of the optical disc, the data arrangement with the logical structure shown in FIG. 3 is a disadvantageous data arrangement.

  In many file systems, the logical structure and physical arrangement of stored data can be managed separately. The same applies to UDF used in Blu-ray Disc.

  Therefore, the logical structure is as specified, and MPU metadata and movie fragment data are recorded together in physical arrangement.

  The data structure shown in the physical arrangement of FIG. 3 is an example in which management information is arranged together as a physical data arrangement on the optical disc. With this data arrangement, MPU metadata and movie fragment metadata can be read at once.

  Here, “data structure” indicates a structure viewed from a higher level than the file system. For example, a logical data structure, a data arrangement in a directory or one file, and the like. On the other hand, “data arrangement” indicates an arrangement viewed from a lower level than the file system. For example, physical data layout. It consists of blocks linked by file names and file names, or link information between blocks indicating data connections. In this case, the block is a set of data managed in association with a physical position on the disk. Depending on the file system, other names may be used.

  The data arrangement shown in the physical arrangement in FIG. 3 reduces the number of head seeks. The data arrangement shown in the physical arrangement of FIG. 3 can shorten the time required for starting reproduction and the time required for random access.

  In addition, it is possible to simply give a copy of management information to another file without considering the logical structure and the physical layout separately. The additional clip file shown in FIG. 3 is an example of such a case. In this case, for example, additional information for facilitating random access can be added to the file.

  In the examples shown so far, the data structure using fragments has been described. However, if transmission in fragment units is not assumed, recording in a data structure without movie fragment metadata is also possible.

  In the BMFF file format, management information and stream data are recorded together in the same file. On the other hand, in the conventional Blu-ray Disc recording method, management information and stream data are managed separately.

  This is because it is suitable for optical disk devices to read management information collectively and set the devices according to the data to be played, and then play back the stream data. The management information and the stream information are recorded in separate areas on the optical disc. This is also because the management data is recorded twice on the optical disk as backup data in preparation for damage to the optical disk. This backup data is recorded at a remote position on the optical disk.

  The management data in BMFF is information managed mainly in clip files in the Blu-ray disc. Therefore, when recording as BMFF, the management data portion of the program is referred from another file on the file system. That is, apparently, the management data portion of the program can be a separate file.

  FIG. 10 is an explanatory diagram when stream data is recorded as BMFF.

  FIG. 10 shows an example in which stream data is recorded as BMFF.

  As a physical arrangement, management data is recorded in a management data area on the optical disc 53. The stream data portion is recorded in the stream data area on the optical disc 53.

  When accessing this data as a stream file, the file system associates the management data portion with the stream data portion so that it can be logically viewed as one BMFF file.

  On the other hand, when this data is accessed as a clip file which is management data of the Blu-ray disc, only the management data portion of the BMFF file is made visible as a file. The clip file is management data of the Blu-ray disc.

  With such an arrangement, a BMFF format file can be created as an MMT standard recording format. Also, the data format can be consistent with the Blu-ray Disc management method.

  In this way, if the header portion of the data recorded as BMFF can be referred to as another file, it is advantageous in data management on the optical disc 53. Also, it becomes easy to create backup data for management data.

  For example, when extracting video data with a PC or the like, if the BMFF file is copied, it can be copied as a BMFF file including both management information and stream data. On the other hand, when creating a backup of the management information in the optical disc, if a copy is created from the BMFFCLIP file, only the management data portion that needs to be backed up can be copied.

  Further, the management data portion of the BMFF can be stored in an area where other management data of the optical disk 53 is stored in accordance with the format of the optical disk that stores the management data and the stream data separately. For this reason, the amount of head seek when preparing for reproduction can be reduced.

<< (B) Recording of Data in the State of Elementary Stream Se Demultiplexed >>
In the above-described example, the method of converting received data to a storage format while partially demultiplexing has been described. The received data is data received from the broadcast wave Ba, the external device Ei, the network Ne, or the like.

  However, when the demultiplexer 21 is made as hardware, it can be extracted and recorded in an ES (Elementary stream) state separated into video or audio. Note that ES is shown as an elementary stream Se in FIG.

  The ES is a compressed video data stream or audio data stream. The ES is divided into units. Here, the “unit” is a meaningful unit in processing. This unit is, for example, a picture or NAL in the case of video data. This unit is, for example, a block in the case of audio data.

  In the following, the video recording format will be described taking ISO BMFF as an example.

  In the method of recording data in a multiplexed state in BMFF, data for managing data with a flow of time is called a track.

  In the recording method in which the multiplexing is not released, there is one track. This is, for example, a state in which video data and audio data are multiplexed, so that data having a time flow is one of the multiplexed streams Sm. The multiplexed stream Sm is expressed as a list of MFUs.

  On the other hand, in the state of the elementary stream Se, for example, video data and audio data exist as separated ES stream data. Therefore, a track for video data and a track for audio data are created separately. The video data track and the audio data track are recorded in an area for storing management data on the data file. The video data and audio data are stored in an area for storing media data.

  Each track indicates a playback time. Each track indicates the position of an area for storing media data corresponding to the reproduction time. The media data is video data or audio data.

  The video data and the audio data are related in a time-synchronized state. Similarly, subtitles and the like are associated with each other including display timing.

  In the new Japanese broadcasting scheme proposal, a combination of video, audio or subtitles used in the program is shown in the MTP of the MMTP packet. In addition, the display timing or playback timing of video, audio, or subtitles is shown in the MTP of the MMTP packet. In addition, the storage location of data such as video, audio, or subtitles is shown in the MTP of the MMTP packet.

  When a data file or the like constituting a program (data broadcasting parts, etc.) is sent by MMT, one file is stored in one MPU. This MPU is referred to as a “data element MPU”. Then, this “data element MPU” is divided and sent as MFU by MMTP. In this case, the MPU header or MPU metadata is also sent at the same time.

  For stream data such as video or audio, the data element MPU is not used to avoid overhead. Instead, AU or NAL is sent directly into MFU and sent by MMTP.

  However, in the time management of the new Japanese broadcasting scheme plan, the MPU is configured in units of random access flags. The time is specified for the MPU. For this reason, there is no problem as a unit of time designation if one GOP is a unit of 0.5 second to 1 second. In this case, the MPU header or MPU metadata is not sent. As described above, a set of a set of MMTP packets with the random access flag as a delimiter is temporarily referred to as “stream transmission unit MPU”.

  The multiplexed data is received and recorded in a file. The data structure used when recording in this file is also MPU. For example, when recording in a file, the entire program can be one MPU. Here, the MPU in this usage is temporarily referred to as “program storage MPU”. It is also possible to record in units of stream transmission units MPU. In this case, the number of MPU files becomes enormous. Further, the stream transmission units MPU can be bundled and stored in the MPU. Here, the MPU in which the stream transmission units MPU are bundled is temporarily referred to as “program storage MPU”.

  In the case of a new broadcasting system in Japan, a “stream transmission unit MPU” is configured in GOP units. The display time of the first frame is specified by the “MPU time stamp descriptor” in the MPT. The time here is NTP time. The display time of the subsequent frame in the GOP is designated by the “extended time stamp descriptor”. The display time of the subsequent frame is indicated by a difference from the display time of the first frame.

  In the state where the multiplexing is canceled and the elementary stream Se is taken out, the MPT is removed. Therefore, it is necessary to create and record management information such as tracks by combining MPT and other control information. A track is data management information with a flow of time. The data having the flow of time is, for example, video data or audio data. The track includes a data type, a reproduction time, a pointer to an actual data recording position, and the like. These data can be created in the same procedure as the time table creation described above.

  In the track, the data extraction position can be specified in fine time units. Therefore, video data and audio data can be mixed and arranged in the order of reproduction time or decoding time. By adopting such a structure, video data and audio data that need to be reproduced at the same time can be extracted while suppressing head seek.

  In this way, a program transmitted by MMT can be recorded as a BMFF format file.

Embodiment 2. FIG.
In the second embodiment, MMTP packets are sequentially recorded. The MMTP packet is a selection of received broadcasts at the program level.

  Data transmitted from one transmitter is created so that a plurality of programs from a plurality of broadcasting stations can be multiplexed. Therefore, when a user views or records a specific program, it is necessary to separate at the broadcast station level and at the program level.

  Here, it is assumed that the separation to the program level has been completed, and the MMTPs constituting one program to be recorded are sequentially taken out and recorded. This state is a state in which data multiplexing is partially canceled. That is, multiplexing is canceled up to the program unit. However, the individual video, audio, subtitle stream or control data constituting the program is multiplexed.

  FIG. 11 is an explanatory diagram for explaining demultiplexing by MPT in the MMTP method. FIG. 12 is an explanatory diagram for explaining the relationship between the time table and the video data. FIG. 13 is a diagram for explaining an example of the structure of MPT data in the MMTP method. FIG. 14 is an explanatory diagram for explaining alignment when an image is mainly considered.

  The MMT stream shown in FIG. 11 is an example of an MMTP packet sequence that constitutes a program. In this MMT stream, for example, video, audio, and subtitles are multiplexed one by one. In actual broadcasting, a plurality of video, audio, subtitles, and the like may be multiplexed.

  Video, audio, subtitles, and the like are elements that make up a program. Elements constituting these programs are called assets. The MMT stream is multiplexed in units of MMTP packets. Therefore, the service information packet and the asset data packet are mixed in the MMT stream.

  The service information packet is a control signal for the multiplexed stream. The service information is, for example, control information for separating video, audio, or subtitles. The service information is control information for reproducing video, audio, and subtitles in synchronization. The service information is information for displaying the name of each asset.

  The asset data packet constitutes, for example, video, audio, and subtitle assets. In the second embodiment, these packets are sequentially recorded on the recording medium. Here, for simplicity, it is assumed that these packets constituting the MMT stream are sequentially recorded as files.

  As shown in FIG. 11, the MMT stream includes a control packet SI (MPT), a video packet V, a video packet V (RAP), an audio packet A, and a caption packet T. The control packet SI (MPT) is a control packet with MPT. The control packet SI (MPT) is a kind of service information. The video packet V (RAP) is a video packet with a RAP flag.

  Actually, many types of control packets SI are transmitted. However, in order to facilitate the explanation, the description here focuses on MPT. According to the broadcast standard, MPT is to be transmitted periodically at intervals of about 100 ms.

  When sequentially reproducing the MMT stream from the beginning, data is sequentially extracted from the beginning of the file in which the MMT stream is recorded. Then, data can be reproduced by sequentially sending the MMT stream to the demultiplexing unit 21. For example, the multiplexed stream Sm shown in FIG.

  The demultiplexing unit 21 sequentially analyzes the received data. When receiving the data packet including the MPT, the demultiplexing unit 21 extracts information used for the demultiplexing and video display timing from the description of the MPT. Then, the demultiplexing unit 21 sets the demultiplexing unit 21 and the decoding units 31, 32, 33, and 34.

  FIG. 13 shows an example of data described in MPT. The data shown in FIG. 13 is based on the ARIB standard STD-B60. For example, MPT describes information for extracting a plurality of assets. Each asset obtains information for acquiring data and information for separating the data based on the identification information in the MMT. The identification information in the MMT is, for example, location information (MMT_general_location_info ()).

  Although omitted in FIG. 13, the detailed information of each asset stream can be obtained from the component descriptor information. Based on this information, as shown in FIG. 11, the multiplexed streams are separated and reproduced.

  Even when broadcasting is recorded, the data of the MMT stream of FIG. 11 is sequentially recorded in a file. The data of the MMT stream is sequentially extracted from the beginning of the file and reproduced. In this case, the MMT packet itself is not necessarily required. It may be recorded in a state where data inside the MMT packet is taken out. Conversely, it may be recorded in the state of an IP packet or the state of a TLV packet.

  Next, consider random access. Here, a time designation jump will be described as an example. However, the same applies to chapter search and fast forward / rewind.

  The random access operation will be described with reference to FIG. The time table TM is data for associating time information on a program with a reproduction position corresponding to the time information. Here, the reproduction position will be described as an offset from the beginning of the file in which the data of the MMT stream is recorded. However, a sector address or a block address may be used as the reproduction position.

  Using this time table TM, data is extracted from the reproduction position corresponding to the time and reproduced. This makes it possible to perform a time designation jump. That is, it is possible to perform reproduction with a designated time.

  Usually, video data is configured in GOP units. The GOP is, for example, a compressed image of about 0.5 seconds to 1 second. The GOP includes, for example, several tens of images. One GOP includes one to a plurality of complete images and several tens of difference images. GOP is efficient because it contains a compressed difference image.

  Further, the display order does not always correspond to the decoding order. Therefore, even if playback is desired from the middle of the GOP, it is necessary to always decode the data from the beginning of the GOP. Therefore, in the MMTP packet, the RAP flag can be added to the head of the GOP. The RAP flag is a flag indicating that random access is possible. In FIG. 12, the position of the packet with the RAP flag is stored in the time table as the head of the GOP. As a result, random access is realized.

  Image data that can reproduce a complete image from one piece of image data is called an I picture. One I picture always exists in one GOP. However, a plurality of I pictures may exist in one GOP. Of the I pictures, the I picture that is first decoded in the GOP is called IRAP because of dependency with other images. By reproducing from this I picture, subsequent image data can be correctly displayed.

  The RAP flag of the MMTP packet is attached to the head packet or control packet of the IRAP image data. Image data composed of difference data from one I picture is called a P picture. By synthesizing the reference destination I picture and the P picture image data, the P picture image can be reproduced. Image data that can be reproduced by referring to a plurality of other images is called a B picture.

  Similar to video data, audio data has the concept of the beginning of a piece of data. A RAP flag is added to the head of the audio data. However, in order to simplify the description, here, the audio data packet is also divided by the GOP of the video data.

  Let us consider the random access operation shown in FIG. 12 corresponding to the packet level reading shown in FIG.

Retrieving position information is extracted from the time table TM. Then, the play is taken out packets from the position _{P 1} of the MMT stream of FIG. Position _{P 1} is the position of the first video packet V (RAP). At this time, MPT information has not been acquired. Therefore, it is not possible to perform separation in units of assets such as video, audio and subtitles.

Control packet SI (MPT) are read is performed sequentially read at the position _{P 2.} Thereby, a parameter for separating the assets is acquired. That is, video, audio, subtitles, and the like can be separated.

At this time, it can not be data to separate from the position P ₁ to the position P _2, if it is not read as a result would beginning of the data of the GOP is lost. This GOP cannot be decoded.

Position P ₃ later, it is possible to decode because it already has acquired the MPT information at the position P _2. Position _{P 3} is the beginning of the next GOP position _{P 1.} For this reason, there is a case where the reproduction is started from the next GOP instead of the GOP which is originally intended to start the reproduction.

In the case of chapter search or fast-forwarding in the same program, if the MPT asset information used for playback matches the asset information of the jump destination MPT, from the position P ₁ that is the head of the jump destination It is possible to play from the starting GOP. However, when a program is played back with a commercial (CM) or program change, there is no guarantee that the asset is the same between the MPT before the jump and the MPT after the jump. Alternatively, when editing is performed by connecting different recorded programs, there is no guarantee that the asset is the same in the MPT before the jump and the MPT after the jump.

  In addition, when an asset is changed, the draft broadcast standard requires that the MPT be updated 0.5 seconds before the actual data is reproduced. For example, it takes time to set a filter for separating assets. Also, it takes time to perform mute processing when the sound is switched. Also, it takes time to perform mute processing when the video is switched. For this reason, even if the MPT is changed at the timing immediately before the actual data is switched, there is a possibility that the process for switching or muting the filter may not be in time.

  The same applies to random access. Even if the MPT can be acquired immediately at the jump destination, the processing system cannot be switched in time, and the first GOP may not be displayed correctly. Alternatively, the display needs to be delayed in order to secure the time required for switching data. Here, the “processing system” is, for example, the demultiplexing unit 21 and the decoding units 31, 32, 33, and 34.

  Therefore, in the second embodiment, as shown in FIG. 12, first, MPT information is stored in the time table TM. Then, when jumping by designating the time, the MPT information is set in the demultiplexing unit 21 and the decoding units 31, 32, 33, and 34. Thereafter, the data at the position where reproduction is started is read out.

  In this way, the processing system (demultiplexing unit 21, decoding units 31, 32, 33, 34, etc.) can be switched in parallel with preparation for reading data such as seek. And the time required for data switching can be shortened.

  In this example, the MPT itself is stored in the time table TM. As another method, the information for separating the assets described in the MPT and the GOP display start time are stored in the time table TM. The time when the GOP display is started can also be used as the time of the time table TM. As described above, there is a method of extracting asset information and storing it in the time table TM.

  As another method, MPT or asset information is recorded in another table. Then, there is a method in which the time table TM has a reference to the corresponding MPT information or asset information. In addition, there is a method of preparing another time table that can be searched at the same time, not by reference, and recording MPT or asset information there. The MPT stores the display start time of a plurality of GOPs. Some or all of these may be stored in the time table TM.

  Similarly, even when the control of the television is changed by the HDR parameter or the like, it may take time to change the backlight luminance or the setting of the liquid crystal drive voltage. In that case, HDR parameters and the like are stored in the time table TM. This makes it possible to control the television in advance. In FIG. 12, the metadata area of the time table TM is shown as an example of an area for storing these parameters.

  When performing special reproduction, there are cases where only IRAP images (IRAP pictures) are reproduced in the GOP and other images are not displayed. Special reproduction is, for example, fast forward or rewind. The IRAP image is the first image in the decoding order. The IRAP image is also placed at the head of the GOP in the data arrangement order. That is, the IRAP image is located at the beginning of the GOP. The IRAP image does not depend on other images. For this reason, the IRAP image can be decoded independently. In this case, the position of the IRAP image can be read from the time table TM.

  However, in order to skip reading efficiently, it is more convenient to know the end of the IRAP image. Therefore, the size of the IRAP image or the end of the IRAP image is stored in the time table TM. As a result, special reproduction can be performed efficiently. In FIG. 12, the IRAP size column of the time table TM is shown as an example of an area for storing this data.

  When image data constituting a video is recorded as a slice, it may be better to access the data in units of slices. “Slice” means that image data is divided in a decodable state. In the case of special reproduction, for example, there is a method in which only a specific slice of an IRAP image in a GOP is reproduced and no other slice is displayed. Special playback is, for example, fast forward or rewind.

  In this case, the position and size of each IRAP image data and slice at the head of the GOP are stored in the time table TM, so that special reproduction can be performed efficiently. Although not shown in FIG. 12, for example, this data can be stored by expanding the IRAP size column of the time table TM.

  In order to smoothly perform fast forward and rewind, a non-IRAP I picture or non-IRAP P picture included in a GOP may be displayed in addition to an IRAP image. In this case, the start position, size, or end position of each of the I picture and P picture is stored in the time table TM. This makes it possible to perform special reproduction efficiently.

  Although not shown in FIG. 12, for example, these data can be stored by expanding the IRAP size column of the time table TM. Alternatively, a table indicating the position, size, or end position for I picture or P picture is prepared separately. The time table TM can also store a reference to this table.

  Next, consider storing the MMT stream in the second embodiment in a disk device such as an HDD or an optical disk.

  Many disk formats use fixed-size data blocks as access units. This access unit is called a sector, block, cluster or page. Here, it is simply called “block”. For example, in the Blu-ray Disc standard, 6144 bytes are called an aligned unit and are handled as one recording unit. In the Blu-ray Disc standard and UDF standard, one block 2048 bytes is often used. UDF (Universal Disk Format) is a file system for optical disks.

  When performing random access, it is possible to access efficiently by aligning the data boundary with this block. In addition to the structure of the disk device, for example, the block size may be determined as an encryption unit. Even in this case, it is efficient if the data can be accessed according to the block boundary as an access unit. Determining the data size or the data storage location in consideration of such access efficiency is called alignment.

  When recording a transport stream (TS), 32 TS packets are connected to form 6144-byte data. The TS packet is 192 bytes. This enables efficient recording in the three UDF blocks. One block of UDF is 2048 bytes. The MMTP packet is a variable length packet. For this reason, it is not possible to simply match the block boundary with the number of packets.

  As shown in FIG. 12, when performing random access, it is desirable to efficiently access the head of the GOP. However, in the case of reproducing from any image in the GOP, it is necessary to read data from the head of the GOP. Therefore, it is considered to record in accordance with the block boundary in GOP units.

  FIG. 14A shows an example of a file that records an MMT stream in which the GOP is used as a data access unit and the head of the GOP is an integral multiple of the block size.

  FIG. 14A shows a part of data of a file in which an MMT stream is recorded. The left side of FIG. 14A corresponds to the front of the file, and the right side corresponds to the back of the file. For example, the packets of the MMT stream are sequentially recorded from the left side to the right side in FIG. FIG. 14A shows only a part of the recorded data. In practice, this data structure is repeatedly recorded.

  Symbol Bb in FIG. 14A indicates a block boundary. “N1”, “n2”, “n3”, “n4”, “n5”, and “nm” indicate integers. “M” is a positive integer. “×” indicates multiplication. Therefore, “block × n” indicates that the data is an integral multiple of the block size.

  When alignment is performed so that the head of the GOP becomes a block boundary, the data size of the GOP is not necessarily an integer multiple of the block size. For this reason, padding (Pad) in FIG. 14 is used. “Padding” means intentionally creating an invalid area in order to align data. There are several methods of padding. For example, one method is to record data indicating invalidity. In addition, there is a method for managing an invalid data area by the OS or the file system.

  In the example of FIG. 14 (A), alignment is performed in blocks in GOP units. For this reason, random access in GOP units can be performed efficiently. In this example, only video data is described. However, actually, as shown in FIG. 11, video data, audio data, caption data, and the like are multiplexed and mixed.

  Therefore, when the stream data is divided at the video data break position, the audio data or subtitle data other than the video is not necessarily divided at an appropriate break position. Therefore, there is a possibility that audio or subtitles are reproduced with a delay during random access. Usually, even if the reproduction of audio or subtitles is delayed, there is no problem if there is no deviation from the video. For this reason, here, other data is also handled as a set of data based on the video break position.

  FIG. 14B shows an example of a file in which MMT streams aligned in units of images are recorded. In FIG. 14B, the combination of a picture (AU) and padding is an integral multiple of the block. In this case, the efficiency is high when access in units of images is required.

  AU is a kind of meaningful chunk of compressed data. In the case of image (picture) data, it is one image. That is, the AU is any one of an IRAP picture, an I picture, a P picture, or a B picture.

  In FIG. 14B, all alignment is performed in image units. However, the IRAP picture is important for random access. The IRAP picture is placed at the beginning of the GOP. Therefore, it is possible to perform alignment by dividing into two, that is, an IRAP picture and a group of other pictures.

  In FIG. 14C, the IRAP picture head, the I picture head, and the P picture head are aligned with block boundaries. In FIG. 14C, the I picture is indicated as “I”, the P picture is indicated as “P”, and the B picture is indicated as “B”. In the case of fast-forwarding or rewinding, it is efficient when using not only an IRAP picture but also an I picture or a P picture. In FIG. 14C, alignment is performed at the head of the IRAP picture, the head of the P picture, or the head of the non-IRAP I picture. However, the alignment at the head of the P picture or the head of the I picture may be omitted.

  FIG. 14D shows an example of a file in which an MMT stream in which alignment is set in consideration of access in units of slices. When the video has a slice configuration, it is possible to efficiently access in units of slices.

  In FIG. 14D, non-VCL data is described at the beginning of image (picture) data. Non-VCL data is parameters such as AUD, VPS or SPS. Subsequent slice # 1 to slice # 4 are VCL data. VCL data is compressed image data.

  Even when decoding is performed in units of slices, non-VCL data is required when decoding each slice. For example, when decoding slice # 2, non-VCL data and slice # 2 data are combined and decoded. Therefore, here, the alignment is performed for each of the non-VCL data and slices # 1 to # 4.

  In practice, non-VCL data is often very small compared to each slice. Also, before decoding slice # 1, non-VCL data is required. From this, it can be considered that the non-VCL data and the slice # 1 are aligned together.

  Alignment in units of slices is not necessarily performed for all pictures. For example, it is also conceivable to use only IRAP pictures for alignment in units of slices. This is because the IRAP picture has a large size and is used for random access.

  A method of aligning in units of MMTP packets is also conceivable. The amount of data of the MMTP packet differs depending on the standard. For example, the maximum value of the MMTP packet in the broadcast standard draft is about 1500 bytes.

  For example, when alignment is performed assuming a 2048-byte block, the amount of padding in the total data amount increases. Then, the recording efficiency is deteriorated.

  On the other hand, access in packet units can be performed efficiently. That is, the alignment in units of MMTP packets is suitable for temporary data storage when data processing in units of packets is required. Data processing in packet units is performed, for example, at the time of recording / playback or editing.

  By using the time table TM and alignment, it is possible to efficiently reproduce data specifying a time.

  The Blu-ray disc has a correspondence table (EP map, EP_map) as a mechanism corresponding to the time table TM of the second embodiment. This EP map indicates the reproduction position by the system clock and the position of the TS packet. The reproduction position is a position for reading out PTS (display start time) and its data.

  The EP map is used for jumping by specifying a time, random access, reproduction by a playlist, or the like. Random access is, for example, fast forward or rewind. The playlist is a list in which the playback part of the stream and the playback order are determined.

  In the EP map of the Blu-ray disc, the data format of the time and playback position is different from that in the case of MMT. However, the mechanism for specifying the time and acquiring the reproduction position data is the same as the case of using the time table TM described in the second embodiment.

  By making the time table TM similar in structure to the EP map, it is possible to record and play back an MMT broadcast on a Blu-ray disc without significantly changing the playback mechanism.

  For example, the time format of MMT playback is the absolute time (world time) at the time of broadcasting. On the other hand, in the EP map of the Blu-ray disc, it is a relative time using an internal clock. Therefore, the time notation of the time table TM is converted into a relative time notation. Alternatively, separate data for conversion between absolute time and relative time is provided. Alternatively, the playback time information used on the Blu-ray disc is changed to the absolute time format.

Embodiment 3 FIG.
In the new broadcasting system in Japan, the multiplexing system and the encoding system are changed. Moreover, the transmission rate is also raised in the new broadcasting system in Japan. For example, according to ARIB TR B-39, a transmission rate of 35 Mbps is assumed for 4K broadcasting, and a transmission rate of 100 Mbps is assumed for 8K broadcasting.

  On the other hand, in the BDXL (registered trademark) disk currently on the market, the transmission rate of the optical disk is around 140 Mbps.

  Therefore, when playing back a disc recorded with 8K broadcast, there is no problem in normal playback and playback is possible. However, when special reproduction such as fast-forwarding is performed on a disc on which 8K broadcast is recorded, there is a problem that it takes time to display an image for one frame.

  In Japanese Patent Laid-Open No. 2000-125259, a recording area of a recording medium is divided into a predetermined data size, a stream is decoded at the time of recording, and a TS packet including a PCT (picture type code) indicating an I picture is And a pointer indicating the start address of the sector is registered in a table provided on the recording medium. Then, special playback operations such as fast-forward and fast-rewind using the recording medium recorded in this way read the pointer address from the pointer table, reproduce only one I picture from the sector indicated by the pointer, and then successively It describes that the sector indicated by the pointer is sequentially reproduced.

  As described above, when performing special reproduction such as fast-forwarding, a method of reproducing only an I picture is often used. The I picture is at a random access point of the video stream. In accordance with the fast forward speed, all the I pictures are displayed in normal fast forward (FF, Fast Forward). In double-speed fast forward (FF × 2), one I picture is skipped and displayed. Further, in the fast forward (FF × 3) at 3 × speed, two I pictures are skipped and displayed. In this way, the playback speed during fast forward is adjusted.

  However, when the ratio of the I picture in the GOP is large, it takes time to read out the I picture, and the update interval of the image (I picture) becomes long.

  The video playback apparatus according to Embodiment 3 can shorten the I-picture readout time and the image (I-picture) update interval during special playback.

  In conventional fast-forward playback, the positions of reproducible I pictures on the disc are managed in a table. Then, using the table, I picture data is read sequentially.

  In an optical disk, it takes time to move the head when changing the reading position. This moving time of the head is called a seek time.

The update interval of the fast-forward image on the screen is expressed by the following formula 1.
Update interval = seek time + I picture data read time + decode time (1)

  Therefore, when the ratio of the I picture in the GOP is large, it takes time to read out the I picture, and the update interval of the image (I picture) becomes long. In 4K / 8K broadcasting, the HEVC compression method with good compression efficiency is adopted. Therefore, the I picture ratio is larger than that of the conventional image compression method.

  Even when the difference between the data rate of the video stream and the maximum reading speed from the disk is small, the I picture read time becomes long and the image update interval becomes long. For example, when a 100 Mbps broadcast stream is read from an optical disc with a maximum read speed of 140 Mbps, it can be read only at a speed of 1.4 times.

  As described above, when 4K / 8K video is recorded on the optical disc, the image update interval at the time of special reproduction becomes longer than that in the past. On the other hand, special playback operations such as fast-forwarding are often performed at the time of playback of video recording broadcasts. When the user specifies a target scene, it is desirable that the image update interval is short.

  Therefore, if the image update interval is long, it becomes difficult for the user to find the target scene when searching for a scene using fast-forwarding.

  The object of the third embodiment is to provide a video playback apparatus that is easy to find a target scene during special playback such as fast-forward playback and has good operability.

  Organize the image update interval during fast-forward from the user's point of view. For example, consider fast-forwarding in which a video with an original playback time of 100 seconds is played back in 10 seconds. Here, it is referred to as 10 times fast forward.

  Normally, the user checks the fast-forwarding video and cancels fast-forwarding when the target scene appears. That is, the normal playback speed is restored. During fast-forward playback, the displayed video is intermittent video. However, it is easier for the user to recognize the displayed scene when there is less missing time information. In addition, it is possible to reduce the deviation of the position at which the fast-forwarding is canceled and the normal playback is resumed after the fast-forwarding operation is completed.

  For example, consider 10-times fast forward. If the update interval of the fast-forwarding image is 1 second, one frame is displayed for the normal reproduction time of 10 seconds. On the other hand, if the update interval of fast-forwarding images is 0.5 seconds, one frame is displayed for the normal playback time of 5 seconds.

  In order to find a target scene during fast-forward playback, it is advantageous that the number of frames (images) extracted from the normal playback time is large. Thus, operability can be improved by displaying many images during fast-forward playback. That is, operability can be improved by shortening the update interval.

  In 4K / 8K broadcasting, a stream format having a plurality of slice segments is defined assuming division decoding.

  For example, when one image is divided into four, the image is divided into four vertically and horizontally (hereinafter also referred to as a square shape) to create a four-divided image. Alternatively, one image is divided into four parts in the vertical direction (hereinafter also referred to as the letter shape). Each slice segment can be decoded independently. This is a consideration for decoding 4K video using four 2K decoders and decoding 8K video using four 4K decoders.

  When a broadcast stream having a plurality of slice segments is recorded on an optical disc, only a part of the slice segments can be reproduced if the data can be read out in units of slice segments.

  If this is used during special playback, the updated image becomes part of the image displayed in normal playback. However, the “I picture data read time” in equation (1) can be reduced to a quarter. In addition, the image update interval during special reproduction can be shortened.

  FIG. 15 is a schematic diagram of a video stream according to the third embodiment.

  As the multiplexing method, both the TS method and the MMT method are conceivable. However, in FIG. 15, unique header information and the like are omitted. Also, padding and the like are omitted.

  In the case of broadcasting, one GOP can contain up to about 60 frames (images or pictures). Then, at least one I picture is included in one GOP. In particular, the I picture that is the starting point for decoding is called an IRAP picture (IRAP picture). In the data order, the IRAP picture is usually arranged at the head of the GOP.

  An I picture (including an IRAP picture), a B picture, and a P picture each represent a frame (an image or a picture).

  The I picture indicates an independent image that can be decoded independently. On the other hand, the B picture and the P picture depend on other images. The B picture and the P picture are difference data from other images. For this reason, the B picture and the P picture cannot be decoded alone.

  The GOP has a complete dependency relationship between images in the GOP. That is, all the images in the GOP constitute a set of data that can be decoded.

  When one image is composed of a plurality of slice segments, each image is composed of a parameter set and a plurality of slice segments. By combining the parameter set and one slice segment, the slice segment becomes a decodable unit. In FIG. 15, only the IRAP picture is described in the slice segment structure. However, the B picture and the P picture can take a similar structure.

  When decoding in parallel using four decoders, data is extracted for each slice segment. Then, a parameter set is added to each data. Thereafter, by supplying each data to each decoder, one image can be decoded in a divided state. When displaying an image, the individually decoded images are combined and displayed as one image.

  FIG. 16 shows an example in which the image is vertically and horizontally divided into four (field shape). An image of slice # 1 is displayed on the upper left of the image. An image of slice # 2 is displayed on the upper right of the image. An image of slice # 3 is displayed at the lower left of the image. An image of slice # 4 is displayed at the lower right of the image.

  FIG. 17 shows an example in which the vertical direction is divided into four (eye shape). The image of slice # 1 is displayed on the top from the top of the image. The image of slice # 2 is displayed second from the top of the image. The image of slice # 3 is displayed third from the top of the image. In the fourth image from the top, an image of slice # 4 is displayed.

  In the case of an undivided video stream, there is only one slice segment in one picture data.

  Consider special reproduction with a video stream having such a divided slice segment structure.

  As described above, in order to improve operability, it is necessary to shorten the display image update interval. Conventionally, data of the entire I picture is read and the entire I picture is displayed. However, it is assumed that only the data of the first slice segment (slice segment # 1) of the I picture is read and only the first slice segment (slice segment # 1) of the I picture is displayed.

  FIG. 18 is an example of a time table (TMS) corresponding to reading in units of divided slice segments. FIG. 19 shows the correspondence between each item of the time table (TMS) and the data. When recording a video stream, the video stream is analyzed. Then, the time table (TMS) is created by taking out the time information or the data delimiter position information.

  Two types of codes TM and TMS are used as the time table. The time table corresponding to the divided slice segments is distinguished from the code TM and the code TMS is used. The time table is recorded on the recording medium together with the video stream. The recording medium is, for example, an optical disk.

  Each item of the time table (TMS) will be described.

  “Time” is the display time of the GOP indicated by the row of each time. The time is recorded in the form of time information in the system clock format, time information in the ntp format, or a difference time from the beginning of the stream. When starting reproduction by designating a time, the time column is searched, and reproduction is started from data in the vicinity of the designated time.

  Here, the “time” is the GOP display time. One GOP usually includes a plurality of images. Since each image has a display time, there are a plurality of display times in the GOP. When recording in the time table as the GOP display time, the display time of the first image in the display order in the GOP can be used. Alternatively, the display time of the first image in the decoding order can be used. In this case, the head image is an IRAP picture.

  Further, the “time” stored in the time table does not necessarily need to coincide with the display time of the image. For example, time information with reduced accuracy or time information for other data can be used instead of the image display time itself. For example, the accuracy of the system clock of the playback device, the relationship with data access, or the data length that can be stored in the time table is considered.

  The “IRAP start position” is a storage position of the IRAP picture. The storage location of the IRAP picture is a GOP random access point. The storage position of the IRAP picture is normally the same as the start position of the GOP. The “IRAP end position” indicates the data end of the entire IRAP picture. By reading data from the IRAP start position to the IRAP end position, there is data that can decode the IRAP picture.

  “# 2 start position” indicates the start position of the second slice segment (slice # 2). “# 2 start position” indicates the end position of the first slice segment (slice # 1). “# 3 start position” indicates the start position of the third slice segment (slice # 3). “# 3 start position” indicates the end position of the second slice segment (slice # 2). “# 4 start position” indicates the start position of the fourth slice segment (slice # 4). “# 4 start position” indicates the end position of the third slice segment (slice # 3). “# 4 end position” represents the end position of the fourth slice segment (slice # 4). Normally, this position is the same as the end position of the IRAP picture.

  The “parameter set” stores a parameter set of IRAP picture data. The parameter set includes, for example, AUD (Access Unit Delimiter), VPS (Video Parameter Set), SPS (Sequence Parameter Set), SPS (Picture Parameter EnferE), and SEI (Supplementary Energy In). include.

  In the time table (TMS) shown in FIG. 19, the end position of the slice segment is stored. For example, the end position of the IRAP picture and the end position of slice # 4. However, it is possible to use a method in which the data size is stored in a table instead of the end position.

  The start position and end position are recorded in the form of a byte position or a block position from the beginning of the data. In addition, when TS is adopted as the multiplexing method, the start position and the end position are recorded in the form of the position of the TS packet. The start position and the end position are recorded as positions necessary for reading data.

  The information indicating the data position of the segment can be a relative position from the beginning of the GOP. As a result, the data size of the position information can be reduced.

  In an optical disc, data is read in units of data that are collected to some extent. For this reason, the position need not be exact. It is also possible to reduce the amount of position information by increasing the unit indicating the position.

  In this example, information on slice segments is also recorded in a single time table. However, slice segment information can also be stored in a separate table. In this case, information about the same GOP and IRAP picture can be extracted from the time stamp information or the entry position in the table.

  A method of storing the parameter set in a separate table is also conceivable. This is because the parameter set is larger than the position information. This is because the data length of the parameter set varies depending on the image.

  When fast-forward playback is performed using only the slice segment # 1 of the I picture, the information required in the time table is the time, the IRAP start position, and the # 2 start position. In this case, the # 2 start position is used as the end position of the slice segment # 1.

  The fast-forward playback using only slice segment # 1 will be described with reference to FIG. GOP (01) to GOP (06) are stream data. Originally, stream data is a continuous data file. However, for ease of explanation, the lines are changed for each GOP. TMS is the time table described above. The TMS (time table) associates time information with the position of image data.

  The IRAP start position and the # 1 start position are also used as the same value on the time table. Necessary values are the start position and end position of slice # 1. The IRAP start position is regarded as the # 1 start position, and the # 2 start position is regarded as the # 1 end position.

  In the case of fast-forward playback, timetable entries are read sequentially. From the IRAP start position and # 2 start position of each GOP, the data of the first slice segment of the I picture of this GOP is read. Then, this data is decoded and displayed on the screen. The IRAP start position is the start position of slice segment # 1. The # 2 start position is the end position of the slice segment # 1.

  By repeating this, fast-forward playback can be performed. In this case, the read data amount is ¼ compared with the case where all I pictures are displayed. For this reason, the image update interval during fast-forward playback can be shortened.

  In this example, the display is updated on a part of the screen. That is, only the image at the position of the first slice segment is displayed. In the case of vertical and horizontal division (field shape), for example, the upper left quarter of the image. In the case of vertical division (eye shape), it is the upper quarter region of the image. Other parts remain unupdated. Alternatively, other parts are not displayed.

  When the user performs a fast-forward operation, it is sufficient that the target scene can be determined. Therefore, it is often better to consider the operability when the update interval is shorter than when the entire screen is visible.

  Also, the first slice segment and the second slice segment are displayed in the same manner. In this way, half of the screen can be displayed and fast forward can be performed.

  Furthermore, when fast-forward playback is performed at high speed, the displayed GOP can be thinned and played back. That is, one GOP is skipped or two are skipped.

  In many fast-forward playback operations, the fast-forward speed can be adjusted by pressing the fast-forward button on the remote controller multiple times. When used in combination with conventional fast-forwarding that displays the entire I picture, the display method can be selected by the number of times the remote control button is pressed.

  For example, when the button is pressed once, the entire I picture of all GOPs is displayed and fast-forward playback is performed. When the button is pressed twice, only some slice segments of the I picture of all GOPs are displayed and fast-forward playback is performed. When the button is pressed three times, one GOP is skipped and only some slice segments of the I picture are displayed and fast-forward playback is performed. When the button is pressed four times, two GOPs are skipped and only some slice segments of the I picture are displayed and fast-forward playback is performed.

  This method cannot be used if the broadcast stream does not have multiple slice segments. However, re-compression or format conversion is often performed when recording a broadcast. At that time, it is also possible to reconstruct the HEVC video stream having a plurality of slice segments. Even when recording externally input video other than broadcast, special reproduction by this method can be performed by creating compressed data as a HEVC video stream having a plurality of slice segments.

<Modification 1>
In the description so far, only a part of the image may be updated during special playback such as fast-forwarding. However, it may be easier to find the target scene if the entire image is updated. Therefore, a method of updating the entire image while considering the data to be read out as a part of the slice segment is considered.

  FIG. 21 is an explanatory diagram of fast-forward playback. In this example, position information of all slices in the time table (TMS) is used.

  In the case of fast-forward playback, timetable entries are read sequentially. From the IRAP start position and # 2 start position of the first GOP, the data of the first slice segment of the I picture of this GOP is read. The IRAP start position is the start position of slice segment # 1. The # 2 start position is the end position of the slice segment # 1. This data is decoded and displayed at the position of slice segment # 1 on the screen. In FIG. 21, the position of slice segment # 1 on the screen is at the upper left of the screen.

  From the # 2 start position and # 3 start position of the next GOP, the data of the second slice segment of the I picture of this GOP is read. The # 2 start position is the start position of slice segment # 2. The # 3 start position is the end position of the slice segment # 2. This data is decoded and displayed at the position of slice segment # 2 on the screen.

  At this time, the decoding of slice segment # 2 requires the parameter set (PS) data of this picture. The parameter set (PS) is arranged at the top of the picture. That is, the parameter set (PS) is arranged before slice segment # 1. Therefore, when the parameter set (PS) is read at the same time as the slice segment # 1, it can be read at a time. However, when reading the parameter set (PS) and slice segment # 2, two readings occur.

  In the optical disk read operation, it takes time to change the read position. Therefore, the following three methods can be considered as the reading operation at this time.

  The first method performs two reading operations of the parameter set (PS) and slice segment # 2. The second method reads the parameter set (PS), slice segment # 1, and slice segment # 2 at a time. The third method uses a parameter set (PS) and data of slice segment # 2. Then, when creating the time table, a copy of the parameter set (PS) data necessary for decoding the I picture is stored in the time table.

  In this description, the third method is described. That is, it is assumed that a copy of parameter set (PS) data is stored in the time table. In this case, the parameter set (PS) data and the slice segment # 2 data stored in the time table are input to the decoder for decoding.

  In the next GOP, similarly, the data of slice segment # 3 is taken out, and the image at the position of slice segment # 3 on the screen is updated. In FIG. 21, the position of slice segment # 3 on the screen is the lower left of the screen.

  In this way, by shifting the slice segment to be displayed each time the image is updated, the entire screen is updated by updating the image several times even if it is a partial update of the image by one update. Can do. Depending on the position of the screen, images at different times are displayed. Images at different times are displayed for each slice segment. However, it is easier to grasp the scene than when displaying a partial image. That is, special reproduction with better operability can be realized.

  In this description, the image is updated in one slice segment. However, it is possible to update images of a plurality of slice segments with one GOP. For example, half of the image can be alternately updated.

  In the above description, images are updated in the order of slice # 1, slice # 2, slice # 3, and slice # 4 in the order of slice numbers. The images need not be updated in the order of slice numbers. It can be in any order during playback.

  FIG. 22 shows an example of the update order of images. FIG. 22 (1) is an example in which images are updated in the order of slice # 1, slice # 2, slice # 3, and slice # 4. FIG. 22 (2) is an example in which images are updated in the order of slice # 1, slice # 2, slice # 4, and slice # 3. In FIG. 22 (2), the image is updated clockwise. FIG. 22 (3) is an example in which images are updated in the order of slice # 1, slice # 4, slice # 3, and slice # 2. FIG. 22 (4) is an example in which images are updated in the order of slice # 1, slice # 3, slice # 4, and slice # 2. In FIG. 22 (4), the image is updated counterclockwise.

  When the slice segment to be displayed at the time of playback can be selected, the order in which images are updated can be reversed between fast forward playback and rewind playback. For example, in the case of fast-forwarding, the rotation is clockwise in FIG. 22 (2), and in the case of rewinding, the rotation is counterclockwise in FIG. In this way, by reversing the image update order between fast-forward and rewind, even when fast-forward and rewind operations are repeated, the current state can be easily grasped and operability is improved. improves.

  FIG. 23 is an example in the case of four divisions (eye shape) in the vertical direction. Also in this case, by reversing the image update order, it becomes easy to grasp fast forward and rewind. For example, the fast forward is updated from slice # 1 to slice 4 (FIG. 23 (1)). Then, rewinding is updated from slice # 4 toward slice 1 (FIG. 23 (2)).

<Modification 2>
In the above description, read position information for all slice segments is stored in the time table. In order to reduce the size of the time table, a method of recording only the read position information of some slice segments is also conceivable.

  FIG. 24 shows a time table (TMS) storing read position information of one slice segment.

  In this example, information on the read position of the slice segment corresponding to each time in the time table is stored. For example, the read position information of slice segment # 1 is stored in the rows corresponding to GOP (01) and GOP (05). In FIG. 23, for example, information on GOP (01) is described in the first row of the time table. Further, information on GOP (05) is described in the fifth row of the time table.

  When recording a stream, the GOPs to be recorded are sequentially counted. Then, the remainder (remainder) obtained by dividing the GOP count value by the number of slice segments in one image is taken. In FIG. 24, the number of slice segments in one image is four. A value obtained by adding 1 to this value (remainder) is set as the value of the slice segment. Then, the position of the slice segment is recorded in the time table.

  In this example, information is described in the slice start position and the slice end position at each time. The order of the slices is the order of slice # 1, slice # 2, slice # 3, and slice # 4. Although not shown in FIG. 24, the IRAP start position and IRAP end position are also recorded. This is for compatibility with other special playback or conventional display of the entire IRAP image.

  When special playback such as fast-forwarding is performed using this time table, the entire screen is updated when fast-forwarding is performed without skipping the GOP. For example, when the GOP is skipped one by one and reproduced, slice # 1 and slice # 3 are updated. Further, for example, when the GOP is skipped and reproduced by three, slice # 1 is updated.

  In order to update the entire image regardless of the fast-forwarding double speed, a slice segment can be selected using a random number or a pseudo-random number when the read position information is stored in the time table. In this case, the slice segment updated on the screen is irregular. However, the entire image can be updated regardless of the multiple of fast-forwarding.

  Examples of the pseudo random number generating means include a linear feedback shift register using an M series.

  In the pseudo-random number generation using the M series, it is possible to specify the number of values and the number of times, and to generate a data string having a uniform appearance probability of each value and having no periodicity between the specified number of times. The number of values is, for example, four values from 1 to 4. The number of times is, for example, 1000 times.

  For example, if a time table is created using these pseudo-random number generation means, slice segments can be selected so that the same pattern is not repeated in the time table. As a result, even when the fast-forward double speed is changed, it is possible to prevent only a specific slice segment from being updated. It is not always necessary to eliminate repetition of the same pattern in the entire time table. If the same pattern is repeated with a sufficiently long period, there is no practical problem. A sufficiently long cycle is, for example, about 1000 rows.

  The number of pseudo random numbers is selected so that periodicity does not occur in the entire time table. However, the calculation amount increases when the periodicity of the pseudo random number is set to be long. And even when it is given in advance as a random number data string, the amount of data increases.

  In the fast forward operation or the rewind operation, fast forward or rewind with a small skip amount is mainly used. For example, GOP skipping is not performed, or one to several tens of GOPs are skipped. For this reason, the period of the pseudo random number can be set short.

  As an example, a time table having a periodicity of about 1000 rows is given. When a 999 GOP is skipped, only a part of the image is updated. However, fast-forward playback and rewind playback when a 999 GOP is skipped are rarely used. Further, even if the skip of 1000 GOPs is adopted instead of skipping the 999 GOPs, the fast-forwarding double speed as seen from the user is hardly changed. For this reason, it can avoid easily.

  If the average length of GOP is 0.5 seconds, a 2-hour video is composed of 14400 GOPs. Then, the periodicity of 1000 rows occurs about 15 times in a 2-hour video. The periodicity becomes a problem when fast-forwarding in which a 2-hour video is played back in 15 frames. Normally, such high-speed fast-forward operation is not performed.

  In this case, the size of the random number table when the pseudo random number sequence is given to the control program as a calculated random number table in advance is estimated. A random number sequence having 4 values and 1000 times is represented by 2 bits. In this case, the total size is 250 bytes. The length of the periodicity of the pseudo random number can be set according to the relationship between operability and device mounting.

<Modification 3>
In the examples so far, when the display position of the slice segment is changed, the read position and end position of the slice segment are stored in the time table. Further, when necessary, the parameter set (PS) is stored in the time table. Therefore, the time table data becomes large. In addition, two readings of the parameter set (PS) and the slice segment occur.

  By switching the order of the slice segments when recording the video stream, these extra operations can be avoided and efficient special reproduction can be performed.

  FIG. 25 is an explanatory diagram of fast-forward playback. Similarly to FIG. 24, GOP (01) to GOP (06) are stream data. Originally, it is a series of data files, but for ease of explanation, the lines are changed in units of GOPs.

  In the stream of FIG. 25, the storage order of slice segments is changed for each IRAP picture at the head of each GOP. Here, the slice segment number indicates the display position on the screen. The number of the slice segment displayed at this display position is shown as the number on the stream data.

  In FIG. 25, as an example, data is arranged as follows. In GOP (01), slice segment # 1 is arranged at the head of the IRAP picture. In GOP (02), slice segment # 2 is arranged at the head of the IRAP picture. In GOP (03), slice segment # 3 is arranged at the head of the IRAP picture. In GOP (04), slice segment # 4 is arranged at the head of the IRAP picture. In GOP (05), slice segment # 1 is arranged at the head of the IRAP picture. In GOP (06), slice segment # 2 is arranged at the head of the IRAP picture.

  In the above description, the slice segment has been described from the viewpoint of data division. However, the management including the display position is realized using tiles in addition to the slice segment of the HEVC standard. Therefore, when the storage position of the slice segment in the time table is exchanged, it is necessary to modify the slice header information and tile information of each slice segment as necessary to ensure consistency. Tile information is included in the parameter set.

  The procedure at the time of fast-forward playback is the same as the description using FIG. The explanation using FIG. 20 is a case where only the slice segment # 1 is reproduced. However, in the example of FIG. 25, the slice segment placed at the head is replaced. For this reason, the slice segment in which the image is updated changes, and the entire screen is updated.

  When the slice segment arranged at the head in a simple order is selected, only the image of a part of the slice segment is updated by the fast-forward double speed. Therefore, the slice segment to be arranged at the head can be determined by using the above-described random number or pseudo-random number.

  As described above, by switching the order of the slice segments at the time of recording, it is possible to improve the operability of scene search by special reproduction such as fast forward and rewind.

  So far, the description has been given by taking four divisions as an example of the slice segment. This is because it is adopted in 4K / 8K broadcasting in Japan.

  Actually, recompression or format conversion may be performed during broadcast recording or recording onto an optical disc. In this case, not only the four divisions but also the slice segment division format can be freely changed.

  For example, one image can be divided into 3 × 3 9 divisions. In this case, it is also possible to update only the central segment by special reproduction. This is because the central segment is likely to contain important information. Further, it can be simply divided into two slice segments. Special playback can also be performed by alternately updating two segments.

  Further, although the example of the optical disk has been described, the same effect can be obtained with other storage devices such as a hard disk drive (HDD) or an SSD (solid state drive).

  Even when there is a limitation in the transmission band such as a network and the delay of data transfer is large, it is possible to suppress the data transfer amount during special reproduction and improve operability.

  In addition, although embodiment of this invention was described as mentioned above, this invention is not limited to these embodiment.

DESCRIPTION OF SYMBOLS 100 Video recording / reproducing apparatus, 11 Tuner / demodulation part, 12 External input part, 13 Network part, 21 Demultiplexing part, 31 Audio decoding part, 32 Video decoding part, 33 Subtitle decoding / rendering part, 34 Data broadcasting and EPG processing Unit, 41 recording / playback control unit, 51 built-in recording device, 52 optical disc drive, 53 optical disc, Af management file, Hm head data, Ba broadcast wave, Sm multiplexed stream, Se elementary stream, Ds audio data, Di ₀ , Di ₁ video data, Dc caption data, Db data broadcast data, Dd display device, Es sound device, Ei external device, Ne network, Ha horizontal axis, Va ₁ , Va ₂ vertical axis, P1, P2, P3 position, Pad padding, SI (MPT) control packet , TP TLV packet, TM, TMS timetable, V video packet, V (RAP) video packet.

Claims (19)

A tuner / demodulator that receives broadcast waves,
A demultiplexing unit that demultiplexes the multiplexed stream of the broadcast wave;
A decoding unit for decompressing the compressed data that has been demultiplexed from the multiplexed stream;
A recording / playback control unit for collecting video data or control information for recording from the data decompressed by the decoding unit;
A video recording / reproducing apparatus, wherein when video data included in the broadcast wave using the MMT format is recorded, packet data is recorded as combined video data.   The video recording / reproducing according to claim 1, wherein a time table including a time at which the collected video data is reproduced and a position at which the video data at the time is recorded is created and recorded together with the collected video data. apparatus. A tuner / demodulator that receives broadcast waves,
A demultiplexing unit that demultiplexes the multiplexed stream of the broadcast wave;
A decoding unit for decompressing the compressed data that has been demultiplexed from the multiplexed stream;
A recording / playback control unit for collecting video data or control information for recording from the data decompressed by the decoding unit;
A video recording / reproducing apparatus for recording the video data as a BMFF format file when recording the video data included in the broadcast wave using the MMT format.   4. The video recording / playback according to claim 3, wherein when the video data is recorded on the optical disc, the management data portion and the stream data portion of the BMFF format file are recorded at different positions on the optical disc. apparatus.   The video recording / reproducing apparatus according to claim 2, wherein asset information is recorded in the time table.   The video recording / reproducing apparatus according to claim 2, wherein HDR metadata is recorded in the time table.   The video recording / reproducing apparatus according to claim 2, wherein a data size or an end position of an IRAP picture that is first decoded in a GOP is recorded in the time table.   The video recording / reproducing apparatus according to claim 2, wherein positions of a plurality of images in the GOP are recorded in the time table. The video data has a structure of slice segments constituting an image,
The video recording / reproducing apparatus according to claim 2, wherein a position of each slice segment is recorded in the time table. When combining or recording the packet data, the video data is separated,
2. The video recording / reproducing apparatus according to claim 1, wherein a head of the divided video data is placed at a block separation position used when accessing the video data.   The video recording / reproducing apparatus according to claim 10, wherein a unit for dividing the video data is a GOP unit.   The video recording / reproducing apparatus according to claim 10, wherein a unit for dividing the video data is a unit of a picture.   The video recording / reproducing apparatus according to claim 10, wherein the position at which the video data is divided is a head of an IRAP picture, a head of an I picture or a head of a P picture.   The video recording / reproducing apparatus according to claim 10, wherein a unit for dividing the video data is a unit of a slice segment.   15. The video recording / reproducing apparatus according to claim 14, wherein when the video data is recorded, information on a reading position of the video data is recorded as management data in units of slice segments of an I picture.   The video recording / reproducing apparatus according to claim 15, wherein the information on the reading position is information on a reading position of the slice segment.   The video recording / reproducing apparatus according to claim 15 or 16, wherein when the video data is recorded, the order of recording the slice segments is changed for each GOP.   The video recording / reproducing apparatus according to claim 15, wherein a part of the slice segment is displayed for each image when the video data is reproduced.   The video recording / reproducing apparatus according to claim 18, wherein a position of the partial slice segment displayed for each image is changed.

Images