JP2009194758A - Video recording apparatus, video reproducing apparatus, video recording/reproducing system, method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique for recording and transmitting video content represented by a plurality of video streams by means of a multiplexed stream allowed to contain no more than one video stream and a null stream which does not specify data to be carried. <P>SOLUTION: A video recording apparatus 100 includes an image processing part 14 for imaging objects to produce a plurality of video signals, an encoding part 15 for encoding one of the plurality of video signals into a main TS and encoding the others into respective sub-TSs, a TS nulling part 11 for converting the encoded sub-TSs into a nulled TS having specific data identifying a null TS to masquerade as a null TS, and a TS multiplexing part 12 for multiplexing the main TS and the nulled TS into a multiplexed stream for recording on a recording medium 300. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a video recording device, a video playback device, a video recording / playback system, a method, and a program. In particular, the video recording device, video playback device, video recording / playback system, method, and program are allowed to include at most one video stream and a null stream in which video data to be held is not defined. The present invention relates to a technique for recording and reproducing a plurality of video streams using a multiplexed stream.

  The spread of high-definition (HD) video that began with television has spread to digital video cameras, and digital video cameras capable of HD photography are also showing signs of widespread use. Digital still cameras are also expected to have HD movie shooting functions.

  In the conventional digital video camera, H.264 is used as a video encoding method mainly for recording HD video. H.264 and MPEG2-VIDEO are used, and MPEG2-TS (Moving Picture Expert Group-Transport Stream) is often used as a multiplexing system, and each system has been simplified to realize a low-priced device. The rules are adopted.

The frame rate for shooting and playback by a video camera is generally 30 Hz, but for a professional camera, shooting is performed at a high frame rate such as 120 Hz (high-speed shooting), and playback is performed at a frame rate of 30 Hz (normal playback). Some of them realize high-quality slow playback. Also in consumer video cameras, a camera capable of high-quality slow playback by high-speed shooting and normal playback is expected.
ISO / IEC13818-1 (MPEG2-TS standard)

  However, according to the simplified operation rule, it is not assumed that high-speed shooting and high-quality slow reproduction by normal reproduction are performed.

  The simplified operational rules simplify MPEG2-TS and only allow at most one video stream to be included in the multiplexed stream. Therefore, in order to record and transmit a frame image with a frame rate exceeding 30 Hz obtained by high-speed shooting, for example, a frame image corresponding to a frame rate of 30 Hz is represented by the first video stream, and the other frame images are represented by the second However, it is not possible to multiplex it into a multiplexed stream represented by a video stream.

  As described above, the multiplexed stream according to the operation rule that is allowed to include at most one video stream is not suitable for recording and transmitting a video stream for high-quality slow reproduction, but also includes a plurality of different scenes. There is a limitation that it is impossible to record and transmit content represented by a plurality of video streams such as a multi-angle video representing a plurality of videos viewed from the above viewpoint and a stereoscopic video by binocular vision.

  The present invention has been made in view of such problems, and uses a plurality of multiplexed streams that are allowed to include at most one video stream and a null stream for which data to be held is not defined. It is an object of the present invention to provide a technique for recording and transmitting video content represented by a video stream.

  In order to solve the above problems, a video recording apparatus according to the present invention includes an imaging unit that obtains a plurality of video signals by photographing a target, and encodes one of the plurality of video signals into a main video stream. An encoding unit that encodes each of the other into a sub-video stream, a stream nulling unit that disguises the one or more sub-video streams as a null stream, and the one or more sub-devices that are disguised as the null stream A stream multiplexing unit that multiplexes the video stream into the multiplexed stream and outputs the multiplexed stream.

  Here, disguising the sub-picture stream as the null stream means converting the sub-picture stream into a stream having specific data for identifying the null stream.

  According to this configuration, since the one or more sub-video streams are disguised as a null stream, even if the multiplexed stream is allowed to include at most one main video stream, the multiplexed stream is nullified. The one or more sub-picture streams can be multiplexed as a stream. As a result, it is possible to record and transmit video content represented by a plurality of video streams using a multiplexed stream that is only allowed to include at most one video stream.

  Further, the null stream and each sub-video stream may be composed of a plurality of packets including a header and a payload.

  In this case, the stream nulling unit has the one or more sub-video streams, specific data for identifying the null stream in a header, and a payload of the packet of the one or more sub-video streams in a payload. It may be disguised as a null stream by converting it into a nullified stream composed of a plurality of packets having the following data.

  According to this configuration, the data originally included in the header of each packet of the sub-video stream is discarded, and the payload data is retained as it is in the null-stream payload. It is possible to disguise the null stream as much as the original data amount of the stream.

  Further, the stream nulling unit has the one or more sub-video streams having specific data for identifying the null stream in a header, and each packet of the one or more sub-video streams is included in a payload as a payload. It may be disguised as a null stream by converting it into a null stream composed of a plurality of packets having individual data obtained by dividing the data into sizes.

  According to this configuration, the sub-picture stream can be disguised as a null stream including header data. When specific data for synchronization is originally included in the head of each packet header of the sub-picture stream, the specific data is held at the head of the payload of the null stream packet. In the stream, it is useful for distinguishing a null stream packet in which the sub-picture stream is disguised from an original null stream packet that does not hold valid video data.

  The stream nulling unit has the one or more sub-video streams having specific data for identifying the null stream in a header, and the entire sub-video stream is divided into payload sizes in a payload. It may be disguised as a null stream by converting it into a null stream composed of a plurality of packets having individual data obtained in this way.

  According to this configuration, unlike the case where the sub-video stream is divided for each packet, there is no empty area due to the fraction of the division in the payload of the null stream. Therefore, not only can the sub-video stream be disguised as a smaller null stream, but when the sub-video stream is restored from the null stream payload, it is not necessary to exclude invalid data in the empty area, thus simplifying the processing. May be possible.

  The stream nulling unit may further set a value for distinguishing each sub-picture stream in the header of each packet of the nullified stream.

  According to this configuration, even from a multiplexed stream in which a plurality of sub-picture streams disguised as null streams are multiplexed, it is possible to restore individual sub-picture streams using values that distinguish sub-picture streams.

  The present invention can be realized not only as such a video recording apparatus but also as a video playback apparatus and video recording / playback system. Further, it can be realized as a video recording method, a video reproduction method, and a program.

  As described above, according to the video stream multiplexing apparatus of the present invention, the sub-picture stream is disguised as a null stream, so that the sub-picture can be converted into a multiplexed stream that can only contain at most one main picture stream. The stream can be multiplexed as a null stream. As a result, it is possible to record and transmit video content represented by a plurality of video streams using a multiplexed stream that is only allowed to include at most one video stream.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

(Embodiment 1)
FIG. 1 is a functional block diagram showing an example of the configuration of a video stream processing system 1 according to Embodiment 1 of the present invention.

  The video stream processing system 1 includes one or more sub-video streams in a multiplexed stream that is allowed to include at most one main video stream and a null stream in which video data to be held is not defined. This is a system that multiplexes and separates the main video stream and each sub-video stream from the multiplexed stream.

  In the first embodiment, the main video stream and the sub-video stream are H.264 according to the simplified operation rules described in the background section. It is assumed that the TS is based on H.264 encoding, and that the multiplexed stream is a multiplexed stream conforming to MPEG2-TS. As described above, a multiplexed stream based on simplified operation regulations is allowed to include at most one video TS and a null TS for which video data to be held is not defined.

  Normally, the null TS is inserted into the multiplexed stream for bit rate adjustment and discarded without being processed in the video playback device.

  In FIG. 1, a video stream processing system 1 includes a TS multiplexer 10 and a TS separator 20. Further shown in FIG. 1 is a storage or transmission line 30.

  In this specification, the terms “main” and “sub” are used only to distinguish a plurality of video streams and a plurality of TSs, and superiority and inferiority such as a frame rate and a resolution between a plurality of video streams and between a plurality of TSs. Does not mean.

  The TS multiplexer 10 is an apparatus that multiplexes a main TS and one or more sub-TSs into a multiplexed stream, and includes a TS nulling unit 11 and a TS multiplexing unit 12.

The TS nulling unit 11 disguises one or more sub-TSs as a null TS.
Hereinafter, the null TS in which the sub-TS is disguised is called a nullified TS, and is distinguished from an original null TS that does not hold valid video data. Impersonating one or more sub-TSs into a null TS is synonymous with converting one or more sub-TSs into a null TS.

  The TS multiplexing unit 12 multiplexes the null TS and the main TS converted by the TS nulling unit 11 into the multiplexed stream.

  The multiplexed stream obtained by the TS multiplexer 10 is stored or transmitted in the storage or transmission path 30 and acquired by the TS separator 20.

  The TS separation device 20 is a device that separates the main TS and one or more sub-TSs from the multiplexed stream, and includes a TS separation unit 21 and a TS restoration unit 22.

The TS separator 21 separates the main TS and the null TS from the multiplexed stream.
The TS restoration unit 22 restores the sub-TS from the nullified TS separated by the TS separation unit 21.

  Note that the TS multiplexer 10 and the TS separator 20 are individually or as a unit, and as a software function, a CPU (Central Processing Unit) (not shown) executes a program recorded in a ROM (Read Only Unit). It may be realized, or may be realized as hardware such as an integrated circuit device.

  Next, operations of the TS multiplexer 10 and the TS separator 20 configured as described above will be described with reference to flowcharts.

  FIGS. 2A and 2B are flowcharts showing examples of operations of the TS multiplexer 10 and the TS separator 20, respectively.

First, TS multiplexing processing in the TS multiplexer 10 will be described.
The TS multiplexer 10 acquires the main TS and the sub TS (S11, S12).

  The TS nulling unit 11 converts each acquired sub TS into a null TS (S13). This conversion is performed, for example, by rewriting the header of the sub TS packet with specific data for identifying the null TS. An example of conversion from the sub-TS to the nulled TS will be specifically described.

  FIG. 3 is a diagram illustrating a format of an MPEG2-TS packet. As shown in FIG. 3, an MPEG2-TS packet is composed of a header and a payload. The header holds control data in a plurality of fields, and the payload holds PES (Packetized Elementary Stream) that is video data.

  In the null TS, the value 0x1FFF is set in the PID field of the header. This value 0x1FFF is an example of specific data for identifying the null stream of the present invention.

  The TS nulling unit 11 rewrites the PID field of the header of each packet of the sub-TS with the value 0x1FFF. As a result, the sub-TS is converted to a nullified TS composed of a plurality of packets having the value 0x1FFF in the PID field of the header and the payload having the PES data of the payload of the original sub-TS. As a result of the sub TS being disguised as a null TS, it can be multiplexed into the multiplexed stream.

  When there are a plurality of sub-TSs, the TS nulling unit 11 sets a value (hereinafter referred to as a TS identifier) for distinguishing each sub-TS in, for example, a continuity counter field of the nulled TS header.

  The TS identifier is preferably a value that can be expressed with a smaller number of bits than the number of bits for expressing the PID value. As the TS identifier, for example, a hash value corresponding to the PID value originally set in the PID field of the header of the sub-TS may be used.

  The TS identifier may be held in a fixed or changeable manner in association with the PID value by, for example, a conversion table as shown in FIG. The place where the conversion table is provided is not limited. The conversion table is provided, for example, inside the TS multiplexer 10 or outside accessible via the storage or transmission path 30.

  The TS identifier can be calculated whenever necessary using a known hash function instead of being held in the conversion table.

  The TS identifier is not necessarily set in the continuity counter field. A transport byte indicator field, a transport priority stream field, a transport stream field, a transport stream field, a transport stream field, a transport stream field, a transport stream field, a transport stream field, a transport stream field, a transport stream field, and a transport stream field. It may be set across one or more of the fields.

  The TS multiplexing unit 12 multiplexes the null TS and the main TS converted by the TS nulling unit 11 into the multiplexed stream (S14).

Next, the TS separation process in the TS separation device 20 will be described.
The TS separator 20 acquires the multiplexed stream obtained by the TS multiplexer 10 via the storage or transmission path 30 (S21).

  The TS separation unit 21 separates the main TS from the acquired multiplexed stream (S22) and separates the null TS (S23). Nulled TS is separated from the acquired multiplexed stream, in the packet having the value 0x1FFF in the header PID field, all data in the payload is uniform (for example, all bits are set to 0 or 1). This is done by extracting packets that are not.

  Since the original null TS payload does not stipulate the video data to be held, all the bits are set to 0 or 1, so the packet other than that is the original sub-TS PES data in the payload. It can be determined that the packet is a null TS packet. Note that a packet determined to be an original null TS is discarded as in the conventional case.

  The TS restoration unit 22 restores the sub-TS from the nullified TS separated by the TS separation unit 21 (S24). This restoration is performed by setting a value different from the value of the PID field of the header of the main TS in the PID field of the header of the separated nulled TS.

  When there are a plurality of sub-TSs and a TS identifier is set in the continuity counter field or the like, a different PID value is set in the PID field of the header of the separated nulled TS according to the set TS identifier.

  When there is a conversion table that defines the correspondence between the PID value and the TS identifier, the PID value corresponding to the TS identifier is referenced from the conversion table and set in the PID field of the header of the separated null TS.

  As described above, according to the video stream processing system 1, the TS multiplexer 10 converts a multiplexed stream that is allowed to include at most one main TS into one or more sub-TSs into a null TS. Since the TS separation device 20 restores the sub-TS from the multiplexed stream including one or more sub-TSs disguised as a null TS, it is allowed to include at most one video stream. It is possible to record and transmit a plurality of video streams with no multiplexed stream.

  According to this configuration, a multiplexed stream that is only allowed to include at most one video stream can be used for a very wide range of applications.

  For example, as described in the section of the problem, among frame images having a frame rate exceeding 30 Hz obtained by high-speed shooting, for example, a main TS representing a frame image corresponding to a frame rate of 30 Hz and other frame images are represented. The sub-TS can be recorded and transmitted by being multiplexed on the multiplexed stream.

  If all the frame images represented by the main TS and the sub-TS separated from such a multiplexed stream are reproduced at a frame rate of 30 Hz by a video reproduction device (not shown), high-quality slow reproduction is realized. Also, only the frame image represented by the main TS can be reproduced at the normal speed.

  In addition, when such a multiplexed stream is played back by a conventional general video playback device, the sub-TS disguised as a null TS is discarded, and only the frame image represented by the main TS is played back at normal speed. The That is, this multiplexed stream is highly compatible with existing systems.

  In addition, a plurality of streams such as a multi-angle video representing a plurality of videos obtained by viewing the same scene from a plurality of different viewpoints and a stereoscopic video by binocular vision using a multiplexed stream that is allowed to include at most one video stream. It is possible to record and transmit the content represented by the video stream.

  Furthermore, recording and transmission of ultra-high-resolution video content by hierarchically encoding video into a plurality of video streams using a multiplexed stream that is allowed to contain at most one video stream. Is possible.

(Modification of Embodiment 1)
In the first embodiment, it has been described that the sub-TS is converted to the null TS by rewriting the header of the sub-TS. However, the conversion from the sub TS to the null TS is not limited to the above example.

Hereinafter, another example of conversion from the sub-TS to the nulled TS will be described.
FIG. 4 is a diagram for explaining a modification of the conversion from the sub TS to the null TS.

  In the TS multiplexer 10, as shown in FIG. 4, the TS nulling unit 11 stores individual data obtained by dividing each packet of the sub-TS into the payload size in the payload of the null TS. The sub-TS may be converted into a null TS.

  In FIG. 4, the header and payload sizes are drawn in a random manner for the sake of clarity. However, when the sub-TS and the nulled TS are MPEG2-TS, the header and payload sizes are 4 bytes each. And 184 bytes.

  Accordingly, the 4-byte data in the header of the sub-TS packet P11 and the first 180 bytes of the PES data in the payload are stored in the payload of the packet P21 in the null TS.

  Further, the remaining 4 bytes of the PES data in the payload of the packet P11 of the sub-TS are stored in the payload of the packet P22 next to the nullified TS. A hatched portion shown in the payload of the packet P22 of the nullified TS represents an empty area (invalid data) generated by a fraction of the sub TS packet division.

The data of the sub-TS packet P12 is stored after the packet P23 of the null TS.
When there are a plurality of sub-TSs, the TS nulling unit 11 performs the above-described conversion, and, as described in the first embodiment, for example, the sub-TS in each continuity counter field of the nullified TS header. You may set the TS identifier which distinguishes TS.

  When the multiplexed stream obtained by such conversion is acquired in the TS separation device 20, the TS separation unit 21 reads the payload from the acquired multiplexed stream among the packets having the value 0x1FFF in the header PID field. The null TS is separated from the multiplexed stream by extracting the packet having the value 0x47 whose first data is fixedly set in the sync byte field of the header and the next packet.

  The TS restoration unit 22 restores the sub-TS by connecting the data set in the payload of the packet extracted by the TS separation device 20 except the invalid data in the empty area.

FIG. 5 is a diagram for explaining another modification of the conversion from the sub-TS to the nulled TS.
In the TS multiplexer 10, the TS nulling unit 11 rewrites the payload of the null TS with individual data obtained by dividing the entire sub-TS into the size of the payload, as shown in FIG. The sub TS may be converted into a null TS.

  When there are a plurality of sub-TSs, the TS nulling unit 11 performs the above-described conversion, and, as described in Embodiment 1, for example, in each continuity counter field of the nullified TS header, each sub-TS A TS identifier for distinguishing between them may be set.

  In the TS separator 20, the TS separator 21 has a uniform value (for example, all bits are 0 or 1) in the packet having the value 0x1FFF in the PID field of the header from the acquired multiplexed stream. The nulled TS is separated from the multiplexed stream by extracting non-packeted packets.

  The TS restoration unit 22 restores the sub-TS by connecting the data set in the payload of the packet extracted by the TS separation device 20 for each packet size.

  According to this configuration, unlike the case where the sub-TS is divided for each packet, no empty area is generated in the nullified TS due to the fraction of the division. For this reason, the sub-TS can be converted into a smaller null TS, and, when the sub-TS is restored from the null TS, there is no need to exclude invalid data in an empty area, so the processing can be simplified (integrated In a hardware implementation such as a circuit device, there is a possibility that a selection circuit for excluding invalid data can be omitted).

  Next, a video recording / playback system having a high image quality slow playback function configured using the video stream processing system described above will be described in detail.

(Embodiment 2)
FIG. 6A is a functional block diagram showing an example of the configuration of the video recording / playback system 400 according to Embodiment 2 of the present invention.

  The video recording / playback system 400 is a video recording / playback system that records a video stream obtained by high-speed shooting and performs high-quality slow playback by normal playback of the recorded video stream. A device 100, a video playback device 200, and a display device 402 are included. The video stream is recorded on the recording medium 300.

  FIG. 6B is a perspective view showing an example of the appearance of the video recording / playback system 400 when implemented as a video camera. The video recording device 100, the video playback device 200, and the recording medium 300 are arranged in a video camera casing.

FIG. 7 is a functional block diagram showing an example of the configuration of the video recording apparatus 100.
The video recording apparatus 100 represents a frame image corresponding to a frame rate of 30 Hz as a main video stream among frame images obtained by photographing an object at a high frame rate exceeding 30 Hz with the imaging optical system 401, and the other The frame images are represented by first and second sub-picture streams, multiplexed into a multiplexed stream, and recorded on the recording medium 300.

  Also in the second embodiment, the main video stream and the sub-video stream are H.264 according to the simplified operation rules described in the background section. It is assumed that the TS is based on H.264 encoding, and that the multiplexed stream is a multiplexed stream conforming to MPEG2-TS.

  The video recording apparatus 100 includes an imaging processing unit 14, an encoding unit 15, a TS nulling unit 11, and a TS multiplexing unit 12.

  Here, the imaging processing unit 14, the encoding unit 15, the TS nulling unit 11, and the TS multiplexing unit 12 are examples of the imaging unit, the encoding unit, the stream nulling unit, and the stream multiplexing unit of the present invention, respectively. . The main TS is an example of the main video stream, and the first sub-TS and the second sub-TS are examples of the sub-video stream.

  For example, the imaging processing unit 14 acquires a frame image from the imaging optical system 401 at a frame rate of 120 Hz, a main video signal representing a frame image corresponding to a frame rate of 30 Hz, and a first sub-video signal representing another frame image. In addition, the second sub-video signal is output.

  FIG. 8 is represented by a main video signal, a first sub-video signal, and a second sub-video signal output from the imaging processing unit 14 among frame images obtained from the imaging optical system 401 at a frame rate of 120 Hz. It is a figure explaining the frame image.

  The main video signal represents an image of one main video frame every four frames among frames obtained from the imaging optical system 401. The first sub-video signal represents an image of the first sub-video frame that is temporally intermediate in the main video frame. The second sub-video signal represents an image of the remaining second sub-video frame. In this example, the main video signal and the first sub video signal represent a video with a frame rate of 30 Hz, and the second sub video signal represents a video with a frame rate of 60 Hz.

  The encoding unit 15 encodes the main video signal, the first sub video signal, and the second sub video signal into the main TS, the first sub TS, and the second sub TS, respectively.

The TS nulling unit 11 converts the first sub-TS and the second sub-TS into a null TS.
The TS multiplexing unit 12 multiplexes the main TS and the null TS in the multiplexed stream, and records the multiplexed stream on the recording medium 300.

FIG. 9 is a functional block diagram illustrating an example of the configuration of the video playback device 200.
The video playback device 200 plays back the video represented by the main video stream, the first sub-video stream, and the second sub-video stream multiplexed on the multiplexed stream recorded on the recording medium 300.

  The video reproduction device 200 includes a TS separation unit 21, a TS restoration unit 22, a decoding unit 23, and a video multiplexing unit 24.

  Here, the TS separation unit 21, the TS restoration unit 22, and the decoding unit 23 are examples of the stream separation unit, the stream restoration unit, and the decoding unit of the present invention, respectively.

  The TS separator 21 acquires a multiplexed stream from the recording medium 300, and separates the main TS and the null TS from the acquired multiplexed stream.

  The TS restoration unit 22 restores the first sub-TS and the second sub-TS from the nullified TS separated by the TS separation unit 21.

  The decoding unit 23 decodes the main TS, the first sub-TS, and the second sub-TS into a main video signal, a first sub-video signal, and a second sub-video signal, respectively. The decoding unit 23 may perform decoding only for the TS selected from the main TS, the first sub-TS, and the second sub-TS according to the given TS selection signal.

  The video multiplexing unit 24 includes an image of the main video frame, an image of the first sub-video frame, and a second sub-video frame represented by the main video signal, the first sub-video signal, and the second sub-video signal obtained by decoding. The video frame images are time-division multiplexed in the order shown in FIG. 8, and a video signal representing the time-division multiplexed frame image is generated and output to the display device 402.

  Note that the video recording apparatus 100 and the video reproduction apparatus 200 are individually or as a unit, and as a software function, a CPU (Central Processing Unit) (not shown) executes a program recorded in a ROM (Read Only Unit). It may be realized, or may be realized as hardware such as an integrated circuit device.

  In particular, the encoding unit 15, the TS nulling unit 11, and the TS multiplexing unit 12 are the integrated encoding unit 101, and the TS demultiplexing unit 21, the TS restoration unit 22, and the decoding unit 23 are the integrated decoding unit 201, respectively. It is desirable to realize these in an integrated circuit device individually or as a single unit.

  Such an integrated circuit device has not only the video signal from the imaging processing unit 14 but also a plurality of video signals acquired from an external input, and can be encoded and multiplexed, and can be separated and decoded. Be expected.

  Next, operations of the video recording apparatus 100 and the video reproduction apparatus 200 configured as described above will be described with reference to flowcharts.

  FIGS. 10A and 10B are flowcharts showing examples of operations of the video recording apparatus 100 and the video reproduction apparatus 200, respectively.

First, video recording processing in the video recording apparatus 100 will be described.
The imaging processing unit 14 includes a main video signal representing a frame image corresponding to a frame rate of 30 Hz, and a first sub-video signal representing another frame image among frame images obtained from the imaging optical system 401 at a frame rate of 120 Hz. In addition, the second sub-video signal is output (S101 and FIG. 8).

  The encoding unit 15 encodes the main video signal, the first sub video signal, and the second sub video signal into the main TS, the first sub TS, and the second sub TS, respectively (S102). This encoding is based on the H.264 standard in accordance with simplified operational rules. H.264 encoding is performed.

  The TS nulling unit 11 converts the first sub-TS and the second sub-TS into a null TS (S103), and the TS multiplexing unit 12 multiplexes the main TS and the null TS in the multiplexed stream (S104). . Since these processes are the processes (S13 and S14 in FIG. 2A) themselves that have been described in detail in the first embodiment, the description thereof will not be repeated here.

  The TS multiplexing unit 12 outputs the multiplexed stream and records it on the recording medium 300 (S105).

Next, video playback processing in the video playback device 200 will be described.
The video reproduction device 200 acquires the multiplexed stream recorded from the recording medium 300 (S201).

  The TS separation unit 21 separates the main TS from the acquired multiplexed stream (S202) and separates the nullified TS (S203). Then, the TS restoration unit 22 restores the sub-TS from the nullified TS separated by the TS separation unit 21 (S204). Since these processes are the processes (S22 and S23 in FIG. 2B) performed by the TS separation device 20 described in detail in the first embodiment, description thereof will not be repeated here.

  The decoding unit 23 selects a TS selected from the main TS, the first sub-TS, and the second sub-TS according to the given TS selection signal, respectively, as the main video signal, the first sub-video signal, and the first sub-TS. Two sub-picture signals are decoded (S205). This decryption is performed in accordance with a simplified operation rule. H.264 decoding is performed.

  The video multiplexing unit 24 time-division multiplexes the frame image represented by the video signal obtained by decoding in the order shown in FIG. 8, and outputs the frame image to the display device 402 at a frame rate of 30 Hz (S206).

  Here, when the decoding unit 23 decodes all of the main TS, the first sub-TS, and the second sub-TS according to the TS selection signal, all the frame images shown in FIG. 8 are 30 Hz. The image is output to the display device 402 at a frame rate, and a high-quality slow reproduction at 1/4 times the frame rate of 120 Hz at the time of imaging is performed.

  When the decoding unit 23 decodes the main TS and the first sub-TS and does not decode the second sub-TS according to the TS selection signal, the main video frame and the first sub-video shown in FIG. The frame image of the frame is output to the display device 402 at a frame rate of 30 Hz, and a high-quality slow playback at 1/2 times the frame rate of 120 Hz at the time of imaging is performed.

  In addition, when the decoding unit 23 decodes only the main TS in accordance with the TS selection signal, normal reproduction is performed.

  Thus, the TS selection signal has a function of instructing the reproduction speed. Depending on the TS selection signal, in addition to switching whether or not to perform decoding by the decoding unit 23, the TS restoration unit 22 switches whether or not to restore the sub-TS from the null TS, or the TS separation unit. Even if it is switched at 21 whether or not the null TS is separated from the multiplexed stream, the playback speed can be switched.

  In the conventional general video reproduction device, the null TS included in the multiplexed stream is discarded because it is not distinguished from the original null TS. As a result, only the frame image represented by the main TS is reproduced at the normal speed. That is, this multiplexed stream is highly compatible with existing systems.

  In addition, the video recording apparatus 100 and the video reproduction apparatus 200 record content represented by a plurality of video streams such as a multi-angle video representing a plurality of videos viewed from a plurality of different viewpoints and a stereoscopic video by binocular vision. It can also be applied to reproduction.

  As described above, the video recording device, the video playback device, and the video recording / playback device of the present invention have been described based on the embodiment. However, the present invention is not limited to this embodiment. Unless it deviates from the meaning of this invention, what made the various deformation | transformation which those skilled in the art conceivable to this Embodiment is also contained in the scope of the present invention.

  The video recording device, video playback device, video recording / playback device, method, and program according to the present invention can be widely used in devices that record and play back video, such as video cameras and portable information terminals.

Functional block diagram showing an example of a configuration of a processing system in the first embodiment (A) Flow chart showing an example of operation of TS multiplexer, (B) Flow chart showing an example of operation of TS separator The figure which shows an example of conversion from sub TS to null TS The figure which shows an example of conversion from sub TS to null TS The figure which shows an example of conversion from sub TS to null TS (A) Functional block diagram showing an example of the configuration of the video recording / playback apparatus in the second embodiment, (B) External view showing an example of the external appearance of the video recording / playback apparatus Functional block diagram showing an example of the configuration of a video recording apparatus The figure explaining the frame image represented by the main video signal, the 1st sub video signal, and the 2nd sub video signal Functional block diagram showing an example of the configuration of a video playback device (A) A flowchart showing an example of the operation of the video recording apparatus, (B) a flowchart showing an example of the operation of the video reproduction apparatus.

Explanation of symbols

1 Video Stream Processing System 10 TS Multiplexer 11 TS Nulling Unit 12 TS Multiplexing Unit 14 Imaging Processing Unit 15 Encoding Unit 20 TS Separating Device 21 TS Separating Unit 22 TS Restoring Unit 23 Decoding Unit 24 Video Multiplexing Unit 30 Storage or Transmission path 100 Video recording device 101 Integrated encoding unit 200 Video reproduction device 201 Integrated decoding unit 300 Recording medium 400 Video recording / reproduction system 401 Imaging optical system 402 Display device

Claims (13)

An imaging unit that obtains a plurality of video signals by imaging a target;
An encoding unit that encodes one of the plurality of video signals into a main video stream and the other into a sub-video stream;
A stream nulling unit that disguises the one or more sub-picture streams as a null stream;
A video recording apparatus comprising: a stream multiplexing unit that multiplexes the one or more sub-video streams disguised as the null stream into a multiplexed stream and outputs the multiplexed stream. Each of the null stream and each sub-video stream is composed of a plurality of packets including a header and a payload.
The stream nulling unit has specific data for identifying the one or more sub-video streams in the header and a null stream in a header, and has payload data of a packet of the one or more sub-video streams in a payload The video recording apparatus according to claim 1, wherein the video recording device is disguised as a null stream by converting into a nullified stream composed of a plurality of packets. Each of the null stream and each sub-video stream is composed of a plurality of packets including a header and a payload.
The stream nulling unit has the one or more sub-video streams, specific data for identifying the null stream in a header, and each packet of the one or more sub-video streams in a payload. The video recording apparatus according to claim 1, wherein the video recording apparatus is disguised as a null stream by converting the data into a null stream composed of a plurality of packets having individual data obtained by dividing the data into two. Each of the null stream and each sub-video stream is composed of a plurality of packets including a header and a payload.
The stream nulling unit has the one or more sub-video streams having specific data for identifying the null stream in a header, and divides the entire sub-video streams into payloads into payload sizes. The video recording apparatus according to claim 1, wherein the video recording device is disguised as a null stream by converting into a nullified stream composed of a plurality of packets having individual data obtained. The stream nulling unit further sets a value for distinguishing each sub-video stream in a header of each packet of the nullified stream. 5. The video recording apparatus described. A video playback device for playing back video from a multiplexed stream in which a main video stream and one or more sub-video streams disguised as a null stream in which video data to be held is not defined are multiplexed,
A stream separation unit for separating the main video stream and a null stream in which the one or more sub-video streams are disguised from the multiplexed stream;
A stream restoration unit that restores the one or more sub-picture streams from the separated null stream;
A stream decoding unit that decodes the separated main video stream and the restored one or more sub-video streams into a main video signal and one or more sub-video signals, respectively. Playback device. The video playback device further includes:
The video multiplexing part which outputs the video signal which carries out the time division multiplexing of the frame image represented by the main video signal and one or more sub-video signals obtained by decoding is provided. Video playback device. Each of the null stream and each sub-video stream is composed of a plurality of packets including a header and a payload.
The stream separation unit, from the multiplexed stream, out of packets having specific data for identifying a null stream in the header, all the data in the payload is not a uniform value, or the data at the head of the payload is the head of the header The one or more sub-picture streams are disguised by extracting packets that are specific data for synchronization to be set to as packets of the one or more sub-picture streams disguised as null streams The video reproduction apparatus according to claim 6 or 7, wherein a null stream is separated. Each of the null stream and each sub-video stream is composed of a plurality of packets including a header and a payload.
The stream restoration unit sets specific data for identifying a valid video stream other than the main video stream in a header of each packet of the separated null stream, thereby allowing the one or more substreams from the null stream. The video reproduction apparatus according to claim 6 or 7, wherein the video stream is restored. The video recording apparatus according to any one of claims 1 to 5,
A video recording / reproducing system comprising: the video reproducing device according to claim 6. An imaging step of obtaining a plurality of video signals by imaging a target;
An encoding step of encoding one of the plurality of video signals into a main video stream and encoding the other into a sub-video stream;
A stream nulling step of disguising the one or more sub-picture streams as a null stream;
And a stream multiplexing step of multiplexing the one or more sub-video streams disguised as the null stream into a multiplexed stream and outputting the multiplexed stream for recording. A video playback method for playing back a video from a multiplexed stream in which a main video stream and one or more sub-video streams disguised as a null stream for which video data to be held is not defined are multiplexed,
A stream separation step of separating the main video stream and a null stream in which the one or more sub-video streams are disguised from the multiplexed stream;
A stream restoration step of restoring the one or more sub-picture streams from the separated null stream;
And a stream decoding step of decoding the separated main video stream and the restored one or more sub-video streams into a main video signal and one or more sub-video signals, respectively. Playback method.   A program that causes a computer to execute at least one of the steps included in the video recording method according to claim 11 and the steps included in the video reproduction method according to claim 12.

Images