JP2012114543A - Video recording device and playback device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve performance, reduce power consumption, and reduce the capacity of video to be generated by decimating encoding and decoding of an image with a motion amount smaller than a predetermined threshold value to reduce a processing amount.SOLUTION: A decimation determiner 16, when a motion amount between an input right-eye image and a right-eye image preceding by one frame is smaller than a predetermined threshold value, determines to decimate the input right-eye image without encoding, and copies the compared image preceding by one frame to a right-eye holding image buffer 14. Then, an image encoder 19 attaches dedicated header information created by a decimation image header information creator 18. Also, when the motion amount is equal to or greater than the predetermined threshold value, the image is determined to be encoded and the image encoder 19 is caused to encode the image. The image encoder 19 attaches header information from a header information creator 17 to the encoded right-eye image. Since a video recording device 10 decimates encoding of an image with a motion smaller than the predetermined threshold value, a processing amount can be reduced.

Description

  The present invention relates to a video recording apparatus and a playback apparatus, and more particularly, to a video recording apparatus that records video shot with a compound eye camera on a recording medium, and a playback apparatus that plays back video shot with a compound eye camera from the recording medium.

  There are many techniques for encoding stereoscopic video, but a technique that is considered to become mainstream in the future is MVC (Multi-view Video Coding). This is a technique that extends the MPEG4 AVC (Moving Picture Experts Group 4 Advanced Video Coding) (H.264) standard, which is the mainstream of the current coding system, to multiple viewpoints.

  As an encoding method, for example, in the case of two viewpoints, images input from two lenses are encoded as different images. At this time, in order to increase the coding efficiency, a reference that is made only before and after an image input from one lens is also referred to an image input from the other lens. By performing such encoding, it is possible to suppress the place where the size of the created video is doubled to a maximum of about 1.5 times.

  By performing the encoding as described above, the size of the encoded image can be suppressed. However, there is no change in encoding all images, and the encoding processing amount cannot be reduced. Similarly, the amount of decoding processing cannot be reduced. As a result, there is a problem that when a stereoscopic video is recorded / reproduced, the processing amount and power consumption are higher than when a normal video is recorded / reproduced.

  Therefore, a technique of not performing encoding and decoding is disclosed (for example, see Patent Document 1). The technique described in Patent Document 1 is a technique that detects the amount of motion of two images and thins out the encoding when the motion is uniform.

Japanese Patent No. 2919211

  However, the technique described in Patent Document 1 can be used only when the movement is uniform before and after in time, and when used in a scene where there is movement, there is a problem that smooth movement does not occur when played back. There is. Also, with the above-described technique, in order to create a thinned image at the time of reproduction, a decoding process of combining from previous and subsequent images is included, so the amount of reproduction processing cannot be reduced.

  The present invention has been made in view of the above points, and by thinning out encoding and decoding of an image with a motion amount less than a predetermined threshold, it is possible to reduce the processing amount, improving performance and reducing power consumption, An object of the present invention is to provide a video recording device and a playback device capable of reducing the capacity of the generated video.

  To achieve the above object, a video recording apparatus according to a first invention generates a left-eye image generating means for generating a left-eye image out of a left-eye image and a right-eye image constituting a stereoscopic image, and generates a right-eye image. A right-eye image generating means, a left-image buffer for storing a left-eye image, a right-image buffer for storing a right-eye image, and a holding image for storing one predetermined image among the left-eye image and the right-eye image A buffer, motion detection means for detecting the amount of motion of one of the images stored in the holding image buffer and one of the previously input images, and motion detected by the motion detection means The amount is compared with a predetermined threshold, and it is determined that thinning is possible when the amount of motion is less than the threshold, and thinning determination means for determining that thinning is not possible when the amount of motion is equal to or greater than the threshold, and thinning is determined by the thinning determination unit. First header information creating means for creating first header information to be used for an image, and second header information creating means for creating second header information to be used for an image determined to be thinned by the thinning judging means And the encoded image for the right eye and the encoded image for the left eye are alternately synthesized by encoding the image determined not to be thinned by the thinning determination unit and adding the second header information. An image code for outputting the first header information in the image stream instead of the encoded image of the image that is determined to be thinned out by the thinning determination unit among the encoded images And recording means for recording the image stream from the image encoding means on a recording medium.

  In order to achieve the above object, the video recording apparatus according to the second invention includes a left-eye image generating means for generating a left-eye image out of a left-eye image and a right-eye image constituting a stereoscopic image, and a right-eye image. A right-eye image generating means, a left-image buffer that stores a left-eye image for each input, a right-image buffer that stores a right-eye image for each input, a left-held image buffer that stores one left-eye image, A holding image buffer for right that stores one image for right eye, and an amount of movement between one image inputted before and one image inputted before the image for left eye and the image for right eye are inputted. When detecting one of the left-eye image and the right-eye image, the motion detection means for detecting the left-eye image and the right-eye image, the left-hold image buffer and the right-hold image buffer Retention When there is a retained image in the image buffer, the retained image is deleted and one image is encoded. There is no retained image in the retained image buffer of one image, and the retained image buffer of the other image. When there is a retained image, it is an image that encodes one image or the other image, and when there is no retained image in both the retained image buffer for the left and the retained image buffer for the right, the amount of motion detected by the motion detection means Is compared to a predetermined threshold, and it is determined that thinning is possible when the amount of motion is less than the threshold, and thinning determination means that determines that thinning is not possible when the amount of motion is equal to or greater than the threshold, First header information creating means for creating first header information to be used for the image, and second header information to be used for an image determined to be thinned by the thinning judging means A second header information creating unit configured to encode the image determined to be encoded by the thinning determination unit and the image determined to be thinning-out, and add the second header information to encode An image stream obtained by alternately synthesizing the right-eye image and the encoded left-eye image are output, and the first header information is replaced in the image stream in place of the image determined to be thinned by the thinning determination unit. And an image encoding means for combining and outputting the image stream, and a recording means for recording the image stream from the image encoding means on a recording medium.

  In order to achieve the above object, the playback device of the third invention is used for playback means for playing back an image stream from the recording medium of the first invention, and for an image determined to be thinned out in the image stream. Decimation determining means for determining whether or not an image in the image stream is a decimation image with reference to the first header information and the second header information used for the image determined to be decimation An image decoding unit that decodes an image that is determined not to be a thinned image by the determination unit, and a left image buffer and a right image buffer that separately store a left-eye image and a right-eye image decoded by the image decoding unit And a holding image buffer to which an image in the image output order described in the first header information is copied when the first header information is input to the thinning determination unit, and a left image buffer Characterized in that it comprises a right image buffer and the display control means Ruru is displayed on the display unit a stereo image for the left eye image and the right-eye image are alternately input one by one from the holding image buffer.

  In order to achieve the above object, a playback device according to a fourth aspect of the invention is used for playback means for playing back an image stream from a recording medium according to the second aspect of the invention, and for an image that is determined to be thinned out. Thinning determination means for determining whether or not an image in the image stream is a thinned image with reference to the first header information and the second header information used for the image determined to be thinned out, and thinning determination An image decoding means for decoding an image determined not to be a thinned image by the means, and a left image buffer and a right image buffer for separately storing a left eye image and a right eye image decoded by the image decoding means, A left holding image buffer to which a left-eye image in the image output order described in the first header information is copied when the first header information is input to the thinning determination unit; A right holding image buffer, a left image buffer, a right image buffer, and a left image, to which the right-eye image in the image output order described in the first header information is copied when the first header information is input to the means; And display control means for displaying on the display unit a stereoscopic image in which left eye images and right eye images are alternately input one by one from a selected buffer among the retained image buffer and the right retained image buffer. And

  According to the present invention, it is possible to reduce the amount of processing by thinning out the encoding and decoding of an image whose motion amount is less than a predetermined threshold, improving performance, reducing power consumption, and reducing the volume of generated video. Can do.

1 is a block diagram of a first embodiment of a video recording apparatus of the present invention. It is a figure which shows an example of the header information added to the image encoded. It is a figure which shows an example of the exclusive header information used instead of the image to thin out. It is a figure which shows typically an example of the thinning-out recording operation | movement of the video recording device of FIG. 2 is a flowchart for explaining the operation of the video recording apparatus of FIG. 1. It is a figure which shows an example of the image stream produced | generated by the image encoder of FIG. 1 is a block diagram of a first embodiment of a playback apparatus of the present invention. It is a figure which shows typically an example of the thinned image reproduction | regeneration operation | movement of the reproducing | regenerating apparatus of FIG. 8 is a flowchart for explaining a thinned-out image reproduction operation of the reproduction apparatus in FIG. It is a block diagram of 2nd Embodiment of the video recording apparatus of this invention. It is a figure which shows typically an example of the thinning-out recording operation | movement of the video recording device of FIG. 11 is a flowchart for explaining the operation of the video recording apparatus in FIG. 10. It is a figure which shows an example of the image stream produced | generated by the image encoder of FIG. It is a block diagram of 2nd Embodiment of the reproducing | regenerating apparatus of this invention. FIG. 15 is a diagram schematically illustrating an example of a thinned image reproduction operation of the reproduction apparatus in FIG. 14. 15 is a flowchart for explaining a thinned-out image reproduction operation of the reproduction apparatus in FIG.

  Next, embodiments of the present invention will be described with reference to the drawings.

  When performing stereoscopic video display, the left-eye image and the right-eye image are output to the odd and even lines of the monitor (for example, the left-eye image is the odd-numbered line and the right-eye image is the even-numbered line). Alternatively, the frame rate is doubled, and the left eye image and the right eye image are alternately output. For this reason, even if one image is paused within a certain range of motion, it appears smooth when reproduced because the other image is moving, and the stereoscopic effect is not impaired. Therefore, the present invention pays attention to this, and when recording a stereoscopic video signal, if the motion is within a certain range, encoding and decoding of one image is thinned out, or encoding of both images is performed. The conventional problem is solved by thinning out the encoding alternately, and each embodiment of the present invention will be described below.

(First embodiment)
FIG. 1 shows a block diagram of a first embodiment of a video recording apparatus according to the present invention. In the figure, a video recording apparatus 10 according to the present embodiment includes a left lens 11L for inputting a left-eye image, and a left imaging device for converting light information input from the left lens 11L into a left-eye video signal that is electronic information. 12L, a left image buffer 13L for storing a left-eye video signal from the left image sensor 12L, a right lens 11R for inputting a right-eye image, and light information input from the right lens 11R for right-eye that is electronic information Right image sensor 12R for converting to a video signal, right image buffer 13R for storing the right eye video signal from the right image sensor 12R, and right holding image buffer for storing one right eye video signal (for one frame) 14, a motion detector 15 that detects the amount of motion of two images, and a thinning-out determination that determines whether or not encoding can be thinned out based on the amount of motion detected by the motion detector 15. 16, a header information creating unit 17 that creates header information used for an image determined to be thinned by the thinning judging unit 16, and header information used for an image judged to be thinned out by the thinning judging unit 16. A thinned-out image header creator 18, an image encoder 19 that encodes a video signal, a microphone 20 that collects sound and converts it into an audio signal, an audio encoder 21 that encodes the audio signal, A multiplexer 22 that multiplexes the encoded video signal and the encoded audio signal, and a recording unit that records the moving image stream data obtained by multiplexing by the multiplexer 22 on the recording medium 23.

  Of the left-eye image and the right-eye image constituting the stereoscopic video, the left lens 11L and the left imaging device 12L constitute the left-eye image generating means of the present invention that generates the left-eye image, and the right lens 11R and the right imaging. The element 12R constitutes a right-eye image generating unit of the present invention that generates a right-eye image. In addition, the microphone 20 and the speech encoder 21 constitute the encoded speech signal generation means of the present invention. The video recording apparatus 10 according to the present embodiment performs recording by thinning one image with a twin-lens camera. In this embodiment, an example of a twin-lens camera will be described. However, the present invention is not limited to this, and the present invention can also be used with a camera having three or more eyes.

  Next, processing contents of the video recording apparatus 10 of the present embodiment will be described.

  The left image buffer 13L temporarily stores a left-eye video signal (hereinafter also referred to as a left-eye image) captured by the left image sensor 12L. On the other hand, the right image buffer 13R temporarily stores a right-eye video signal (hereinafter also referred to as a right-eye image) captured by the right imaging element 12R. Each of the left image buffer 13L and the right image buffer 13R can store video signals of a plurality of images. The image encoder 19 encodes the left-eye video signal by adding the header information from the header information generator 17 as it is, as in normal recording. As shown in FIG. 2, the header information includes thinned-out image information 31 indicating whether or not it is a thinned-out image, a serial number 32 from the beginning, an output order number 33, and decoding information 34 necessary for decoding. Is described.

  On the other hand, for the right-eye video signal, the image encoder 19 encodes the right-eye image that is not thinned through the thinning-out determination unit 16 by adding the same header information as the left, and thins out the right-eye image. Does not add dedicated header information. Since the right-eye image to be thinned is not decoded, the dedicated header information does not need to have information related to decoding. As shown in FIG. 3, the dedicated header information includes thinned image information 31 indicating whether or not the image is a thinned image, a serial number 32 from the top, a number 33 in the output order, and an output image instead. The number 36 of the image to be performed is described. The multiplexer 22 multiplexes the encoded video signal encoded by the image encoder 19 with the encoded audio signal from the audio encoder 21 to generate a moving image stream and records it on the recording medium 23. .

  In the present embodiment, the right eye image is thinned out, but the left eye image may be thinned out. For example, when a person with better visual acuity sees, a thinned-out left eye image may appear to be a beautiful image. This may also be taken into account because of the influence of the dominant eye.

  Next, a thinning recording method in the video recording apparatus 10 will be described with reference to FIGS. 4 and 5 together with FIG. FIG. 4 schematically shows the thinning-out recording operation, and FIG. 5 shows a flowchart for explaining the recording operation.

  First, an image is input (step S1). Subsequently, it is checked whether or not there is a right eye image in the right held image buffer 14 (step S2). Immediately after the start of recording, the right eye image is not stored in the right retained image buffer 14 (hereinafter, this is referred to as “no retained image”). The right holding image buffer 14 is used to hold an image with which the amount of motion is compared when thinning is performed. This is to prevent an image used for comparison from disappearing. This is because the image buffer is normally overwritten from the old image. If the image in the designated image buffer is not overwritten, the right held image buffer 14 need not be used.

  When there is no held image in the right held image buffer 14, the motion detector 15 and the right eye image that is temporally held in the right image buffer 13R and the input right eye image that is currently being encoded are input. The amount of motion with the image (that is, the amount of motion of the two adjacent right-eye images) is compared with a predetermined threshold (step S9). If the amount of motion is equal to or greater than a predetermined threshold, the thinning determination unit 16 determines that the right-eye image to be encoded is to be encoded, and causes the image encoder 19 to encode (step S13). Image 1 to image 3 in FIG. 4 are right-eye images determined to be encoded in step S13. Subsequently, the image encoder 19 adds the header information shown in FIG. 2 created by the header information creator 17 to the encoded right-eye image (step S14).

  On the other hand, when the amount of motion is less than a predetermined threshold, the thinning determination unit 16 determines that the right-eye image to be encoded is thinned without being encoded (step S10). Subsequently, the thinning determination unit 16 copies the previous right-eye image used for the comparison to the right held image buffer 14 (step S11). Subsequently, the image encoder 19 adds the dedicated header information shown in FIG. 3 created by the thinned-out image header information creating unit 18 instead of the image judged to be thinned out (step S12). The image 4 in FIG. 4 is the right-eye image determined to be thinned out in step S 10, and the previous image 3 used for comparison at this time is copied to the right holding image buffer 14.

  The threshold value may be changed according to the image quality mode to be recorded. For example, the value is decreased in the high image quality mode, and is increased in the low image quality mode. In addition, the dedicated header information is used to clearly indicate that the header is thinned out and to specify an image to be output instead during reproduction. A serial number is written in the header information of each image, and an image to be output instead can be designated by indicating the number to be output in the dedicated header information.

  On the other hand, when the right eye image is stored in the right retained image buffer 14 (hereinafter, this is referred to as “there is a retained image”), the motion detector 15 stores the retained image in the right retained image buffer 14. Then, the amount of motion with the input image to be encoded is compared with a predetermined threshold (step S3). FIG. 4 shows that a motion comparison is performed between the image 3 held in the right held image buffer 14 and the input image 5 to be encoded.

  The decimation determining unit 16 determines that the input image is to be encoded when the amount of motion is greater than or equal to a predetermined threshold value, and causes the image encoding unit 19 to encode (step S6). Further, the thinning determination unit 16 deletes the image held in the right held image buffer 14 (step S7). Subsequently, the image encoder 19 adds the header information shown in FIG. 2 created by the header information creator 17 to the encoded right-eye image (step S8).

  On the other hand, when the amount of motion is less than the predetermined threshold, the thinning determination unit 16 determines that the image is thinned without being encoded (step S4). Subsequently, the image encoder 19 adds the dedicated header information shown in FIG. 3 created by the thinned image header information creating unit 18 instead of coding the image determined to be thinned (step S5). Image 5 and image 6 in FIG. 4 are images determined to be thinned out in step S4, and image 7 is an image determined to be encoded in step S6, indicating that retained image 3 is deleted at this time. 4 is an image determined to be encoded in step S13.

  FIG. 6 shows an example of an image stream generated by the image encoder 19. In this example, the left eye image R and the right eye image L are alternately multiplexed. In this example, the fourth, fifth, and sixth right-eye images are thinned out, and these right-eye images have only dedicated header information and no image data. The dedicated header information describes that the third right-eye image is used during reproduction.

  The multiplexer 22 multiplexes this image stream with the audio stream (step S15), divides it into TS packets, and records them on the recording medium 23 by a known recording means (step S16).

  As described above, according to the video recording apparatus 10 of the present embodiment, when a stereoscopic video signal is recorded, the coding of an image whose motion is less than a predetermined threshold is thinned out, so that the processing amount can be reduced.

  Next, a first embodiment of the playback apparatus of the present invention will be described.

  FIG. 7 shows a block diagram of a first embodiment of a playback apparatus according to the present invention. As shown in the figure, the playback device 40 of this embodiment includes a playback means for a recording medium 41 on which stream data is recorded, a separator 42 that divides the stream data into images and sounds, and header information of the images. A decimation determiner 43 that determines whether the image is a decimation image, an image decoder 44 that decodes the image, a left image buffer 45L that stores the decoded left-eye image, and a right that stores the decoded right-eye image Image buffer 45R, right holding image buffer 46 for storing one right-eye image, display control unit 47 for displaying stereoscopic video on the monitor 48, audio decoder 491 for decoding audio, and audio output And a speaker 492. The playback device 40 of the present embodiment plays back a video obtained by thinning and recording one image with a twin-lens camera. Here, the thinned image is the right-eye image. Note that a playback device for video recorded by a camera with three or more eyes can be configured in the same manner as in this embodiment.

  As with the recording medium 23, the recording medium 41 records a moving image stream in which an image stream and an audio stream having the format shown in FIG. The separator 42 separates the moving image stream read from the recording medium 41 into an image stream and an audio stream.

  The thinning determination unit 43 analyzes header information in each image of the separated image stream, and supplies the encoded left-eye image to the image decoder 44 as it is in normal reproduction to be decoded. At the same time, it is determined whether or not the encoded right-eye image is a thinned image from the header information. If it is not a thinned image, it is supplied to the image decoder 44 for decoding and stored in the right image buffer 45R. If the image is an image, the image to be output is searched for instead of the image held in the right image buffer 45R, and the searched image is copied to the right held image buffer 46.

  The right holding image buffer 46 is used to output a right eye image instead of the right eye image that is being thinned. This is to prevent the right-eye image used for the image from disappearing. This is because the image buffer is normally overwritten from the old image. If the image in the designated image buffer is not overwritten, the right holding image buffer 46 need not be used.

  The image decoder 44 decodes the input encoded image, stores the decoded left-eye image in the left-eye image buffer 45L, and stores the decoded right-eye image in the right-eye image buffer 45R. The display control unit 47 alternately takes out the left-eye image and the right-eye image one by one from the left image buffer 45L and the right image buffer 45R at an appropriate timing, and displays them on the monitor 48 as a stereoscopic video.

  Next, the playback processing operation of the playback device 40 of the present embodiment will be described with reference to FIGS. 8 and 9 together with FIG. FIG. 8 is a diagram schematically showing the thinned image reproduction operation of the reproduction device 40, and FIG. 9 is a flowchart for explaining the thinned image reproduction operation of the reproduction device 40.

  First, a reproduced image stream reproduced from the recording medium 41 by a known reproducing means and further separated by the separator 42 is input (step S21). The thinning determination unit 43 analyzes header information in the reproduced image stream and determines whether the encoded right-eye image is a thinned image (step S22).

  If it is determined that the input encoded right-eye image in the reproduction image stream is not a thinned image, the thinning determination unit 43 checks whether there is an image in the right retained image buffer 46 (step S27). ). If there is no retained image in the right retained image buffer 46, the input encoded right eye image is supplied to the image decoder 44 for decoding (step S31). Then, the image decoder 44 accumulates the decoded right eye image in the right image buffer 45R (step S32). The right image buffer 45R outputs the accumulated right eye image to the display control unit 47 when the order of the output indicated by the header information is reached (step S33). Image 1, image 2, and image 3 in FIG. 8 are not thinned images, and show the right-eye image when there is no held image in the right held image buffer 46.

  If the thinning determination unit 43 determines in step S27 that there is a stored image in the right stored image buffer 46, the thinning determination unit 43 supplies the input encoded right-eye image to the image decoder 44 for decoding (step S27). S28). In this case, the thinning determination unit 43 deletes the held image in the right held image buffer 46 because the thinned image has once ended (step S29).

  Then, the image decoder 44 accumulates the decoded right eye image in the right image buffer 45R (step S30). The right image buffer 45R outputs to the display control unit 47 when the order of outputting the stored right-eye images is reached (step S33). An image 7 in FIG. 8 indicates a right-eye image that has not been thinned out and is input when there is a retained image in the right retained-image buffer 46.

  On the other hand, if the thinning determination unit 43 determines in step S22 that the input encoded right-eye image in the reproduction image stream is a thinned image, whether or not there is a held image in the right held image buffer 46. (Step S23). When there is no retained image in the retained image buffer for right 46, the thinning determination unit 43 performs the thinning without decoding the input encoded right-eye image (step S25), and substitutes the number of the image to be output instead. Is obtained from the header information added to the inputted thinned image, and the right-eye image of the header information having the number of the image to be output is copied from the right image buffer 45R to the right retained image buffer 46 (step S26). ). The right holding image buffer 46 outputs the copied right eye image to the display control unit 47 when the order of image output is reached (step S33).

  An image 4 in FIG. 8 is a thinned image, and there is no retained image in the retained image buffer for right 46 at the time of input. Further, since the number of the image to be output described in the header information of the thinned image 4 is the image 3, the image 3 having the number as the header information is copied to the right holding image buffer 46. Instead of the thinned image 4, the image 3 is output to the display control unit 47.

  If the thinning determination unit 43 determines that there is a retained image in the right retained image buffer 46 in step S23, the thinning determiner 43 thins the input encoded right eye image without decoding (step S24). At this time, instead of the input thinned image, the held image in the right held image buffer 46 is output when the order of output is reached (step S33). Image 5 and image 6 in FIG. 8 indicate that the images are thinned out when the image 3 is held in the right holding image buffer 46, and the image 5 and the image 6 are replaced with the image 5. 3 is output.

  The display control unit 47 displays each input image on the monitor 48.

(Second Embodiment)
FIG. 10 shows a block diagram of a second embodiment of the video recording apparatus according to the present invention. In the figure, the same components as those in FIG. In FIG. 10, a video recording apparatus 50 according to the present embodiment includes a left lens 11L that inputs a left-eye image, and a left imaging element that converts light information input from the left lens 11L into a left-eye video signal that is electronic information. 12L, a left image buffer 13L for storing a left-eye video signal from the left image sensor 12L, a left holding image buffer 51 for storing one left-eye video signal (for one frame), and a right lens for inputting a right-eye image 11R, a right image sensor 12R that converts light information input from the right lens 11R into a right eye video signal that is electronic information, and a right image buffer 13R that stores a right eye video signal from the right image sensor 12R. A right holding image buffer 14 for storing one right eye video signal (for one frame), and a motion detector 52 for detecting the motion amount of two temporally different images. A decimation determining unit 53 that determines whether or not to perform decimation on the basis of the amount of motion detected by the motion detector 52; The information generator 17, the thinned-out image header generator 18 that generates header information to be used for images that are determined to be thinning-out by the thinning-out determiner 53, the image encoder 54 that encodes the video signal, and the audio A microphone 20 that converts sound into an audio signal, an audio encoder 21 that encodes the audio signal, a multiplexer 55 that multiplexes the encoded video signal and the encoded audio signal, and a multiplexer 55 And a recording means for recording the moving image stream data obtained by multiplexing on the recording medium 56. The video recording apparatus 50 of the present embodiment records both images alternately with a twin-lens camera. In this embodiment, an example of a twin-lens camera will be described. However, the present invention is not limited to this, and the present invention can also be used with a camera having three or more eyes.

  Next, a thinning recording method in the video recording apparatus 50 will be described with reference to FIGS. 11 and 12 together with FIG. FIG. 11 schematically shows the thinning-out recording operation, and FIG. 12 shows a flowchart for explaining the recording operation.

  First, an image is input (step S41). Subsequently, the thinning determination unit 53 checks whether there is a retained image in its retained image buffer (step S42). The own retained image buffer is the left retained image buffer 51 when encoding the left-eye image, and the right retained image buffer 14 when encoding the right-eye image. Immediately after the start of recording, there is no held image in the right held image buffer 14 and the left held image buffer 51. The right holding image buffer 14 and the left holding image buffer 51 are used to hold an image whose motion amount is compared when thinning is performed. This is to prevent an image used for comparison from disappearing. This is because the image buffer is normally overwritten from the old image. If the image in the designated image buffer is not overwritten, the right holding image buffer 14 and the left holding image buffer 51 need not be used.

  If there is a retained image in its own retained image buffer on the encoding side, the thinning determination unit 53 deletes the image in its retained image buffer (step S43), and the image input this time is the image encoder 54. To be encoded (step S47). When the left-eye image 5 in FIG. 11 is input, the left-eye image 3 is deleted because the left-eye image 3 is stored in the left-held image buffer 51, and the left-eye image 5 input this time is stored in step S43. Encoded. Similarly, since the right-eye image 4 is held in the right-eye holding image buffer 14 when the right-eye image 6 in FIG. 11 is input, the right-eye image 4 is deleted, and the right-eye image 6 input this time is input. Are encoded in step S43.

  Further, when it is determined in step S42 that there is no retained image in its retained image buffer, the thinning determination unit 53 checks whether there is a retained image in the other retained image buffer (step S44). The other held image buffer is the right held image buffer 14 when the left-eye image is encoded, and the left held image buffer 51 when the right-eye image is encoded.

  When it is determined in step S44 that there is a retained image in the other retained image buffer, the thinning determination unit 53 determines whether the current input image is a right-eye image or a left-eye image (step S45). ). When it is determined in step S45 that the current input image is a right-eye image, the thinning determination unit 53 supplies the right-eye image to the image encoder 54 for encoding (step S47). When the right eye image 4 in FIG. 11 is input, it is determined in step S42 that there is no retained image in the right retained image buffer 14, and if the left eye image 3 is retained in the left retained image buffer 51 at this time, the step is performed. In step S44, it is determined in step S45 that the input image is a right-eye image, and the right-eye image 4 is encoded in step S47. Following step S47, the image encoder 54 adds the header information shown in FIG. 2 generated by the header information generator 17 to the encoded image (step S48).

  When it is determined in step S44 that there is no retained image in the other retained image buffer, or when it is determined in step S45 that the current input image is the left-eye image, the thinning determiner 53 determines the motion detector 52. The amount of motion between the previous image detected in step 1 and the current input image to be encoded is compared with a predetermined threshold (step S46). If the amount of motion is greater than or equal to the threshold, the thinning determination unit 53 supplies the current input image to the image encoder 54 and encodes it (step S47). For example, the left-eye image 3 in FIG. 11 is encoded in step S47 as a result of determining in step S46 that the amount of motion with the previous left-eye image 2 is greater than or equal to the threshold value. The right-eye image 3 in FIG. 11 is similarly encoded.

  On the other hand, if it is determined in step S46 that the amount of motion is less than the threshold value, the thinning determination unit 53 determines that the input image at that time is not encoded, determines that the image is not encoded, and is used for comparison. The stored image is copied to the retained image buffer (step S49). For example, when the left-eye image 4 of FIG. 11 is input, since the stored image does not exist in both of the two stored image buffers 51 and 14, the input left-eye image 4 is the previous left-eye image 3. Is determined to be less than the threshold value in step S46, the left eye image 3 used for comparison is copied to the left retained image buffer 51 in step S49. The right-eye image 5 in FIG. 11 is also thinned out in the same manner, and the right-eye image 4 compared with the right-eye image 5 is copied to the right holding image buffer 14. Then, the image encoder 54 does not encode the image determined to be thinned by the thinning determiner 53, and adds the dedicated header information shown in FIG. 3 created by the thinned image header information creator 18 (step S50). ).

  The threshold value used in step S46 may be changed according to the image quality mode to be recorded. For example, the value is decreased in the high image quality mode, and is increased in the low image quality mode. In addition, the dedicated header information is used to clearly indicate that thinning has been performed and to specify the image quality to be output instead during reproduction. An image to be output instead can be specified by writing a serial number in the header of each image and indicating the number to be output instead of the dedicated header information. Repeat these tasks.

  After step S48 or S50, the multiplexer 55 multiplexes the image stream obtained by the image encoder 54 with the audio stream obtained by the audio encoder 21 (step S51), and divides it into TS packets. Then, recording is performed on the recording medium 56 by a known recording means (step S52).

  FIG. 13 shows an example of an image stream generated by the image encoder 54. In this example, the left and right images are alternately multiplexed. In this example, it is assumed that the left-eye image L4 and L6 and the right-eye image R5 and R7 are thinned out. The left-eye images L4 and L6 and the right-eye images R5 and R7 have only header information H, and there is no image data S. The header information H is dedicated, and it is described that the previous image is used for each reproduction.

  As described above, according to the video recording apparatus 50 of the present embodiment, when a stereoscopic video signal is recorded, the amount of processing can be reduced because the coding of an image whose motion is less than a predetermined threshold is thinned out.

  Next, a second embodiment of the playback apparatus of the present invention will be described.

  FIG. 14 shows a block diagram of a second embodiment of the playback apparatus according to the present invention. In the figure, the same components as those in FIG. 7 are denoted by the same reference numerals, and the description thereof is omitted. As shown in FIG. 14, the playback device 60 of this embodiment refers to playback means for playing back stream data from a recording medium 61, a separator 42 that separates stream data into images and sounds, and header information of the images. A decimation determining unit 62 for determining whether the image is a decimation image, an image decoder 44 for decoding the image, a left image buffer 45L for storing the decoded left eye image, and a left holding image buffer for storing one left eye image 63, a right image buffer 45R for storing the decoded right eye image, a right holding image buffer 46 for storing one right eye image, a display control unit 47 for displaying a stereoscopic image on the monitor 48, and decoding audio And a speech decoder 491 for outputting the sound and a speaker 492 for outputting the sound. The playback device 60 according to the present embodiment plays back video recorded on the recording medium 61 by alternately thinning out both the left and right images with a twin-lens camera. Note that a playback device for video recorded by a camera with three or more eyes can be configured in the same manner as in this embodiment.

  Similarly to the recording medium 56, the recording medium 61 records a moving image stream in which an image stream and an audio stream having the format shown in FIG. The separator 42 separates the video stream read from the recording medium 61 into an image stream and an audio stream. The thinning determination unit 62 analyzes the header information of the image stream, decodes it if it is not a thinned image, and stores it in the respective image buffer 45L or 45R. If it is a thinned image, it is not decoded, but instead an image to be output is retrieved and copied from the image buffer to the corresponding retained image buffer 63 or 46. The image is output from the left and right image buffers 45L, 45R, 46 or 63 to the display control unit 47 at an appropriate timing. The display control unit 47 causes the monitor 48 to display a stereoscopic video in which left-eye video and right-eye video are alternately input one by one.

  Next, the playback processing operation of the playback device 60 of the present embodiment will be described with reference to FIGS. 15 and 16 together with FIG. FIG. 15 is a diagram schematically showing the thinned image reproduction operation of the reproduction device 60, and FIG. 16 is a flowchart for explaining the thinned image reproduction operation of the reproduction device 60. In FIG. 16, the same processing steps as those in FIG. 9 are denoted by the same reference numerals, and the description thereof is omitted.

  First, a reproduced image stream reproduced from the recording medium 61 by a known reproducing means and further separated by the separator 42 is input (step S21). The thinning determination unit 62 analyzes header information in the reproduced image stream and determines whether or not the encoded image is a thinned image (step S50). When it is determined that the input encoded image in the reproduction image stream is not a thinned image, the thinning determination unit 62 checks whether there is an image in the retained image buffers 46 and 63 corresponding to the image. (Step S51). The retained image buffers 46 and 63 are used to output an image instead of the image that is being thinned. This is to prevent an image used for image output from being erased. This is because the image buffer is normally overwritten from the old image. If the image in the designated image buffer is not overwritten, the retained image buffers 46 and 63 need not be used.

  If there are no retained images in the retained image buffers 46 and 63, the input encoded image is supplied to the image decoder 44 for decoding (step S31). Then, the image decoder 44 stores the decoded image in the corresponding right image buffer 45R or left image buffer 45L (step S52).

  The right image buffer 45R and the left image buffer 45L output the stored images to the display control unit 47 when the order of the output indicated by the header information is reached (step S53). The left-eye image 1, the left-eye image 2, the left-eye image 3, the right-eye image 1, the right-eye image 2, and the right-eye image 3 in FIG. An image when there is no retained image in the retained image buffer 63 is shown.

  If the thinning determination unit 62 determines in step S51 that there are stored images in the stored image buffers 46 and 63, the thinned determination unit 62 supplies the input encoded image to the image decoder 44 for decoding (step S28). . In this case, the thinning determination unit 62 deletes the retained images in the retained image buffers 46 and 63 because the thinned image has been temporarily finished (step S29).

  Then, the image decoder 44 stores the decoded image in the corresponding right image buffer 45R or left image buffer 45L (step S54). The right image buffer 45R and the left image buffer 45L output the stored images to the display control unit 47 when the order of the output indicated by the header information is reached (step S53). A left-eye image 5 and a left-eye image 7 in FIG. 15 indicate left-eye images that are input when there is a retained image in the left retained-image buffer 63, and are a right-eye image 6 and a right-eye image 8. Indicates a right-eye image that has not been thinned out and is input when there is a retained image in the right retained-image buffer 46.

  On the other hand, if the thinning determination unit 62 determines in step S50 that the input encoded image in the reproduction image stream is a thinned image, the thinning determination unit 62 stores the stored images in the right holding image buffer 46 and the left holding image buffer 63. It is checked whether or not there is (step S55). If there is no retained image in the retained image buffer, the decimation determining unit 62 performs decimation without decoding the input encoded image (step S25), and instead decimates the image number to be output. The header information image obtained from the header information added to the image and having the number of the image to be output is copied from the image buffer 45R or 45L to the right retained image buffer 46 or the left retained image buffer 63 (step S56). . The right holding image buffer 46 and the left holding image buffer 63 output the copied images to the display control unit 47 when the order of output is reached (step S53).

  The left-eye image 4, the left-eye image 6, the right-eye image 5, and the right-eye image 7 in FIG. 15 are thinned images, and there are no retained images in the retained image buffers 46 and 63 at the time of input. In addition, an image having the header number of an image to be output described in the header information of the thinned image is copied to the right holding image buffer 46 and the left holding image buffer 63. The display controller 47 outputs an image copied to the retained image buffers 46 and 63 instead of the thinned image. Which image is used is the header information image having that number because the number of the image to be output is described instead of the header information added to the thinned image.

  The display control unit 47 displays each input image on the monitor 48.

  In the present embodiment, there is no thinned image and there is a retained image.

10, 50 Video recording device 11L Left lens 11R Right lens 12L Left image sensor 12R Right image sensor 13L, 45L Left image buffer 13R, 45R Right image buffer 14, 46 Right hold image buffer 15, 52 Motion detector 16 , 43, 54, 62 Decimation determination unit 17 Header information creation unit 18 Decimation header information creation unit 19 Image encoder 20 Microphone 21 Speech encoder 22, 55 Multiplexer 23, 41, 56, 61 Recording medium 40, 60 Playback device 42 Separator 44 Image decoder 47 Display controller 48 Monitor 51, 63 Left image buffer

Claims (4)

Left-eye image generating means for generating the left-eye image out of the left-eye image and the right-eye image constituting the stereoscopic video;
Right-eye image generating means for generating the right-eye image;
A left image buffer for storing the left eye image;
A right image buffer for storing the right eye image;
A holding image buffer for storing one predetermined image of the left-eye image and the right-eye image;
A motion detecting means for detecting a motion amount between the one image and the one image input before the one image stored in the held image buffer;
A decimation determination in which the amount of motion detected by the motion detecting means is compared with a predetermined threshold, and it is determined that thinning is possible when the amount of motion is less than the threshold, and thinning-out is determined when the amount of motion is equal to or greater than the threshold. Means,
First header information creating means for creating first header information used for an image determined to be thinned by the thinning judging means;
Second header information creating means for creating second header information to be used for an image determined to be thinned out by the thinning judging means;
The encoded image for the right eye and the encoded image for the left eye are alternately encoded by encoding the image determined not to be thinned by the thinning determination unit and adding the second header information. The synthesized image stream is output, and the first header information is synthesized into the image stream in place of the coded image of the coded image determined to be thinned by the thinning judging unit. Image encoding means for outputting,
Recording means for recording the image stream from the image encoding means on a recording medium;
A video recording apparatus comprising: Left-eye image generating means for generating the left-eye image out of the left-eye image and the right-eye image constituting the stereoscopic video;
Right-eye image generating means for generating the right-eye image;
A left image buffer for storing the left eye image for each input;
A right image buffer for storing the right eye image for each input;
A left holding image buffer for storing one image for the left eye;
A holding image buffer for the right for storing one image for the right eye;
A motion detecting means for detecting a motion amount between the one image input before and the one image input before the one of the left-eye image and the right-eye image;
When encoding one of the left-eye image and right-eye image that is input, the encoded image is stored in the input image-holding image buffer of the left-holding image buffer and the right-holding image buffer. When there is an image, the retained image is deleted and the one image is encoded, and there is no retained image in the retained image buffer of the one image, and the retained image in the retained image buffer of the other image If there is a stored image in both the left stored image buffer and the right stored image buffer, the motion detected by the motion detecting means is used. Compare the amount with a predetermined threshold, and determine that thinning is possible when the amount of motion is less than the threshold, and determine that thinning is not possible when the amount of motion is equal to or greater than the threshold. And the judgment means,
First header information creating means for creating first header information used for an image determined to be thinned by the thinning judging means;
Second header information creating means for creating second header information to be used for an image determined to be thinned out by the thinning judging means;
Encode the image determined as the image to be encoded by the decimation determination unit and the image determined as the decimation failure, and add the second header information, and encode the encoded right-eye image. And outputting the image stream in which the left-eye image is alternately synthesized, and synthesizing the first header information in the image stream in place of the image judged to be thinned by the thinning judging means. Image encoding means for outputting;
Recording means for recording the image stream from the image encoding means on a recording medium;
A video recording apparatus comprising: Reproducing means for reproducing the image stream from the recording medium according to claim 1;
With reference to the first header information used for an image determined to be thinned out in the image stream and the second header information used for an image determined not to be thinned out, the image in the image stream is a thinned image. Thinning judgment means for judging whether or not,
Image decoding means for decoding an image determined not to be the thinned image by the thinning determination means;
A left image buffer and a right image buffer for separately storing a left eye image and a right eye image decoded by the image decoding means;
A holding image buffer to which an image in the image output order described in the first header information is copied when the first header information is input to the thinning determination unit;
Display control means for displaying on the display unit a stereoscopic video in which left-eye images and right-eye images are alternately input one by one from the left image buffer, right image buffer, and holding image buffer;
A playback apparatus comprising: Reproducing means for reproducing the image stream from the recording medium according to claim 2;
With reference to the first header information used for an image determined to be thinned out in the image stream and the second header information used for an image determined not to be thinned out, the image in the image stream is a thinned image. Thinning judgment means for judging whether or not,
Image decoding means for decoding an image determined not to be the thinned image by the thinning determination means;
A left image buffer and a right image buffer for separately storing a left eye image and a right eye image decoded by the image decoding means;
A left holding image buffer to which a left-eye image in the image output order described in the first header information is copied when the first header information is input to the thinning determination unit;
A right holding image buffer to which a right-eye image in the image output order described in the first header information is copied when the first header information is input to the thinning determination unit;
A stereoscopic image in which left-eye images and right-eye images are alternately input from the selected buffer among the left image buffer, right image buffer, left hold image buffer, and right hold image buffer is displayed on the display unit. Display control means for displaying;
A playback apparatus comprising:

Images