JP6316640B2 - VIDEO RECORDING DEVICE, VIDEO REPRODUCTION DEVICE, AND VIDEO RECORDING PROGRAM - Google Patents

Description

  The present invention relates to a high-definition video recording apparatus, reproducing apparatus, and recording program.

In recent years, as a high-definition video system having a resolution higher than that of high-definition (resolution 1920 × 1080), for example, research and development of super high-definition (resolution 7680 × 4320) having 16 times the number of pixels of high-definition has been promoted.
Super Hi-Vision has an extremely high data rate compared to conventional Hi-Vision video systems. In particular, a full-spec (full color total number of pixels: 99 million pixels, frame frequency: 120 Hz) uncompressed signal requires a large signal processing system with a data rate of 144 Gbps. For this reason, in order to realize a compact and agile Super Hi-Vision video system, a color system called dual green (DG) is often used for the purpose of reducing the data rate.

  A DG Super Hi-Vision signal (hereinafter referred to as a DG signal) is a Bayer-type sampling structure in which a single frame image is used as a color system of a single-plate image sensor (red (R), green ( G1 and G2) and blue (B) of four types of images) (see, for example, Non-Patent Document 1).

  In order to record such a DG signal, for example, for an input DG signal, a single frame image is spatially divided into a plurality of images, and each of them is recorded as a plurality of recording media or a plurality of images as individual low resolution images. A recording apparatus that performs recording in a recording area has been proposed (see, for example, Patent Document 1). When reproducing a DG signal recorded by such a device, a plurality of divided images associated as the same frame are read in parallel from a plurality of recording media or a plurality of recording areas, and the display device The image is reconstructed.

  Specifically, for example, the recording apparatus handles a DG signal by dividing a unit frame image into four vertical and horizontal sections to obtain 16 HD images (resolution 1920 × 1080). Further, for example, as shown in FIG. 13, the recording apparatus divides a unit frame image having a Bayer-type sampling structure into R, G1, G2, and B, and four low-resolution images (resolution 3840 × 2160). DG signal is handled by reconfiguration. By such a method, consistency between the DG signal and a known standardized video signal processing technique is taken.

JP 2006-019804 A

“4000 scanning lines, 4-plate super high-definition video camera”, Tomoyuki Yamashita et al., Video Information Media 58 (3), pp. 383-391, March 2004

However, in the above-described recording apparatus, in order to record a single frame image of a DG signal on a plurality of recording media or a plurality of recording areas, image data is distributed, and management of the recording medium, management of recorded image data, There was a problem that operation became complicated.
In addition, when compressing the data rate by video encoding, continuity with adjacent pixels in the original image is lost at the boundary portion of the spatially divided image, so the compression efficiency decreases, In the case of encoding by an irreversible process, there is a problem that compression distortion occurs.
Furthermore, when a DG signal recorded by such a device is reproduced, a timing synchronization circuit for reading the divided image data in parallel is required, which complicates the configuration of the reproduction device and reduces the frame. There is also a problem that the processing load for reconstructing the image is large.

  It is an object of the present invention to provide a video recording device, a video playback device, and a video recording program that can efficiently handle high-definition video without dispersing image data.

  The video recording apparatus according to the present invention is one of up-conversion, down-conversion, and image transformation of at least one color component for a single frame image of a color system composed of a plurality of color components having a first sampling structure. A first conversion unit that converts the image into a single frame image of a color method having a second sampling structure, the first conversion unit including the color component including the converted color component, and the first conversion An encoding unit that encodes the image data converted by the unit using a predetermined compression method.

According to this configuration, the video recording apparatus treats the input video as a color-type single frame image having a sampling structure different from that at the time of input, thereby spatially dividing the single frame image into a plurality of images. And can be recorded on a recording medium as a single image data. As a result, the video recording apparatus can efficiently handle high-definition video without dispersing image data, which facilitates management of recording media and management and operation of recorded image data.
In addition, the video recording device can apply the encoding technology while maintaining the continuity of the original image, so the compression efficiency is improved and compression distortion can be suppressed even when encoding is performed using an irreversible process. It becomes.

  The second sampling structure may correspond to a predetermined standardized compression method.

  According to this configuration, since the image data recorded by the video recording apparatus can be handled as a single encoded signal having a standardized sampling structure in the recording medium, the recorded data is not dispersed, It is possible to easily perform operations such as browsing, organizing, and backing up recorded data using a standardized efficient encoding technique. In addition, since a timing synchronization circuit for synchronously reading data distributed on a plurality of recording media is not required at the time of reproducing a recorded video, it is possible to prevent the apparatus configuration from becoming complicated.

  The video recording apparatus may include a conversion method determination unit that determines whether conversion is performed by the up-conversion, down-conversion, or image transformation method.

  According to this configuration, the video recording apparatus can change the image conversion algorithm in accordance with the color system of the input video signal and convert it to a video signal having the same sampling structure. In this way, the same video encoding technology can be applied to different input video signal color schemes, so that the circuit configuration can be simplified and a compact video recording apparatus corresponding to various input signals can be realized. .

  The conversion method determination unit may determine a conversion method according to a type of the first sampling structure.

  According to this configuration, the video recording apparatus determines the conversion method according to the type of the sampling structure, so that it can automatically cope with a plurality of color methods, and convenience is improved.

  The conversion method determination unit may determine a conversion method according to a designated compression mode.

  According to this configuration, since the video recording apparatus determines the conversion method according to the designated compression mode, it can automatically cope with a plurality of color methods, and convenience is improved.

  The video reproducing apparatus according to the present invention includes a decoding unit that decodes the single frame image of the color scheme having the second sampling structure encoded by the predetermined compression scheme in the video recording device, and the decoding unit The image data decoded in step 3 is converted into at least one color component by any one of up-conversion, down-conversion, and image transformation, and is composed of a plurality of color components including the converted color component. A second conversion unit that converts the image into a single frame image of a color system having the sampling structure.

  A video recording program according to the present invention is a video program for causing a computer to record an input video, and the control unit of the computer has a single color system composed of a plurality of color components having a first sampling structure. A second sampling structure in which at least one color component is converted by one of up-conversion, down-conversion, and image transformation for one frame image, and is composed of a plurality of color components including the converted color component A conversion step of converting the image data into a single frame image of a color method having the above and an encoding step of encoding the image data converted by the conversion step by a predetermined compression method.

  According to the present invention, high-definition video can be handled efficiently without dispersing image data.

It is a figure which shows the function structure of the video recording / reproducing apparatus which concerns on embodiment. It is a figure which shows the function structure of the pre-processing part which concerns on embodiment. It is a figure which shows the function structure of the post-processing part which concerns on embodiment. It is a figure which shows the 1st example of the image conversion in the up-conversion part which concerns on embodiment. It is a figure which shows the 2nd example of the image conversion in the up-conversion part which concerns on embodiment. It is a figure which shows an example of the image conversion in the down conversion part which concerns on embodiment. It is a figure which shows an example of the image conversion in the image deformation part which concerns on embodiment. It is a figure which shows typically the procedure which simulated the video recording / reproducing apparatus which concerns on embodiment. It is a figure which shows typically the procedure which simulated the technique of the conventional recording / reproducing apparatus as a comparison object with the video recording / reproducing apparatus which concerns on embodiment. It is a figure explaining two methods of up-conversion concerning an embodiment. It is a 1st figure which shows the evaluation value of the simulation result which concerns on embodiment. It is a 2nd figure which shows the evaluation value of the simulation result which concerns on embodiment. It is a figure which shows an example of the image division in the conventional recording device.

Hereinafter, an example of an embodiment of the present invention will be described.
The video recording / reproducing apparatus 1 according to the present embodiment is a signal processing apparatus that records and reproduces a video signal. The video recording / playback apparatus 1 transforms, up-converts, or down-converts a single-frame image of a color system, which is used in a video signal and includes a plurality of color components having a predetermined sampling structure, with respect to at least one color component. And handle as a single frame image of a color system having different sampling structures.

FIG. 1 is a diagram showing a functional configuration of a video recording / reproducing apparatus 1 according to the present embodiment.
The video recording / reproducing apparatus 1 includes an input unit 11, a preprocessing unit 12, an encoding unit 13, a recording unit 14, a recording medium 15, a decoding unit 16, a postprocessing unit 17, and an output unit 18. Prepare.

The input unit 11 acquires a video signal from an external device.
The input video from the external device may be a video signal output from a CCU (Camera Control Unit) or other video processing device in addition to a full resolution or DG Super Hi-Vision camera.

  The pre-processing unit 12 performs at least one color component up-conversion, down-conversion, or image transformation on a single-frame image of a color system composed of a plurality of color components having a first sampling structure. Convert. As a result, the pre-processing unit 12 generates a color single frame image having a second sampling structure composed of a plurality of color components including the converted color component. Details of the preprocessing unit 12 will be described later.

  The encoding unit 13 encodes the image data converted by the preprocessing unit 12 by a predetermined compression method to compress the bit rate. For encoding video, a known standardized technique corresponding to the second sampling structure can be used.

  The recording unit 14 records the signal encoded by the encoding unit 13 on the recording medium 15. At the time of video reproduction, the recording unit 14 reads the encoded signal from the recording medium 15.

  The decoding unit 16 decodes the encoded signal read from the recording medium 15, and outputs the decoded single frame image signal of the color scheme having the second sampling structure to the post-processing unit 17.

  The post-processing unit 17 converts at least one color component of the color method single frame image having the second sampling structure decoded by the decoding unit 16 by any one of up-conversion, down-conversion, and image transformation methods. To do. As a result, the post-processing unit 17 is composed of a plurality of color components including the converted color components, and is input to the video recording / reproducing apparatus 1 and has a first sampling structure, or a first color-format single frame image. A single frame image of a color system having a sampling structure of 3 is generated. Details of the post-processing unit 17 will be described later.

  The output unit 18 outputs the video signal converted into the color system having the first sampling structure to an external device such as a monitor.

FIG. 2 is a diagram illustrating a functional configuration of the preprocessing unit 12 according to the present embodiment.
The pre-processing unit 12 includes a conversion method determination unit 121, an up-conversion unit 122, a down-conversion unit 123, an image transformation unit 124, and a color conversion processing unit 125.

The conversion method discriminating unit 121 discriminates whether the input video is to be converted by any one of up-conversion, down-conversion, and image transformation.
At this time, the conversion method determination unit 121 determines the sampling structure of the input video, and determines the conversion method according to the determined type of the first sampling structure.

Specifically, the conversion method determination unit 121 distributes the input video to the up-conversion unit 122, the down-conversion unit 123, or the image transformation unit 124 according to a preset conversion list. Note that an input video having a sampling structure determined to be unnecessary for conversion according to the conversion list is directly transferred to the color conversion processing unit 125.
Here, the conversion list may be uniquely determined in the video recording / playback apparatus 1 or may be arbitrarily selected by the user via an external interface or the like.
Moreover, the conversion method discrimination | determination part 121 may discriminate | determine a conversion method according to the compression mode (for example, high compression mode) designated by the user, for example.

  The up-conversion unit 122 up-converts the single frame image into the second sampling structure by interpolating the missing pixels in any color component. As the conversion method, for example, a known technique such as pixel interpolation or super-resolution technique may be used.

  The down-conversion unit 123 down-converts the single frame image into the second sampling structure by reducing the number of pixels of any color component. As a conversion method, for example, a known technique such as thinning out pixels or averaging a plurality of pixels may be used.

  The image deforming unit 124 deforms the single frame image into the second sampling structure by moving the position of the pixel of any color component. As the conversion method, a known technique such as replacement of a specific pixel or vector operation may be used.

  The color conversion processing unit 125 converts a video signal input as an RGB signal (full color signal) into a YPbPr signal or a YCbCr signal (color difference signal) that can be processed by the encoding unit 13. If the input image is a YPbPr signal or a YCbCr signal, the color conversion processing unit 125 outputs the image signal without converting it. If the encoding technique used by the encoding unit 13 is compatible with RGB signals, the conversion processing of the color conversion processing unit 125 is omitted.

FIG. 3 is a diagram illustrating a functional configuration of the post-processing unit 17 according to the present embodiment.
The post-processing unit 17 includes a color conversion processing unit 171, a conversion method determination unit 172, an up-conversion unit 173, a down-conversion unit 174, and an image transformation unit 175.

  The color conversion processing unit 171 converts the YPbPr signal or YCbCr signal decoded by the decoding unit 16 into an RGB signal.

The conversion method determination unit 172 determines the color method of the video signal output from the output unit 18. The color system at the time of output may be the first sampling structure similar to the video signal input by the input unit 11, or the user may arbitrarily select the third sampling structure.
The video signal received from the color conversion processing unit 171 is distributed to the up-conversion unit 173, the down-conversion unit 174, or the image transformation unit 175 according to this determination. Note that the video signal determined not to be converted is directly transferred to the output unit 18.

FIG. 4 is a diagram illustrating a first example of image conversion in the up-conversion unit 122 according to the present embodiment.
In this example, a DG signal having a resolution of 7680 × 4320 is input, and the up-conversion unit 122 detects missing pixels in the R and B regions of the image data 20 obtained by extracting only the G component for a single frame image of the DG signal. Interpolation generates image data 20a with high resolution.

  The up-conversion unit 122 packages the G component image data 20a, the R component image data 21 and the B component image data 22 to provide a color image having a resolution of 7680 × 4320 having a 4: 2: 0 sampling structure. Has been converted. In the case of the sampling structure, it is more preferable that the image converted into the color difference signal by the color conversion processing unit 125 can be consistent with many standardized known compression techniques.

FIG. 5 is a diagram illustrating a second example of image conversion in the up-conversion unit 122 according to the present embodiment.
In this example, a DG signal having a resolution of 7680 × 4320 is input, and the up-conversion unit 122 detects missing pixels in the R and B regions of the image data 30 obtained by extracting only the G component from the single frame image of the DG signal. Interpolation generates image data 20a with high resolution. Further, the up-conversion unit 122 generates high-resolution image data 31a and 32a by interpolating missing pixels in regions adjacent to each other in the vertical direction from the R component image data 31 and the B component image data 32, respectively. .

  The up-conversion unit 122 packages the G component image data 30a, the R component image data 31a, and the B component image data 32a, so that a color image having a resolution of 7680 × 4320 having a 4: 2: 2 sampling structure. Has been converted. In the case of the sampling structure, it is more preferable that the image converted into the color difference signal by the color conversion processing unit 125 can be consistent with many standardized known compression techniques.

FIG. 6 is a diagram illustrating an example of image conversion in the down-conversion unit 123 according to the present embodiment.
In this example, a full-resolution Super Hi-Vision signal is input, and the down-conversion unit 123 performs 2m + 1 columns of image data 41 obtained by extracting only the R component and the B component for a single frame image (RGB 4: 4: 4). Image data 41a and 42a with reduced resolution are generated by thinning out pixels of 2m + 2 columns of the image data 42 from which only the image data is extracted. Here, m represents an integer from 0 to 3839.

The down-conversion unit 123 packages the G component image data 40, the R component image data 41a, and the B component image data 42a, thereby providing a color image with a resolution of 7680 × 4320 having a 4: 2: 2 sampling structure. Has been converted. In the case of the sampling structure, it is more preferable that the image converted into the color difference signal by the color conversion processing unit 125 can be consistent with many standardized known compression techniques.
Note that the down-conversion unit 123 may thin out pixels in the common column (2m + 1 or 2m + 2) for the R component and the B component.

FIG. 7 is a diagram illustrating an example of image conversion in the image deforming unit 124 according to the present embodiment.
In this example, a DG signal with a resolution of 7680 × 4320 is input, and the image transformation unit 124 extracts the 2n + 1 and 2n + 2 rows of the image data 50 from which only the G component is extracted from the single frame image of the DG signal. By arranging the G component in the (n + 1) th row of the same column, the deformed image data 50a is generated. Similarly, for the R generation and the B component, the image deforming unit 124 arranges the components in the 2n + 1 row and the 2n + 2 row in the n + 1 row of the same column, so that the R component image data 51a and the B component respectively Image data 52a is generated. Here, n represents an integer from 0 to 2159.

  The image transformation unit 124 packages the G component image data 50a, the R component image data 51a, and the B component image data 52a, thereby providing a color image with a resolution of 3840 × 2160 having a 4: 2: 2 sampling structure. Has been converted. In the case of the sampling structure, it is more preferable that the image converted into the color difference signal by the color conversion processing unit 125 can be consistent with many standardized known compression techniques.

<Example>
Hereafter, the procedure and result of the simulation performed in order to show the effect of this embodiment are shown.
In this example, a high-definition image (resolution: 1980 × 1080, 8 bits) is used as an original image to be recorded and reproduced, but the same result is obtained when a super high-definition image or another high-definition image is input. I can expect.

  FIG. 8 is a diagram schematically showing a procedure for simulating the video recording / reproducing apparatus 1 according to the present embodiment.

(Procedure 1) The original image is sampled into a Bayer pattern (see FIG. 13).
(Procedure 2) Of the images sampled in (1), the G component is up-converted by interpolating the missing pixels in the R and B regions.
(Procedure 3) The RGB signals sampled by the 4: 2: 0 method according to (2) are encoded.
(Procedure 4) The encoded signal is decoded.
(Procedure 5) The G component of the decoded signal is sampled in a Bayer type.
(Procedure 6) PSNR (peak signal-to-noise ratio) is calculated and evaluated for the G component of the input image and the G component generated in (5).

  FIG. 9 is a diagram schematically showing a procedure for simulating a technique of a conventional recording / reproducing apparatus as a comparison target with the video recording / reproducing apparatus 1 according to the present embodiment.

(Procedure 1) Sampling the original image into a Bayer type.
(Procedure 2) Among the sampled images, the components of 2k + 1 rows and 2k + 2 rows of the G component are spatially divided and reconstructed into two images. Here, k is an integer from 0 to 539.
(Procedure 3) In addition to the two images obtained in (2), the R component and the B component are individually encoded.
(Procedure 4) The encoded signal is decoded.
(Procedure 5) Two decoded G images are combined and returned to the same configuration as the original image.
(Procedure 6) PSNR is calculated and evaluated for the G component of the input image and the G component generated in (5).

  In the above two simulations, JPEG2000 lossy compression was used for encoding. For G image up-conversion, two methods of Gradient-Linear and ACPI were used.

FIG. 10 is a diagram for explaining two methods of up-conversion according to the present embodiment.
In any of the methods, the equations in the case of interpolating the G component at the R pixel position are exemplified. In addition, when interpolating the G component at the position of the B pixel, R is read as B.

  In these methods, G based on the surrounding G component, or G component and R component, depending on the magnitude relationship between the value α indicating discontinuity in the vertical direction and the value β indicating discontinuity in the horizontal direction. The components are interpolated.

  11 and 12 are diagrams showing evaluation values of simulation results according to the present embodiment. The results of inputting two types of image data are shown in the respective drawings.

The horizontal axis represents the frame size of the encoded image, and the vertical axis represents the PSNR with the original image for the G component of the Bayer array. The smaller the frame size, the higher the compression ratio, and the larger the PSNR, the higher the reproducibility of the original image.
When the method according to the present embodiment (Gradient-Linear or ACPI) is used, PSNR degradation particularly when encoding at a high compression rate is improved compared to the conventional method (Current method).

  According to the present embodiment, the video recording / playback apparatus 1 treats an input video as a color-type single frame image having a sampling structure different from that at the time of input, thereby spatially converting a single frame image into a plurality of images. Without being divided, it can be recorded on the recording medium as a single image data. As a result, the video recording / playback apparatus 1 can efficiently handle high-definition video without dispersing image data, so that management of recording media and management and operation of recorded image data are facilitated.

  In addition, since the video recording / reproducing apparatus 1 can apply the encoding technique while maintaining the continuity of the original image, the compression efficiency is improved and the compression distortion is suppressed even when encoding is performed by an irreversible process. Is possible.

  Furthermore, since the image data recorded by the video recording / reproducing apparatus 1 can be handled as a single encoded signal having a standardized sampling structure in the recording medium, the recording data is not dispersed and standardized. The recorded data can be easily browsed, organized, and backed up using the efficient encoding and decoding techniques. In addition, since a timing synchronization circuit for synchronously reading data distributed on a plurality of recording media is not required at the time of reproducing a recorded video, it is possible to prevent the apparatus configuration from becoming complicated.

Further, the video recording / reproducing apparatus 1 can change the image conversion algorithm of the preprocessing unit 12 according to the color scheme of the input video signal, and input the video signal having the same sampling structure to the encoding unit 13. By doing in this way, since the same video coding technique can be applied in the encoding unit 13 and the decoding unit 16 to the color schemes of different input video signals, the circuit configurations of the encoding unit 13 and the decoding unit 16 are changed. A small video recording / playback apparatus 1 that can be simplified and can handle various input signals can be realized.
Further, since the video recording / reproducing apparatus 1 determines the conversion method according to the type of the sampling structure or according to the designated compression mode, it can automatically cope with a plurality of color methods, and the convenience is improved.

  Further, the video recording / playback apparatus 1 can realize a video editing machine by adding functions such as frame order change and color adjustment. Since this video editing machine does not require parallel processing to a plurality of recording media or a plurality of recording areas unlike a conventional recording apparatus, high-speed processing is possible.

  As mentioned above, although embodiment of this invention was described, this invention is not restricted to embodiment mentioned above. Further, the effects described in the present embodiment are merely a list of the most preferable effects resulting from the present invention, and the effects of the present invention are not limited to those described in the present embodiment.

In the above-described embodiment, the video recording / playback apparatus 1 having both the video recording function and the playback function has been described. However, the present invention is not limited to this.
For example, the decoding unit 16, the post-processing unit 17, and the output unit 18 may be separated as separate playback devices. Further, the recording unit 14 and the recording medium 15 may be separated as separate devices.

  The video recording / reproducing apparatus 1 described above has been mainly described as a DG system or full resolution super high-definition video recording / reproducing apparatus. It can be widely applied to imaging devices and the like.

  In the present embodiment, the configuration and operation of the video recording / playback apparatus have been mainly described. However, the present invention is not limited to this, and includes each component, and is configured as a method or program for recording and playing back video. Also good.

  Further, the present invention may be realized by recording a program for realizing the function of the video recording / reproducing apparatus on a computer-readable recording medium, causing the computer system to read and execute the program recorded on the recording medium. Good.

  The “computer system” here includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a portable medium such as a flexible disk, a magneto-optical disk, a ROM, and a CD-ROM, and a hard disk built in the computer system.

  Furthermore, “computer-readable recording medium” means that a program is dynamically held for a short time, like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. It is also possible to include one that holds a program for a certain time, such as a volatile memory inside a computer system that becomes a server or client in that case. Further, the program may be for realizing a part of the above-described functions, and may be capable of realizing the above-described functions in combination with a program already recorded in the computer system. .

1. Video recording / playback device (video recording device, video playback device)
11 Input unit 12 Pre-processing unit (first conversion unit)
13 Encoding Unit 14 Recording Unit 15 Recording Medium 16 Decoding Unit 17 Post-Processing Unit (Second Conversion Unit)
DESCRIPTION OF SYMBOLS 18 Output part 121 Conversion system discrimination | determination part 122 Up-conversion part 123 Down-conversion part 124 Image transformation part 125 Color conversion process part 171 Color conversion process part 172 Conversion system discrimination | determination part 173 Up-conversion part 174 Down-conversion part 175 Image transformation part

Claims (5)

The sampling structure of a dual green for a single frame image of lifting poked color scheme, the green color component and up-converting DOO, by image deformation the color components red and blue, 4: 2: 0 sampling structure A first conversion unit for converting into a single frame image of a color system having;
An image recording apparatus comprising: an encoding unit that encodes the image data converted by the first conversion unit by a predetermined compression method. The sampling structure of a dual green for a single frame image of lifting poked color scheme, green, by up-converting each color component of red and blue, 4: 2: single frame of a color system having a second sampling structure A first conversion unit for converting into an image;
An image recording apparatus comprising: an encoding unit that encodes the image data converted by the first conversion unit by a predetermined compression method. The sampling structure of a dual green for a single frame image of lifting poked color scheme, green, by each image deformation the color components red and blue, 4: 2: single frame of a color system having a second sampling structure A first conversion unit for converting into an image;
An image recording apparatus comprising: an encoding unit that encodes the image data converted by the first conversion unit by a predetermined compression method. The video recording apparatus according to any one of claims 1 to 3, a decoding unit for decoding the image picture data encoded by said predetermined compression method,
The image data decoded by said decoding unit includes green, the color components of the red and blue converted respectively, and a second conversion unit that converts the single frame image of a color scheme with a sampling structure of the dual green, a Video playback device. The video recording program for functioning a computer as a video recording device in any one of Claims 1-3 .

Images