JP2011091592A - Image encoder, code converter, image recorder, image reproduction device, image encoding method, and integrated circuit - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To encode a high-resolution image by an encoder corresponding conventional resolution, and to provide a reproduction image at a high frame rate with respect to performance of the encoder. <P>SOLUTION: Low-resolution images generated by repeating a process of reducing high-resolution images for M frames to be converted to low-resolution images, and dividing each frame of the next N frames into a plurality of frames to be converted to low-resolution images are encoded to generate a reduced compression image stream and a divided compression image stream. By decoding only the reduced compression image stream when performing decoding, content confirmation is enabled even by a decoder corresponding to conventional resolution. A high-resolution image equivalent to frames before and after a reduced image is restored by decoding and synthesizing the divided compression image stream, and a reproduction high-resolution image is obtained by converting the reduced image to a high-resolution image by using it. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image encoding device, a code conversion device, an image recording device, an image reproducing device, an image encoding method, and an integrated circuit, and in particular, an image recording device such as a video camera that encodes and records a high-resolution image. An image encoding device built in the image recording device, a code conversion device for converting high resolution image data encoded by the image encoding device into conventional resolution image data, and a high image recorded by the image recording device The present invention relates to an image reproducing apparatus that reproduces a resolution image.

  As a conventional image encoding apparatus, MPEG2 or H.264 is used. There is a device that encodes HDTV (High Definition Television) using a standard encoding method such as H.264. In these image encoding methods, high compression efficiency is realized by compressing and encoding an image signal using inter-frame prediction.

  Encoding HDTV with such an image encoding device has become common in recent years and has been put to practical use in digital broadcasting and video cameras. If this is further applied to a high-resolution image having the number of pixels of 4096 × 2160 or 3840 × 2160 (hereinafter abbreviated as 4K2K), it is necessary to significantly increase the processing speed. If an image recording apparatus that incorporates such an image encoding apparatus and captures a high-resolution image such as 4K2K is configured, it is necessary that not only the image encoding apparatus but also peripheral circuits support high-resolution images. In this respect as well, it is difficult to realize the apparatus, and the image recording apparatus becomes very expensive.

  As described above, a conventional image encoding apparatus that encodes a high-definition image such as 4K2K divides each frame of a high-resolution image into a plurality of small images, and each of the divided plurality of small images. There has been an image encoding device that performs high-speed processing by performing parallel processing on different encoding units (see, for example, Patent Document 1).

JP 2003-304539 A

  However, in the conventional image encoding device described in Patent Document 1, one frame is divided into a plurality of small images, and each is compressed in parallel by separate divided image compression / expansion units, so that the circuit scale increases. In addition, since there is a large amount of calculation processing for compression encoding, there is a problem that power consumption increases.

  Further, there is a problem that the compressed image stream is a high-definition image and has a unique data structure, and the content cannot be confirmed by a conventional standard compatible image reproducing apparatus that supports only a conventional resolution image. Furthermore, when a compressed image stream is converted into one having a conventional resolution, conversion from a unique data structure results in problems such as an increase in circuit scale and processing amount.

  Therefore, the present invention has been made in view of the above problems, and an image having a higher resolution than the first resolution is encoded by an encoding device that supports a first resolution such as 1920 × 1080. Another object of the present invention is to provide an image encoding apparatus and an image recording apparatus that enable reproduction at a high frame rate with respect to the performance of the encoding apparatus.

  In order to achieve the above object, the present image coding apparatus has an image signal having a first screen resolution that is continuously input in time and having a second screen resolution smaller than the first resolution. An encoding apparatus that performs encoding using an encoding unit capable of encoding an image signal, and reduces a frame included in the image signal having the first screen resolution that is continuously input in time. An image reducing unit that generates a reduced image with the second screen resolution, an image dividing unit that divides a frame included in the image signal into a divided image having the second screen resolution, and the image The reduction unit and the image division unit are controlled to generate a reduced image based on M (M is an integer) continuous frames among the frames included in the image signal, and output in time series The first process and the M frames A converted image is generated by alternately repeating a second process of generating a divided image based on N (N is an integer) consecutive frames and outputting the divided image in time series, and outputs the converted image An image signal control unit, a divided compressed image stream obtained by compressing and encoding the divided image and the reduced image into separate streams, using the converted image output from the image signal control unit as input, and compressing the divided image; And an encoding unit that outputs a reduced compressed image stream obtained by compressing the reduced image.

  According to this configuration, high-resolution images are converted into conventional low-resolution reduced images and divided images and encoded, thereby enabling encoding at a high frame rate with respect to the performance of the encoding device. At the same time, by using the reduced image and the divided image at the time of encoding, it is possible to improve the image quality as compared with the case of encoding all using the reduced image.

  The image signal control unit may output the number of frames M processed in the image reduction unit and the number of frames N processed in the image dividing unit as frame number information.

  According to this configuration, since the image decoding apparatus can input the M and N frame number information and perform corresponding decoding, the selection of the M and N frame numbers can be freely performed, There is an effect that it is possible to flexibly cope with the demands of reproduction image quality and circuit scale.

  The image signal control unit may output pixel position relationship auxiliary data indicating a correspondence relationship between the pixel position of the reduced image and the pixel position of the divided image.

  According to this configuration, when restoring a high resolution image corresponding to a reduced image from the high resolution image synthesized from the divided image and the reduced image, it is possible to restore the positional relationship of the pixels together. In addition, since the decoding side can acquire the positional relationship of the pixels set by the encoding side, it is possible to freely select the filter characteristics used when creating the reduced image.

  In addition, the encoding unit, when encoding the reduced image, the reduced compressed image whose frame rate is increased by inserting a picture composed of skipped macroblocks between the compressed image stream of the reduced image You may make it the structure which produces | generates a stream.

  According to this configuration, it is possible to change the frame rate of the reduced compressed image stream so as to comply with the conventional standard. As a result, by decoding only the reduced image, it is possible to check the content of the image even with an image decoding apparatus that supports only the conventional standard.

  The encoding unit may be configured to perform code amount control using a buffer size smaller than the buffer size in the standard of the encoding method in the code amount control when encoding the reduced image.

  According to this configuration, the high-resolution image obtained by combining the divided images is combined with the reduced compressed image stream, and the code amount control within the range of the conventional standard is performed while increasing the frame rate. The compressed image stream can be easily created.

  Further, the encoding unit adds stuffing to a picture made up of skipped macroblocks when encoding the reduced image, so that the data is approximately the same as other interframe predictive encoded pictures. It may be configured to generate a reduced-compression image stream having a predetermined amount.

  According to this configuration, a reduced version of the high-resolution image obtained by combining the divided images is replaced with a picture composed of skipped macroblocks, thereby increasing the frame rate and maintaining the range within the range of the conventional standard. A compressed image stream subjected to code amount control can be easily created.

  In addition, the image reduction unit reduces each of the N frames to generate a divided-part reduced image having a resolution smaller than the first resolution, and the encoding unit reduces the reduced compression image stream to the reduced compressed image stream. You may make it the structure characterized by performing compression encoding so that an image and the said division part reduction image may be included.

  According to this configuration, when a reduced image is encoded, a compressed image stream having a frame rate compatible with the conventional standard and capable of reproducing a smooth video is inserted without inserting a picture composed of skipped macroblocks. Can be easily created.

  Further, the encoding unit may be configured to perform inter-frame encoding processing when encoding the reduced image and to perform intra-frame encoding processing when encoding the divided image.

  According to this configuration, the reduced image can obtain a high compression rate by inter-frame coding and can be a compressed image stream conforming to the conventional standard, and the divided image from which the high-resolution image is restored is Even when the time difference is wide, encoding can be performed with high image quality by intra-frame encoding.

  Further, the image encoding apparatus may include a multiplexing unit that multiplexes the divided compressed image stream as user data of the reduced compressed image stream.

  According to this configuration, the divided compressed image stream is not recorded in a separate folder from the reduced compressed image stream, and the recording medium is compatible with a playback apparatus that only supports the conventional resolution by simply recording according to the conventional standard. Can be generated.

  Further, a configuration may be adopted in which a multiplexing unit that multiplexes the divided compressed image stream as a packet having another PID in the reduced compressed image stream.

  According to this configuration, the divided compressed image stream is not recorded in a separate folder from the reduced compressed image stream, and the recording medium is compatible with a playback apparatus that only supports the conventional resolution by simply recording according to the conventional standard. Can be generated. Further, since it is divided into packets different from the reduced compressed image stream, it is possible to decode only the divided compressed image stream without decoding the reduced compressed image stream as necessary.

  Further, a configuration may be provided in which a multiplexing unit that multiplexes the frame number information as user data on at least one of the divided compressed image stream and the reduced compressed image stream.

  According to this configuration, since it is not necessary to determine the ratio and period of the divided image and the reduced image in advance, the encoding device performs an encoding process that reproduces the optimum image quality in accordance with the characteristics of the input high-resolution image to be encoded. And can be shared by the decoding device.

  The frame number information may be configured to include a multiplexing unit that multiplexes at least one of the divided compressed image stream and the reduced compressed image stream as a packet having another PID.

  According to this configuration, since it is not necessary to determine the ratio and period of the divided image and the reduced image in advance, the encoding device performs an encoding process that reproduces the optimum image quality in accordance with the characteristics of the input high-resolution image to be encoded. And can be shared by the decoding device. Further, since the packet is separate from the compressed image stream, the decoding apparatus can know the information on the number of M and N frames without decoding the compressed image stream.

  The frame number information may be provided with a multiplexing unit that multiplexes at least one of the divided compressed image stream and the reduced compressed image stream as a descriptor in MPEG Systems.

  According to this configuration, since it is not necessary to determine the ratio and period of the divided image and the reduced image in advance, the encoding device performs an encoding process that reproduces the optimum image quality in accordance with the characteristics of the input high-resolution image to be encoded. And can be shared by the decoding device. In addition, since it is multiplexed with the descriptor used to multiplex other information, means for analyzing the descriptor originally present in the decoding apparatus can be diverted, and the circuit scale can be kept small. Can do.

  Further, a configuration may be adopted in which a multiplexing unit that multiplexes the pixel position relationship auxiliary data as user data on at least one of the divided compressed image stream and the reduced compressed image stream.

  According to this configuration, the processing contents of the reduction processing for generating a reduced image and the super-resolution processing for generating a high-resolution image at the time of decoding can be selected with a trade-off between circuit scale and image quality.

  Further, the pixel position relationship auxiliary data may be configured to include a multiplexing unit that multiplexes at least one of the divided compressed image stream and the reduced compressed image stream as a packet having another PID.

  According to this configuration, the processing contents of the reduction processing for generating a reduced image and the super-resolution processing for generating a high-resolution image at the time of decoding can be selected with a trade-off between circuit scale and image quality. Further, since the packet is separate from the compressed image stream, the decoding apparatus can know the pixel position relationship auxiliary data without decoding the compressed image stream.

  In addition, a configuration may be provided in which the pixel position relationship auxiliary data includes a multiplexing unit that multiplexes at least one of the divided compressed image stream and the reduced compressed image stream as a descriptor in MPEG Systems.

  According to this configuration, the processing contents of the reduction processing for generating a reduced image and the super-resolution processing for generating a high-resolution image at the time of decoding can be selected with a trade-off between circuit scale and image quality. In addition, since it is multiplexed with the descriptor used to multiplex other information, means for analyzing the descriptor originally present in the decoding apparatus can be diverted, and the circuit scale can be kept small. Can do.

  Also, the code conversion apparatus according to the present invention compresses a reduced image having a resolution smaller than the first resolution and an image signal having a resolution smaller than the first resolution and composed of skipped macroblocks. A divided compressed image obtained by compressing and coding a divided image obtained by dividing an encoded reduced compressed image stream and an image signal having the first resolution into a plurality of frames having a resolution smaller than the first resolution. An input unit that receives a stream, an expansion unit that restores an image signal having the first resolution before division based on a plurality of divided images included in the divided compressed image stream, and the expansion unit A reduction unit that reduces the image signal restored in step 1 to the resolution of the divided image and generates a reduced image after reproduction; and compression-encodes the reduced image after reproduction and performs compression after reproduction A compression unit that generates an image stream, and a converted compressed image stream generated by replacing each picture included in the reduced image stream after reproduction with an image signal including skipped macroblocks included in the reduced compressed image stream However, it may be configured to include an output code processing unit.

  According to this configuration, a compressed image stream obtained by compressing a high resolution image can be converted into a compressed image stream of a low resolution image with a small amount of calculation processing.

  The image encoding apparatus further includes a recording unit that records the divided compressed image stream and the reduced compressed image stream on a recording medium having a predetermined file system, and the recording unit includes the file system. Further, a divided image folder and a reduced image folder are generated, and the divided compressed image stream is recorded in the divided image folder, and the reduced compressed image stream and the reduced compression generated in the encoding unit are recorded. The reduced image control information used when reproducing the image stream may be recorded in the reduced image folder.

  According to this configuration, the reduced image is recorded in the folder structure compatible with the conventional standard, and the divided image is recorded in the folder structure of the part not in the conventional standard. In addition, only the reduced image can be reproduced.

  In addition, the image signal control unit generates a first reduced image group based on M (M is an integer) frames that are temporally continuous among frames included in the image signal. And generating a second reduced image group based on N (N is an integer) frames continuous to the M frames, and the encoding unit includes the first reduced image, the second reduced image, and the second reduced image group. The reduced image and the reduced compressed image stream are generated by arranging and compressing the reduced image in time series, and the reduced image control information used when reproducing the reduced compressed image stream is generated. Generating a divided image folder and a reduced image folder on the file system, and recording the divided compressed image stream in the divided image folder, the reduced compressed image stream, and A reduced image control information used in reproducing the reduced compressed image stream generated in serial encoding unit may be configured, characterized in that recorded in the reduced image folder.

  According to this configuration, a high-resolution image having a resolution exceeding the conventional standard can be encoded and recorded with a small circuit scale and an arithmetic processing amount. Furthermore, since it is necessary to reproduce only one of the reduced compressed image stream and the divided compressed image stream at the time of reproduction, the calculation processing amount can be further reduced. Further, even if the recording medium recorded with this configuration is installed in an image reproducing apparatus that supports only the conventional resolution, an image with the conventional resolution can be reproduced.

  Further, the encoding unit compresses the divided image of the image signal having the first resolution before the reduction of the second reduced image by using the motion vector detected when the second reduced image is encoded. You may make it the structure characterized by encoding.

  According to this configuration, since the motion vector of the entire divided image or a large area (hereinafter referred to as the entire motion vector) can be obtained from the reduced image, the divided image can be compressed without adding a circuit for detecting the entire motion vector. The processing efficiency can be increased and the quality of the high-resolution image during reproduction can be improved.

  Further, the present image reproducing apparatus is configured by a reduced image having a resolution smaller than the first resolution recorded on a removable recording medium, and a skipped macroblock having a resolution smaller than the first resolution. A compressed image stream obtained by compressing and encoding the image signal and a divided image obtained by dividing the image signal having the first resolution into a plurality of frames having a resolution smaller than the first resolution. A read unit that reads out the encoded divided compressed image stream from the recording medium, and restores the image signal before reduction based on the reduced image included in the reduced compressed image stream, and is included in the divided compressed image stream A decompression unit that restores an image signal before division from the divided image, and an output unit that outputs the restored image signal in time series

  In addition, the decompression unit uses the image signal restored from the divided image among the image signals output before and after temporally when the reduced image related to restoration is output in time series, and the reduction related to the restoration is performed. You may make it the structure characterized by restoring an image.

  Further, according to the present image encoding method, an image signal having a first screen resolution that is input continuously in time is encoded with an image signal having a second screen resolution smaller than the first resolution. An encoding method that performs encoding using a possible encoding unit, wherein a frame included in an image signal having the first screen resolution that is continuously input in time is reduced to reduce the second An image reduction step for generating a reduced image with a screen resolution, an image division step for dividing a frame included in the image signal into a divided image having the second screen resolution, the image reduction step, and the image A first step of controlling the division step, generating a reduced image based on M (M is an integer) frames that are temporally continuous among the frames included in the image signal, and outputting the reduced image in time series; M pieces A converted image is generated by alternately repeating a second process of generating a divided image based on N (N is an integer) continuous frames and outputting it in time series, and outputs the converted image A divided compressed image stream obtained by compressing and encoding the divided image and the reduced image into separate streams, the image signal control step and the converted image output from the image signal control unit as inputs, and compressing the divided image; And an encoding step for outputting a reduced compressed image stream obtained by compressing the reduced image.

  According to this configuration, a high-resolution image having a high frame rate with respect to the performance of the encoding device or the decoding device can be reproduced using the decoding device that decodes the compressed image stream having the conventional resolution. . In addition, a recording medium recorded by an image recording apparatus compliant only with the conventional standard can be reproduced as usual.

  In addition, the present invention can be realized not only as such an image encoding device but also as an integrated circuit including characteristic components of the image encoding device. Further, the present invention can be realized as an image encoding method in which the operation of a characteristic component included in the image encoding device is a step, or can be realized as a program that causes a computer to execute such a characteristic step. You can also Such a program can be distributed via a recording medium such as a CD-ROM (Compact Disc-Read Only Memory) and a transmission medium such as the Internet.

  As described above, the present invention enables encoding and recording with a small circuit scale and calculation processing amount, and further encodes a high-resolution image with an encoding device compatible with the conventional resolution, and It is possible to provide an image encoding device and an image recording device that enable reproduction at a high frame rate with respect to performance.

1 is a perspective view showing an external appearance of an image recording / reproducing apparatus using an image encoding apparatus according to Embodiment 1. FIG. The figure which shows the structure of the image recording / reproducing apparatus using the image coding apparatus which concerns on this Embodiment 1. FIG. The figure which shows each part signal in the image recording / reproducing apparatus using the image coding apparatus which concerns on this Embodiment 1. The figure which shows the example of a file structure currently recorded on the recording medium which concerns on this Embodiment 1. Flowchart showing the flow of playback operation by the image recording / playback apparatus according to the first embodiment. The figure explaining the relationship of the files below the BDMV main folder in the file structure of the recording medium concerning this Embodiment 1. The figure which shows each part signal in the high-resolution reproduction mode of the image recording / reproducing apparatus which concerns on this Embodiment 1. FIG. The figure which shows the example of the relative relationship of the pixel position of a 4K2K image and a reduction image. The figure which shows each part signal in the image recording / reproducing apparatus using the image coding apparatus which concerns on this Embodiment 2. FIG. The figure which shows the structural example of the compressed image stream which concerns on this Embodiment 2. The figure which shows the structural example of PES of MPEG-2 specification The figure which shows the structural example of TS of MPEG-2 specification The figure which shows the structural example of the image conversion apparatus which concerns on this Embodiment 3. A 4K2K high-resolution image output from the memory control unit 501 to the image reduction unit 506, a reduced compressed image stream recorded on the recording medium 10, and a conversion compression of 24 frames per second recorded on the recording medium 20 by the recording processing unit 508. Diagram explaining the relationship with the image stream The figure which shows the image of the buffer control in the compression / decompression part 205 of the image recording / reproducing apparatus using the image coding apparatus which concerns on this Embodiment 4. FIG. The figure which shows the image of the buffer control in the compression / decompression part 205 of the image recording / reproducing apparatus using the image coding apparatus which concerns on this Embodiment 5. FIG. The figure which shows the structure of the image recording / reproducing apparatus 1002 using the image coding apparatus which concerns on this Embodiment 6. FIG. The figure which shows each part signal at the time of the recording in the image recording / reproducing apparatus 1002 using the image coding apparatus based on this Embodiment 6. FIG. The figure which shows the process at the time of reproducing | regenerating the recording medium 10 recorded by the image recording / reproducing apparatus 1002 using the image coding apparatus based on this Embodiment 6. FIG.

  Embodiments of an image encoding device according to the present invention will be described below in detail with reference to the drawings.

(Embodiment 1)
FIG. 1 is a perspective view showing an appearance of an image recording / reproducing apparatus using the image encoding apparatus according to the first embodiment. As shown in FIG. 1, the image recording / reproducing apparatus 1000 is, for example, a digital video camera. The image recording / reproducing apparatus 1000 has a function as an image recording apparatus that records a captured high resolution image on the recording medium 10. The image recording / reproducing apparatus 1000 has a function as an image reproducing apparatus that reproduces a high-resolution image recorded on the recording medium 10. Here, for the sake of explanation, it is assumed that the high-resolution image is a 4K2K image having 3840 × 2160 pixels.

  The recording medium 10 is a recording medium that can be attached to and detached from the image recording / reproducing apparatus 1000, and is, for example, a semiconductor memory or an optical disk.

  FIG. 2 is a diagram showing a configuration of an image recording / reproducing apparatus 1000 using the image encoding apparatus according to the first embodiment. As shown in FIG. 2, the image recording / reproducing apparatus 1000 includes a camera unit 100, a recording / reproducing unit 200, an external memory 300, an output terminal 301, a display device 302, a super-resolution processing unit 320, and an external memory. 330.

  The image reduction unit 110 built in the camera unit 100 corresponds to the image reduction unit of claim 1. Similarly, the image dividing unit 111 corresponds to the image dividing unit of claim 1, and the camera image signal processing unit 104 corresponds to the image signal control unit of claim 1. A compression / decompression unit 205 built in the recording / reproducing unit 200 corresponds to the encoding unit of claim 1.

  Hereinafter, the camera unit 100 will be described.

  The camera unit 100 includes a lens group 101, an imaging unit 102, an A / D conversion unit 103, a camera image signal processing unit 104, and a camera control unit 105.

  The lens group 101 includes a plurality of optical lenses. The lens group 101 collects light on the imaging unit 102.

  The imaging unit 102 is configured by an imaging element or the like, and images light input via the lens group 101. Specifically, the imaging unit 102 converts the input optical signal into an analog signal (electric signal), and outputs the analog signal to the A / D conversion unit 103.

  The A / D conversion unit 103 converts the analog signal output from the imaging unit 102 into a digital signal. The A / D conversion unit 103 outputs the converted digital signal to the camera image signal processing unit 104.

  The camera image signal processing unit 104 performs image signal processing such as noise removal and image quality adjustment on the digital signal output from the A / D conversion unit 103 to generate an image signal. The camera image signal processing unit 104 generates a 4K2K image signal of 24 frames per second by performing image signal processing on the digital signal. In addition, the camera control unit 105 controls each part of the internal configuration of the camera unit 100 while exchanging control signals with the recording / playback unit 200 to capture a 4K2K image signal.

  The camera image signal processing unit 104 in the present embodiment includes an image reduction unit 110 and an image division unit 111. These operations will be described with reference to FIG.

  FIG. 3 is a diagram illustrating signals in the image recording / reproducing apparatus 1000 using the image encoding apparatus according to the first embodiment.

  FIG. 3A shows a 4K2K image signal generated by the camera image signal processing unit 104.

  FIG. 3B shows an image signal output from the camera image signal processing unit 104 to the memory control unit 201.

  FIG. 3C shows a reduced compressed image stream generated by compressing the reduced image in the compression / decompression unit 205.

  FIG. 3D shows a divided compressed image stream generated by compressing the divided images in the compression / decompression unit 205.

  FIG. 3E shows a divided compressed image stream output from the compression / decompression unit 205 by inserting a P picture composed of skipped macroblocks into the reduced compression image stream in the compression / decompression unit 205.

  The image reduction unit 110 and the image division unit 111 are alternately supplied with the 4K2K image signal shown in FIG. 3A every other frame.

  The image reducing unit 110 reduces the input frame 350 of the 4K2K image to 1/2 in the horizontal direction and 1/2 in the vertical direction to the recording / reproducing unit 200 as a reduced image 360 of HDTV size having the number of pixels of 1920 × 1080. Output.

  The image dividing unit 111 divides the input frame 351 of the 4K2K image into, for example, four divided images 361, 362, 363, and 364 having a horizontal pixel number of ½ and a vertical pixel number of ½, The image is output to the recording / reproducing unit 200 as an HDTV-sized divided image having a number of pixels of 1920 × 1080.

  Although the 4K2K image signal is alternately supplied to the image reduction unit 110 and the image division unit 111 every other frame, for example, after the M frame is supplied to the image reduction unit 110 continuously, N continues to the M frame. A configuration may be adopted in which frames are supplied to the image dividing unit 111. Here, M and N are each positive integers.

  Here, the camera image signal processing unit 104 time-division-multiplexes the reduced image 360 and the divided images 361, 362, 363, and 364 as the HDTV size image format of 60 frames per second shown in FIG. This is output to the recording / playback unit 200 (hereinafter referred to as HDTV size multiplexed image).

  Next, the recording / reproducing unit 200 will be described.

  The recording / playback unit 200 includes a memory control unit 201, an image output unit 202, an image reduction unit 203, a GUI multiplexing unit 204, a compression / decompression unit 205, a recording / playback processing unit 206, a system control unit 207, a recording And a control unit 208.

  Under the control of the system control unit 207, the memory control unit 201 extracts a reduced image from the HDTV size multiplexed image output from the camera image signal processing unit 104, and temporarily stores the reduced image in the reduced image area 310 on the external memory 300. If necessary, the data is read out and output to the image output unit 202, the image reduction unit 203, and the compression / expansion unit 205. Further, the memory control unit 201 extracts a divided image from the HDTV size multiplexed image output from the camera image signal processing unit 104 under the control of the system control unit 207 and temporarily stores the divided image in the divided image region 311 on the external memory 300. The data is stored, read as necessary, and output to the image output unit 202 and the compression / decompression unit 205.

  Here, when outputting the reduced image and the divided image to the image output unit 202 and the compression / decompression unit 205, the memory control unit 201 outputs the HDTV size multiplexed image shown in FIG.

  The image output unit 202 outputs the HDTV size multiplexed image output from the memory control unit 201 via the output terminal 301 after performing image processing such as edge enhancement and noise cancellation. Therefore, the image output unit 202 does not need to support 4K2K images, and a circuit corresponding to conventional HDTV images can be used.

  The image reduction unit 203 further reduces the reduced image signal output from the memory control unit 201 according to the resolution of the display device 302, and outputs the reduced image signal to the GUI multiplexing unit 204. At this time, the reduced image signal is a signal of 12 frames per second. However, when reading from the reduced image area 310 on the external memory 300, the reduced image signal is repeatedly read as necessary to meet the frame rate required by the display device 302. Can be output as a video signal of a different format. Similarly to the image output unit 202, the image reduction unit 203 does not need to support 4K2K images, and a circuit corresponding to a conventional HDTV image can be used.

  The GUI multiplexing unit 204 multiplexes information such as camera information and recording medium information as a graphic on the reduced display image output from the image reduction unit 203 under the control of the system control unit 207, and outputs the multiplexed image to the display device 302. .

  The compression / decompression unit 205 converts the HDTV size multiplexed image output from the memory control unit 201 to MPEG-2 or H.264. Based on an encoding method such as H.264, the data is compressed and multiplexed. The compression / decompression unit 205 outputs the compressed image stream obtained by the compression multiplexing to the recording / playback processing unit 206.

  At this time, since the compression / decompression unit 205 compresses the HDTV size multiplexed image, the performance as the encoding device only needs to be sufficient to compress the HDTV size image signal of 60 frames per second. However, under the control of the system control unit 207, the reduced image and the divided image in the input HDTV size multiplexed image are distinguished, and compression is performed by switching the internal processing so that the compression processing becomes independent.

  When compressing the reduced image, the compression processing is performed using the inter-frame reference of only the reduced images (for example, the frame 360 and the frame 365) in the HDTV size multiplexed image shown in FIG. In this case, if the inter-frame compression process is performed using only the I picture and the P picture without using the B picture, the order of the frames to be compressed becomes the same as the order of the input frames, so that the control does not have to be complicated.

  The compressed image stream generated from the reduced image is in the format as shown in FIG. 3C. Here, in order to make the stream compliant with the conventional standard, a P picture composed of skipped macroblocks between each frame is used. (Hereinafter abbreviated as skip P picture) is inserted.

  For example, H. When compression based on H.264 is performed, a macroblock called a skipped macroblock that uses a pixel of a prediction signal that has been motion-guaranteed using a prediction motion vector as it is as an encoded image without sending any macroblock information・ Mode is available.

  By composing one P picture with only this skipped macroblock, an image in which the previous picture is motion-compensated with a predicted motion vector is used as an encoded image as it is, and a compressed image stream for one picture without any compression processing. Can be generated. It should be noted that pictures that can be configured only by skipped macroblocks can be configured not only by P pictures but also by B pictures.

  By inserting this skip P picture, the compression / decompression unit 205 generates a pseudo 24p-format compressed image stream shown in FIG. 3E, and separates it from the compressed image stream of the divided images for recording / reproduction processing. The data is output to the unit 206. Therefore, the compression / decompression unit 205 performs compression processing so that the reduced compressed image stream including the skip P picture conforms to the conventional standard including rate control.

  Similarly, when compressing divided images, compression processing is performed using inter-frame references between divided images at the same position in the 4K2K image in the HDTV size multiplexed image shown in FIG. Do. For example, the divided image 352 (4K1-a) located at the upper left can be referred to only from the divided image 353 (4K3-a) at the same position. The compression / decompression unit 205 generates a compressed image stream shown in FIG. 3D from the divided images, separates the compressed image stream from the reduced image, and outputs it to the recording / playback processing unit 206.

  The recording / playback processing unit 206 records the compressed image stream output from the compression / expansion unit 205 on the recording medium 10.

  The recording / playback processing unit 206 records the reduced compressed image stream and the divided compressed image stream output from the compression / expansion unit 205 in separate folders formed on the recording medium 10.

  The system control unit 207 controls the memory control unit 201 by receiving information on the order of reduced images and divided images in the HDTV size multiplexed image output from the camera image signal processing unit 104 via the camera control unit 105. To do.

  Further, the system control unit 207 uses a user operation received via an input device not described as operation information, and based on the operation information, the camera control unit 105, the memory control unit 201, the image output unit 202, the image reduction unit 203, By controlling the GUI multiplexing unit 204, compression / decompression unit 205, recording / playback processing unit 206, and recording control unit 208, recording processing and playback processing in the image recording / playback apparatus 1000 are executed.

  The recording control unit 208 controls a drive protocol for driving the recording medium 10 or a recording protocol such as a recording address input procedure.

  FIG. 4 is a diagram showing an example of a file configuration recorded on the recording medium 10 according to the first embodiment.

  As shown in FIG. 4, a BDMV main folder 410 and a HIGHRESO main folder 430 are formed at the highest level of the directory structure in one of the file systems recorded on the recording medium 10.

  The BDMV main folder 410 is a folder for conventional compatibility, and is recorded by a conventional resolution image, that is, a reduced-compression image stream from a 4K2K high-resolution image or an image recording apparatus only compatible with the conventional standard, in a recording format corresponding to the conventional standard. The compressed image stream is recorded.

  In the following description of the BDMV main folder 410, a reduced image from a 4K2K high resolution image and a conventional resolution image recorded by an image recording apparatus that supports only the conventional standard are collectively expressed as a conventional resolution image. To do.

  The HIGHRESO main folder 430 is a high-resolution shooting folder, and records a divided compressed image stream from 4K2K in a unique recording format.

  Under the BDMV main folder 410, an info file 411 (info), a menu file 412 (menu), a playlist folder 413 (PLAYLIST), a clip info folder 414 (CLIPINF), a stream folder 415 (STREAM), A backup folder 416 (BACKUP) is formed.

  The info file 411 (info) includes management information for the entire directory.

  The menu file 412 (menu) includes information constituting the menu.

  A playlist file 417 (01001.plst) is stored under the playlist folder 413 (PLAYLIST). The playlist file 417 (01001.plst) includes information for reproducing a conventional resolution image. Specifically, the playlist file 417 includes information related to the content playback order. The playlist file 417 includes information specifying the clip info file 418 and a playback time.

  A clip info file 418 (01000.clpi) is stored under the clip info folder 414 (CLIPINF). The clip info file 418 (01000.clpi) includes information for reproducing a conventional resolution image. Specifically, the clip info file 418 indicates correspondence between time information and the position of the compressed image stream (stream file 419). The clip info file 418 includes a time table for realizing partial reproduction and special reproduction of the corresponding compressed image stream. The clip info file 418 includes information such as the image size of the corresponding compressed image stream and the compression method.

  A stream file 419 (01000.m2ts) is stored under the stream folder 415 (STREAM). The stream file 419 (01000.m2ts) includes an AV data file.

  In the backup folder 416 (BACKUP), for example, a copy of the info file 411 (info), the menu file 412 (menu), the playlist folder 413 (PLAYLIST), and the clip info folder 414 (CLIPINF) during the editing operation is saved. The As a result, it is possible to prevent the conventional resolution image from being rendered unreproducible due to power interruption or the like during editing.

  Under the HIGHRESO main folder 430, a playlist folder 433 (EXTLAYLIST), a clip info folder 434 (EXTCLIPINF), a stream folder 435 (EXTSTREAM), and a backup folder 436 (BACKUP) are formed.

  A playlist file 437 (01001.plse) is stored under the playlist folder 433 (EXTPLALIST). The playlist file 437 (01001.plse) includes information for reproducing the divided compressed image stream. Specifically, the playlist file 437 includes information regarding the playback order of image frames or fields related to the divided compressed image stream. The playlist file 437 includes information specifying the clip info file 438 and a playback time.

  A clip info file 438 (01000.clpe) is stored under the clip info folder 434 (EXTCLIPINF). The clip info file 438 (01000.clpe) includes information for reproducing the divided compressed image stream. Specifically, the clip info file 438 shows the correspondence between the time information and the position of the compressed image stream (stream file 439), and the correspondence between the divided image and the position in the original 4K2K high resolution image. The clip info file 438 includes a time table for realizing partial reproduction and special reproduction of the corresponding compressed image stream. The clip info file 438 includes information such as the image size of the corresponding compressed image stream and the compression method.

  A stream file 439 (01000.m2te) is stored under the stream folder 435 (EXTSTREAM). The stream file 439 (01000.m2te) includes a divided compressed image stream file.

  In the backup folder 436 (BACKUP), for example, a copy of the playlist folder 433 (PLAYLIST) and the clip info folder 434 (CLIPINF) during the editing operation is stored.

  Further, the BDMV main folder 410 and the HIGHRESO main folder 430 have the same folder hierarchy configuration.

  As described above, the image recording / reproducing apparatus 1000 according to Embodiment 1 records a reduced compressed image stream having a resolution of 1920 × 1080 conforming to the conventional standard in the BDMV main folder 410 and below, and the divided compressed image stream as HIGHRESO. Record in the main folder 430 and below.

  By recording in this way, the file structure under the BDMV main folder 410 becomes the same as the folder structure of the conventional standard. As a result, even when an image reproducing apparatus conforming to a conventional standard that does not support high-resolution imaging reproduces the recording medium 10, it can reproduce a reduced image having a resolution of 1920 × 1080. Furthermore, functions such as playlist editing can be used as usual in an image reproduction device compliant with the conventional standard.

  Next, a reproduction operation by the image recording / reproducing apparatus 1000 according to Embodiment 1 will be described.

  FIG. 5 is a flowchart showing the flow of the reproducing operation by the image recording / reproducing apparatus 1000. Here, the image recording / reproducing apparatus 1000 normally reproduces only the reduced image from the recording medium 10 on which the conventional resolution image is recorded or the recording medium 10 on which the high resolution image is recorded by the image recording / reproducing apparatus compliant with the conventional standard. It is assumed that a playback mode and a high resolution mode in which both a reduced image and a divided image are played back from the recording medium 10 on which a high resolution image is recorded by the image recording / playback apparatus 1000 are provided.

  When the recording medium 10 on which a high-resolution image is recorded by the image recording / reproducing apparatus 1000 is reproduced by an image reproducing apparatus compliant only with the conventional standard, the same operation as in the normal reproduction mode described here is performed, and the reduction is performed. Play and output images.

  First, the system control unit 207 determines the playback mode set by the user (S220).

  When the playback mode is the normal playback mode (No in S220), the recording / playback processing unit 206 is recorded by a conventional resolution image to be played back, that is, a reduced compressed image stream from 4K2K or an image recording device that supports only the conventional standard. The compressed image stream is read from the recording medium 10 (S211).

  FIG. 6 is a diagram for explaining the relationship between files under the BDMV main folder 410 in the file structure of FIG.

  In the normal playback mode, the system control unit 207 uses three types of files: a playlist file 417, a clip info file 418, and a stream file 419 for playback control.

  The playlist file 417 includes information specifying the clip info file 418 and a playback time. The recording / playback processing unit 206 sequentially refers to the clip info file 418 included in the playlist file 417 to be played back. The recording / playback processing unit 206 reads the stream file 419 specified by the time table included in the clip info file 418.

  For example, the playlist file 417 (01001.plst) includes a clip (# 1), a clip (# 2), and the like, and is reproduced in the order of the clip (# 1) and the clip (# 2). Indicated. The recording / playback processing unit 206 reads the stream file 419 (01000.m2ts) associated with the clip (# 1) by the clip info file 418 (01000.clpi). For the clip (# 2), the recording / playback processing unit 206 reads the stream file 419 (02000.m2ts) associated with the clip info file 418 (02000.clpi).

  The compression / decompression unit 205 separates and decompresses the conventional resolution image read by the recording / playback processing unit 206 (S212).

  The memory control unit 201 temporarily stores the conventional resolution image separated and expanded by the compression / decompression unit 205 in the external memory 300 and outputs the image to the image output unit 202 and the image reduction unit 203. The image output unit 202 performs image processing such as edge enhancement and noise cancellation on the conventional resolution image output from the memory control unit 201, and then outputs the image via the super-resolution processing unit 320 and the output terminal 301. (S213). At this time, the super-resolution processing unit 320 does not process the input image and outputs it to the output terminal 301 as it is.

  The image reduction unit 203 reduces the conventional resolution image output from the memory control unit 201 according to the resolution of the display device 302 and outputs the reduced image to the GUI multiplexing unit 204. The GUI multiplexing unit 204 multiplexes information such as camera information and recording medium information as graphics in the conventional resolution image output from the image reduction unit 203 under the control of the system control unit 207, and outputs it to the display device 302.

  As described above, in the normal playback mode, the image recording / reproducing apparatus 1000 converts a compressed image stream conforming to the conventional standard or a reduced compressed image stream corresponding to 12 frames per second reduced from 4K2K into a format of 24 frames per second. The file is read from the BDMV main folder 410 and reproduced.

  The file structure below the BDMV main folder 410 is the same as the folder structure of the conventional standard. As a result, even when an image reproducing apparatus conforming to a conventional standard that does not support high-resolution imaging reproduces the recording medium 10, it can reproduce an image in a 24-frame format per second (the content is equivalent to 12 frames per second). Furthermore, functions such as playlist editing can be used as usual in an image reproduction device compliant with the conventional standard.

  On the other hand, when the playback mode is the high resolution mode (Yes in S220), the system control unit 207 determines whether or not the divided compressed image stream corresponding to the reduced image from 4K2K to be played back is recorded on the recording medium 10. (S221).

  In the high-resolution playback mode, the system control unit 207 refers to the playlist file 437 below the HIGHRESO main folder 430 in addition to the playlist file 417 below the BDMV main folder 410.

  In the case of the high resolution shooting mode, the recording / playback processing unit 206 records the divided compressed image stream corresponding to one clip of the reduced image from 4K2K in the HIGHRESO main folder 430. In the divided image playlist file 437, a clip information file 438 of divided images that can be used at the time of reproduction corresponding to a reduced image from 4K2K and a reproduction time are described.

  The recording / playback processing unit 206 first refers to a reduced image playlist file 417 and a divided image playlist file 437 from 4K2K.

  Hereinafter, a case where the first clip (# 1) included in the playlist file 417 is reproduced will be described. Here, the clip corresponding to the clip (# 1) does not exist in the playlist file 437 of divided images. That is, the clip (# 1) is a compressed image stream recorded in the normal shooting mode or a compressed image stream recorded by a conventional standard image recording apparatus that does not support high-resolution shooting.

  Since the divided compressed image stream is not recorded (No in S221), the image recording / reproducing apparatus 1000 performs the same processing (S211 to S213) as in the normal reproduction mode. Note that if the divided compressed image stream corresponding to the recording medium 10 is not recorded (No in S221), the system control unit 207 may display the content of the error to the user and end the reproduction process.

  Next, a process for playing back the clip (# 2) will be described. A clip (# 21) corresponding to the clip (# 2) exists in the playlist file 437 of the divided compressed image stream. That is, the clip (# 2) is a compressed image stream recorded in the high resolution shooting mode.

  Since the divided compressed image stream is recorded (Yes in S221), the recording / playback processing unit 206 reads from the recording medium 10 the reduced image from 4K2K to be played back and the divided compressed image stream related to the reduced image. (S222). Specifically, the recording / playback processing unit 206 reads the stream file 439 (02000.m2te) of the divided compressed image stream associated with the clip info file 438 (02000.clpe). In addition, the recording / playback processing unit 206 reads a reduced image stream file 419 (02000.m2ts) associated with the clip info file 418 (02000.clpi).

  Next, the compression / decompression unit 205 separates and decompresses the reduced compressed image stream and the divided compressed image stream read by the recording / playback processing unit 206 (S222). At this time, the system control unit 207 controls the recording / playback processing unit 206, the compression / decompression unit 205, and the memory control unit 201, and the HDTV size multiplexed image (FIG. 3 (b) in FIG. ) As an HDTV size image format of 60 frames per second as shown in FIG. 5), and outputs only a reduced image to the image reduction unit 203.

  The image reduction unit 203 reduces the reduced image output from the memory control unit 201 according to the resolution of the display device 302 and outputs the reduced image to the GUI multiplexing unit 204. The GUI multiplexing unit 204 multiplexes information such as camera information and recording medium information as a graphic on the image output from the image reduction unit 203 under the control of the system control unit 207, and outputs it to the display device 302. Accordingly, the processing contents of the image reduction unit 203 and the GUI multiplexing unit 204 are the same in both the normal reproduction mode and the high resolution reproduction mode.

  The image output unit 202 outputs the HDTV size multiplexed image output from the memory control unit 201 to the super-resolution processing unit 320 after performing image processing such as edge enhancement and noise cancellation. Therefore, the image output unit 202 does not need to support 4K2K images, and a circuit corresponding to conventional HDTV images can be used.

  The super-resolution processing unit 320 temporarily stores the input HDTV size multiplexed image in the external memory 330, then combines the divided images to restore a 4K2K high-resolution image, and the high-resolution image and the reduced image are combined. By using this, a 4K2K high-resolution image corresponding to the reduced image is generated.

  FIG. 7 is a diagram illustrating signals of each part of the image recording / reproducing apparatus 1000 in the high-resolution reproduction mode.

  FIG. 7A shows a divided compressed image stream recorded under the BDMV main folder 410 of the recording medium 10.

  FIG. 7B is a diagram showing a reduced compressed image stream recorded under the HIGHRESO main folder 430 of the recording medium 10.

  FIG. 7C is a diagram showing an HDTV size multiplexed image output from the image output unit 202.

  FIG. 7D shows an HDTV size multiplexed image stored in the external memory 330.

  FIG. 7E is a diagram illustrating a 4K2K high-resolution image generated by the super-resolution processing from the reduced image and the divided image in the super-resolution processing unit 320.

  The compression / decompression unit 205 separates and decompresses the divided compressed image stream shown in FIG. 7 (a) and the reduced compressed image stream shown in FIG. 7 (b) read by the recording / playback processing unit 206, respectively. Output to the control unit 201.

  The memory control unit 201 temporarily stores the decompressed compressed image and the divided image in the external memory 300, and then temporally multiplexes the HDTV size multiplexed image shown in FIG. The image is output to the image processing unit 320. At this time, as shown in FIG. 7, the decoded image of the skip P picture (for example, the frame 370 and the frame 371) is not multiplexed on the HDTV size multiplexed image.

  The super-resolution processing unit 320 temporarily stores the input HDTV size multiplexed image in the external memory 330. At this time, the divided images are stored in the memory in the order of addresses applied to the 4K2K high-resolution image, and read to restore the image in which the reduced image and the high-resolution image shown in FIG. The

  The super-resolution processing unit 320 reads out the image signal shown in FIG. 7D from the external memory 330, and uses a repetitive signal of the high-resolution image and the reduced image, and uses a high resolution of 4K2K corresponding to the reduced image. Generate an image.

  For example, after a reduced image (frame 380) is enlarged to a 4K2K image size by linear interpolation, it is divided into motion detection blocks every 16 × 16 pixels, and the horizontal and vertical coordinates are close to each other on the high resolution image (frame 381). The motion vector detection process is executed with respect to the 16 × 16 pixel block in FIG.

  As a result of the motion vector detection process, a motion detection block having a high correlation between the motion detection block of the reduced image and the 16 × 16 pixel block on the high-resolution image indicated by the motion vector has a high value indicated by the motion vector. Copy 16 × 16 pixels on the resolution image. On the other hand, for a motion detection block with low correlation, a high-resolution image 390 corresponding to the reduced image 380 is generated by using the motion detection block generated by enlarging the reduced image by linear interpolation.

  By performing the processing described above, a portion with a small amount of motion between the reduced image 380 and the high-resolution image 381 originally uses high-resolution pixel data, so that a good high-resolution image can be obtained. On the other hand, a portion with a large amount of motion uses low-definition images because pixel data generated by enlarging processing by linear interpolation is used. However, since the amount of movement is large, it is difficult to recognize that the definition is low due to the visual characteristics of the viewer. Therefore, the super-resolution processing unit 320 can output a high-resolution image that is generally good.

  However, when the image is reduced or enlarged by linear interpolation, the pixel position of the processed image with respect to the pixel position of the original image changes depending on the characteristics of the filter used. Therefore, in the above super-resolution processing, when a reduced image (380 in FIG. 7) is enlarged to a 4K2K image size by linear interpolation, there is a relative relationship between the pixel position after the enlargement and the pixel position of the reduced image. 3, it is necessary to use a linear interpolation filter that matches the relative relationship between the pixel position of the 4K2K image and the pixel position of the reduced image when the 4K2K image is reduced.

  FIG. 8 shows an example of the relative relationship between the pixel positions of the 4K2K image and the reduced image. FIG. 8A shows an example of the pixel position of 4K2K, and FIGS. 8B and 8C show examples of the pixel position of the reduced image.

  When the 4K2K image of FIG. 8A is reduced to 1/2 horizontal and 1/2 vertical to generate a reduced image equivalent to HDTV, a simple average of 2 horizontal pixels × 2 vertical pixels (pixels shown in black) When the reduction process is performed by obtaining the above, the x position in FIG. 8B becomes the pixel position of the reduced image. On the other hand, when the reduction processing is performed by simple thinning out from 2 horizontal pixels × vertical 2 pixels, that is, by representing by one pixel in the upper left corner, the black circle position in FIG. 8C becomes the pixel position of the reduced image. Therefore, even when a reduced image is enlarged to a 4K2K image size, it is necessary to enlarge using different filter coefficients in the case of FIG. 8B and FIG. 8C. About the relative relationship of this pixel position, it becomes possible to guarantee the compatibility between apparatuses by predetermining by a standard beforehand, and to obtain a favorable high-resolution image.

  As described above, in the high-resolution playback mode, the image recording / playback apparatus 1000 according to Embodiment 1 reads a reduced compressed image stream, which is a compressed image stream having a normal resolution compliant with the conventional standard, from the BDMV main folder 410 and divides it. The compressed image stream is read from the HIGHRESO main folder 430. The image recording / reproducing apparatus 1000 restores and reproduces the high-resolution image using the read reduced compressed image stream and divided compressed image stream.

  As described above, according to the image recording / reproducing apparatus 1000 according to the first embodiment, a high-resolution image having a resolution exceeding the conventional standard can be recorded and reproduced on the recording medium 10.

  At this time, in the recording / reproducing unit 200, the control of the arrangement of the image in the external memory 300 in the memory control unit 201 and the generation, recording and reproduction of the compressed image stream in the compression / decompression unit 205 and the recording / reproduction processing unit 206 However, since the image itself handles an image of a conventional resolution size, encoding and recording can be performed with a small circuit scale and an arithmetic processing amount.

  In particular, since the images handled by the image output unit 202, the image reduction unit 203, and the GUI multiplexing unit 204 are also images of the conventional resolution size, the recording / playback unit 200 controls the arrangement of images in the external memory 300 in the memory control unit 201, and When the processing related to the generation and recording / reproduction control of the compressed image stream in the compression / decompression unit 205 and the recording / reproduction processing unit 206 is implemented as software on the microprocessor, the recording / reproduction unit 200 is implemented with hardware implemented in accordance with the conventional resolution. The wear can be used as it is, which is further useful.

  Furthermore, a high resolution image is encoded by an encoding device that supports the conventional resolution, and at the time of reproduction, a high resolution image and a reduced image of the conventional resolution are alternately obtained, and super-resolution processing is performed using this characteristic. It is possible to provide an image encoding apparatus and an image recording apparatus that can reproduce a high-resolution image that is favorable at a high frame rate with respect to the performance of the encoding apparatus by performing the high-resolution image.

  In addition, when the recording medium 10 recorded by the image recording / reproducing apparatus 1000 according to the first embodiment is mounted on an image reproducing apparatus conforming to a conventional standard that does not support high-resolution image reproduction, The image reproducing apparatus can reproduce an image having a resolution of 1920 × 1080 conforming to the conventional standard by reproducing only the reduced image. That is, the image recording / reproducing apparatus 1000 can ensure compatibility with a conventional standard image reproducing apparatus.

  Further, the image recording / reproducing apparatus 1000 according to the first embodiment records the compressed image stream of the conventional resolution corresponding to the conventional standard and the divided compressed image stream in one file system of the recording medium 10, so that the management is possible. Easy. That is, the image recording / reproducing apparatus 1000 according to the first embodiment can record a high-resolution image that can be easily managed.

  Further, the image recording / reproducing apparatus 1000 according to Embodiment 1 forms the folder structure of the HIGHRESO main folder 430 in accordance with the folder structure of the BDMV main folder 410. Thereby, it is possible to easily take a correspondence between the reduced image from 4K2K and the divided image. Furthermore, the processing for designating and playing back the stream files 419 and 439 from the playlist files 417 and 437 via the clip info files 418 and 438 is performed for the conventional resolution image or the reduced compressed image stream from 4K2K and the divided compressed image stream. Since sharing is possible, the implementation of the system control unit 207 can be simplified.

  The image recording / reproducing apparatus 1000 according to the first embodiment has been described above, but the present invention is not limited to this embodiment.

  For example, in the above description, one main folder of the divided compressed image stream corresponds to one main folder of the reduced compressed image stream. However, the divided compressed image stream may be divided into a plurality of divided compressed image streams. These may be recorded separately in a plurality of main folders.

  In the above description, the image recording / reproducing apparatus 1000 in FIG. 5 determines whether or not the divided compressed image stream exists after determining whether or not the high-resolution reproduction mode is set (S220) (S221). However, after determining whether or not the divided compressed image stream exists (S221), it may be determined whether or not the high-resolution playback mode is set (S220). Further, the image recording / reproducing apparatus 1000 may determine whether or not the high-resolution reproduction mode is set at every predetermined time (S220).

  In the above description, the folder configuration shown in FIG. 4 is shown. However, the conventional standard compatible image and the high resolution image may be stored in different folders, and the present invention is not limited to this. Absent. That is, the playlist file 417, the clip info file 418, and the stream file 419 may be recorded in the BDMV main folder 410 or a folder formed hierarchically below the BDMV main folder 410. The playlist file 437, the clip info file 438, and the stream file 439 may be recorded in the HIGHRESO main folder 430 or a folder formed hierarchically below the HIGHRESO main folder 430.

  In addition, the substance of information included in the playlist file 417, the clip info file 418, and the stream file 419 does not need to be stored in the BDMV main folder 410, and the playlist file 417, the clip info file 418, and the stream file 419 are stored. Only information that identifies the substance of the included information may be stored in the BDMV main folder 410. That is, the playlist file 417, the clip info file 418, and the stream file 419 may be recorded in association with the BDMV main folder 410.

  Similarly, the substance of information included in the playlist file 437, the clip info file 438, and the stream file 439 does not need to be stored in the HIGHRESO main folder 430, and the playlist file 437, the clip info file 438, and the stream file 439 are not stored. Only the information specifying the substance of the information included in the file may be stored in the HIGHRESO main folder 430. That is, the playlist file 437, the clip info file 438, and the stream file 439 may be recorded in association with the HIGHRESO main folder 430.

  In the above description, it is assumed that the super-resolution processing unit 320 is connected to the external memory 330 in FIG. 2, but this can also be configured to be used as the external memory 300.

(Embodiment 2)
Next, the second embodiment will be described.

  The image encoding apparatus according to the second embodiment is a modification of the image encoding apparatus according to the first embodiment, that is, the compression / decompression unit 205, and the image recording apparatus, that is, the image recording / reproducing apparatus 1000.

  In the second embodiment, instead of recording the reduced compressed image stream and the divided compressed image stream in different folders on the recording medium, by devising the multiple processing after changing the ratio of the reduced image and the divided image, An image encoding apparatus and an image recording / reproducing apparatus that enable encoding with a smaller processing amount and simplify the processing of the recording / reproducing processing unit 206 will be described.

  The image recording / reproducing apparatus using the image coding apparatus according to the second embodiment has the same block diagram level configuration as the image recording / reproducing apparatus 1000 shown in FIG. However, the details of the generation method of the reduced image and the divided image by the camera image signal processing unit 104 and the multiplexing method after compression by the compression / decompression unit 205 are different from those of the first embodiment.

  FIG. 9 is a diagram showing signals in each part in the image recording / reproducing apparatus 1000 using the image coding apparatus according to the second embodiment.

  FIG. 9A is a diagram illustrating a 4K2K image signal generated in the camera image signal processing unit 104.

  FIG. 9B is a diagram illustrating an image signal output from the camera image signal processing unit 104 to the memory control unit 201.

  FIG. 9C shows a reduced compressed image stream generated by compressing the reduced image in the compression / decompression unit 205.

  FIG. 9D is a diagram showing a divided compressed image stream generated by compressing the divided images in the compression / decompression unit 205.

  FIG. 9E is a diagram showing a divided compressed image stream output from the compression / decompression unit 205 by inserting a skip P picture into the reduced compressed image stream in the compression / decompression unit 205.

  Every time 11 sheets of 4K2K image signals of 24 frames per second shown in FIG. 9A are supplied to the image reducing section 110, one sheet is supplied to the image dividing section 111. The image reducing unit 110 reduces the input frame 350 of the 4K2K image to 1/2 in the horizontal direction and 1/2 in the vertical direction to the recording / reproducing unit 200 as a reduced image 360 of HDTV size having the number of pixels of 1920 × 1080. Output. Further, the image dividing unit 111 divides the input 4K2K image frame 351 into four divided images 361, 362, 363, and 364 having a horizontal pixel number of 1/2 and a vertical pixel number of 1/2. The image is output to the recording / reproducing unit 200 as an HDTV-sized divided image having a number of pixels of 1920 × 1080.

  Unlike FIG. 3, since 11 reduced images are inserted between the divided images, the camera image signal processing unit 104 performs time-axis multiplexing on the reduced image 360 and the divided images 361, 362, 363, and 364. The format of the HDTV size image of 30 frames per second shown in FIG. The camera image signal processing unit 104 outputs the HDTV size multiplexed image to the recording / playback unit 200.

  Under the control of the system control unit 207, the memory control unit 201 extracts a reduced image from the HDTV size multiplexed image output from the camera image signal processing unit 104, and temporarily stores the reduced image in the reduced image area 310 on the external memory 300. If necessary, the data is read out and output to the image output unit 202, the image reduction unit 203, and the compression / expansion unit 205. Further, the memory control unit 201 extracts a divided image from the HDTV size multiplexed image output from the camera image signal processing unit 104 under the control of the system control unit 207 and temporarily stores the divided image in the divided image region 311 on the external memory 300. The data is stored, read as necessary, and output to the image output unit 202 and the compression / decompression unit 205.

  Here, when outputting the reduced image and the divided image to the image output unit 202 and the compression / decompression unit 205, the memory control unit 201 outputs the HDTV size multiplexed image shown in FIG. 9B. The image output unit 202 outputs the HDTV size multiplexed image output from the memory control unit 201 via the output terminal 301 after performing image processing such as edge enhancement and noise cancellation.

  The compression / decompression unit 205 converts the HDTV size multiplexed image output from the memory control unit 201 to MPEG-2 or H.264. Based on an encoding method such as H.264, the data is compressed and multiplexed. The compression / decompression unit 205 outputs the compressed image stream obtained by the compression multiplexing to the recording / playback processing unit 206.

  At this time, since the compression / decompression unit 205 compresses the HDTV size multiplexed image, the performance as the encoding device only needs to be sufficient to compress the HDTV size image signal of 30 frames per second. That is, the performance as an encoding device may be a performance corresponding to a so-called 1080i signal format that is generally used when compressing an HDTV signal. However, under the control of the system control unit 207, the reduced image and the divided image in the input HDTV size multiplexed image are distinguished, and compression is performed by switching the internal processing so that the compression processing becomes independent.

  When compressing the reduced image, the compression processing is performed using the inter-frame reference of only the reduced images (for example, the frame 360 and the frame 365) in the HDTV size multiplexed image shown in FIG. 9B. In this case, if the inter-frame compression process is performed using only the I picture and the P picture without using the B picture, the order of the frames to be compressed becomes the same as the order of the input frames, so that the control does not have to be complicated.

  The compressed image stream generated from the reduced image is in the format as shown in FIG. 9C. Here, in order to make the stream compliant with the conventional standard, a skip P picture is inserted at a rate of 1 frame per 11 frames. Thus, the compression / decompression unit 205 can generate the pseudo 24p format compressed image stream shown in FIG.

  On the other hand, when compressing a divided image, since the time between images is greatly opened and the correlation is low, it is preferable to use intraframe compression without using interframe reference. Here, the compression / decompression unit 205 generates a compressed image stream shown in FIG. 9D by intra-frame compression processing from the divided images.

  In the image coding apparatus according to the second embodiment, that is, the compression / expansion unit 205, the divided image is one frame per 11 frames, and the ratio to the reduced image is small. It is assumed that the compressed image stream is multiplexed with the reduced compressed image stream and output. By doing so, it becomes possible to record the recording medium 10 including the divided compressed image stream only with the folder structure under the BDMV main folder in FIG.

  Similar to the image encoding apparatus according to the first embodiment, the compression / decompression unit 205 outputs the separate compressed image stream and the divided compressed image stream as separate streams without multiplexing the reduced compressed image stream and the folder configuration shown in FIG. You may record as follows.

  There can be a plurality of methods for multiplexing the divided compressed image stream into the reduced compressed image stream. Hereinafter, this multiplexing method will be described.

(First multiplexing method)
FIG. 10 is a diagram illustrating a structure example of a compressed image stream according to the second embodiment. Here, the video signal is H.264. It is assumed that the data is compressed based on the H.264 standard and multiplexed into the TS (transport stream) format of the MPEG-2 system standard as a multiplexing method.

  At this time, the compressed image stream before multiplexing, that is, the so-called video ES (elementary stream), as shown in FIG. 10, is an AU delimiter, SPS, PPS, SEI, main picture, redundant picture, EOS (end of sequence). ) And EOS (end of stream). In the reduced compressed image stream, data obtained by compressing the reduced image is stored in the main picture.

  In this configuration, there are a plurality of types of SEI, and among them, there is a type called User Data Unregistered SEI that can multiplex arbitrary user data. By multiplexing the divided compressed image stream here, it is possible to reproduce only the reduced image by ignoring the divided compressed image stream in an image reproducing apparatus compliant with the conventional standard that does not support reproduction of a high resolution image. .

(Second multiplexing method)
In the case of multiplexing in the MPEG-2 system standard TS format, the elementary stream in FIG. 10 is first multiplexed in the PES (packetized elementary stream) format.

  FIG. 11 is a diagram illustrating a configuration example of a PES of the MPEG-2 standard. A PES packet storing a compressed image stream (ES) is a variable-length packet, and an input ES divided in units of pictures is packet data for one PES packet. One PES packet is obtained by adding a PES header starting with a PES start code to the head of this packet data.

  FIG. 12 is a diagram illustrating a configuration example of a TS of the MPEG-2 standard. The TS packet is a fixed-length packet, and the PES is divided in accordance with the size of the payload (data insertion portion), and the PES is transmitted in the payload of a plurality of TS packets. In addition, packet identification information (PID) indicating the packet type is multiplexed in the TS header added to the head of the TS packet. The PID value is given a unique value depending on the contents of the payload such as image data and audio data.

  In the present embodiment, the PES storing the reduced compressed image stream and the PES storing the divided compressed image stream are multiplexed as TS packets having different PID values. At this time, the PID value assigned to the TS packet storing the compressed image stream in the conventional standard is given to the TS packet storing the reduced compressed image stream, and the TS packet storing the divided compressed image stream is A PID value that is not used in the conventional standard or assigned to user data is assigned.

  As a result, a conventional standard-compliant image playback device that does not support playback of high-resolution images can select and play back only TS packets that store the reduced compressed image stream while ignoring the divided compressed image stream. Become. Further, in the recording / reproducing apparatus of the present embodiment, if necessary, it is possible to select and reproduce only TS packets storing only the divided compressed image stream.

  In this embodiment, since the ratio of the divided image is small with respect to the reduced image, the super-resolution processing unit 320 generates a 4K2K high-resolution image corresponding to the reduced image, compared to the first embodiment. This is disadvantageous in terms of image quality.

  For example, after a reduced image is enlarged to a 4K2K image size by linear interpolation, it is divided into motion detection blocks for every 16 × 16 pixels, and a 16 × 16 pixel block at a position where horizontal and vertical coordinates are close to each other on a high resolution image The motion vector detection process is executed between. At this time, depending on the reduced image, the time from the immediately preceding high-resolution image is wide open, and the accuracy of motion vector detection is reduced. As a result, the portion that is determined to have a small amount of motion is reduced, and there is a possibility that the number of locations that use motion detection blocks generated by enlarging the reduced image by linear interpolation may increase.

  However, since the portion with a small amount of motion between the reduced image and the high-resolution image uses originally high-resolution pixel data, the high-resolution is better than that generated by enlarging all image frames from the reduced image. An image can be obtained.

(Multiple pixel position related auxiliary data)
In the first embodiment, compatibility between devices is determined by preliminarily determining the relative relationship between the pixel position of the high-resolution image and the pixel position of the reduced image when the image is reduced or enlarged by linear interpolation. To obtain a good high-resolution image. In the second embodiment, in order to improve the performance of the reduction processing for generating a reduced image and the super-resolution processing at the time of reproduction, that is, the reproduction image quality after recording / reproduction, Make it possible to select any reduction filter.

  For this reason, it is necessary to multiplex information on the relative relationship between the pixel positions as pixel position relationship auxiliary data into the reduced compressed image stream, or to record it separately on the recording medium 10.

  As one method, the camera image signal processing unit 104 outputs pixel position relationship auxiliary data to the camera control unit 105 according to the reduction filter, and transmits the data to the compression / decompression unit 205 via the system control unit 207 from the camera control unit 105. There is a method in which the compression / decompression unit 205 multiplexes the reduced compressed image stream. Alternatively, if the reduction method in the camera image signal processing unit 104 is fixed, the pixel position relationship auxiliary data by the reduction method is stored in advance by the system control unit 207 or the compression / decompression unit 205, and the compression / decompression unit 205 performs reduction / compression. There is a method of multiplexing the image stream.

  When multiplexing to a reduced-compressed image stream, there is a method of multiplexing to an elementary stream as user data (for example, using User Data Unregistered SEI in H.264 shown in FIG. 10). As described with reference to FIG. 12, there is a method of multiplexing as TS packets having a special PID value.

  Furthermore, as another multiplexing method, there is a method of multiplexing as a Descriptor of the MPEG-2 system standard. This is a data format stored in the payload of a TS packet having a specific PID value in the TS format shown in FIG. Since this Descriptor multiplexes information other than the pixel position related auxiliary data, it is normally mounted in a decoding apparatus that supports the MPEG-2 system standard. Therefore, this multiplexing method is used for multiplexing and separating pixel position related auxiliary data. Therefore, there is an advantage that it is not necessary to add a special circuit.

  As another method, there is a method in which the pixel position related auxiliary data is separately recorded on the recording medium 10 instead of being multiplexed on the compressed image stream. In this case, since the data does not exist in the conventional standard, it is desirable to record it as a file placed in a place other than the subordinate of the BDMV main folder 410 in FIG.

  With the configuration described above, when reproducing a high-resolution image from the recording medium 10, the pixel position relationship auxiliary data is output from the compression / decompression unit 205 or the recording / reproduction processing unit 206 to the super-resolution processing unit 320 via the system control unit 207. Thus, it is possible to select an optimum filter coefficient in the super-resolution processing unit 320.

  As described above, according to the image recording / reproducing apparatus 1000 according to the second embodiment, a high-resolution image having a resolution exceeding the conventional standard can be recorded and reproduced on the recording medium 10.

  At this time, in the recording / reproducing unit 200, the control of the arrangement of the image in the external memory 300 in the memory control unit 201 and the generation, recording and reproduction of the compressed image stream in the compression / decompression unit 205 and the recording / reproduction processing unit 206 However, since the image data itself handles an image having a resolution size, encoding and recording can be performed with a small circuit scale and an operation processing amount.

  Furthermore, an encoding device that supports the conventional resolution encodes a high-resolution image, and at the time of reproduction, an image in which the high-resolution image is inserted appropriately between the reduced images of the conventional resolution is obtained. Perform high-resolution processing using a resolution image and generate a high-resolution image of a frame corresponding to a reduced image, thereby reproducing a good high-resolution image at a high frame rate relative to the performance of the encoding device. Therefore, it is possible to provide an image encoding apparatus and an image recording apparatus that can perform the above-described functions.

  Further, when the recording medium 10 recorded by the image recording / reproducing apparatus 1000 according to the second embodiment is attached to an image reproducing apparatus conforming to a conventional standard that does not support high-resolution image reproduction, The image reproducing apparatus ignores the divided images multiplexed in the compressed image stream and reproduces only the reduced image, thereby reproducing an image having a resolution of 1920 × 1080 conforming to the conventional standard. That is, the image recording / reproducing apparatus 1000 can ensure compatibility with a conventional standard image reproducing apparatus.

  Further, the image recording / reproducing apparatus 1000 according to the second embodiment is easy to manage because the reduced compressed image stream and the divided compressed image stream are multiplexed and recorded under the BDMV main folder 410 of the recording medium 10. That is, the image recording / reproducing apparatus 1000 according to the second embodiment can record a high-resolution image that can be easily managed.

  The image recording / reproducing apparatus 1000 according to the second embodiment has been described above, but the present invention is not limited to this embodiment.

  For example, in the above description, one frame of high-resolution image is inserted into four divided frames every 11 frames of the reduced image, but the number of frames of the reduced image M (M is an integer) and the high-resolution image that becomes the divided image The number of image frames N (N is an integer) and the number of divisions of the high-resolution image can be freely set. Furthermore, in order to be able to select these frame numbers M and N, the M and N frame number information can be multiplexed into the compressed image stream in the same manner as the pixel position related auxiliary data, and the multiplexing destination is also the same. In addition, a method of multiplexing as elementary data in the elementary stream, a method of multiplexing as a TS packet having a special PID value, and a method of multiplexing as a Descriptor of the MPEG-2 system standard can be used. Further, instead of multiplexing the compressed image stream, it is also possible to record M and N frame number information on the recording medium 10.

(Embodiment 3)
Next, the third embodiment will be described.

  In the third embodiment, an image conversion process for generating a conventional resolution HDTV size image at 24 frames per second from a reduced compressed image stream and a divided compressed image stream recorded on the recording medium 10 will be described. This image conversion process is a process for creating a recording medium that displays a smooth image on an image reproducing device that supports only the conventional resolution.

  FIG. 13 is a diagram illustrating a configuration example of an image conversion apparatus according to the third embodiment.

  The image conversion apparatus 1001 includes a recording / playback unit 500 and an external memory 300. The recording / playback unit 500 includes a memory control unit 501, a playback processing unit 502, a separation / expansion unit 503, a playback control unit 504, a system control unit 505, an image reduction unit 506, a compression multiplexing unit 507, and a recording process. Unit 508 and a recording control unit 509.

  The recording / reproducing unit 500 is used with the recording medium 10 and the recording medium 20 attached thereto. The recording medium 10 is generated in advance by the image recording / reproducing apparatus 1000 using the image encoding apparatus according to the first embodiment, and the 4K2K high-resolution image is divided into a reduced image and a divided image every other frame and is compressed and encoded. The reduced compressed image stream and the divided compressed image stream are recorded separately in a BDMV main folder and a HIGHRESO main folder.

  The reproduction processing unit 502 reproduces the divided compressed image stream recorded on the recording medium 10 and outputs it to the separation / decompression unit 503. The separation / decompression unit 503 separates and decompresses the divided compressed image stream and outputs the divided image to the memory control unit 501. The memory control unit 501 once records the divided images in the external memory 300, reads them, combines them, and outputs them to the image reduction unit 506 as 4K2K high resolution images.

  The image reduction unit 506 reduces the input 4K2K high resolution image and outputs it as a reduced image after reproduction. The compression multiplexing unit 507 compresses the post-reproduction reduced image and outputs the post-reproduction reduced compressed image stream to the recording processing unit 508. The recording processing unit 508 processes the post-reproduction reduced compressed image stream as necessary, and records the converted compressed image stream of 24 frames per second on the recording medium 20.

  The reproduction control unit 504 controls a drive device that drives the recording medium 10 or a recording protocol such as a recording address input procedure. The recording control unit 509 controls a recording device such as a drive device that drives the recording medium 20 or a recording address input procedure.

  The system control unit 505 uses a user operation received via an input device not described as operation information, and based on the operation information, the memory control unit 501, the reproduction processing unit 502, the separation / decompression unit 503, the reproduction control unit 504, and the image reduction By controlling the unit 506, the compression multiplexing unit 507, the recording processing unit 508, and the recording control unit 509, reproduction processing, image conversion processing, and recording processing in the image conversion apparatus 1001 are executed.

  FIG. 14 shows a 4K2K high-resolution image output from the memory control unit 501 to the image reduction unit 506, a reduced compressed image stream recorded on the recording medium 10, and 24 times per second recorded by the recording processing unit 508 on the recording medium 20. It is a figure explaining the relationship with the conversion compression image stream of a frame.

  FIG. 14A is a diagram illustrating a 4K2K high-resolution image output from the memory control unit 501 to the image reduction unit 506.

  FIG. 14B is a diagram showing a reduced compressed image stream recorded on the recording medium 10.

  FIG. 14C is a diagram showing a converted compressed image stream of 24 frames per second recorded on the recording medium 20 by the recording processing unit 508.

  The image conversion processing in the recording / playback unit 500 includes the reference structure of the reduced compressed image stream recorded in the recording medium 10 shown in FIG. 14B and the conversion recorded in the recording medium 20 shown in FIG. It varies depending on the reference structure of the compressed image stream.

(Image conversion method 1)
In the reduced compressed image stream of FIG. 14B, the P picture 612 refers to the preceding P picture 610 instead of the skip P picture 611 (reference 651), and the conversion compression of FIG. In the image stream, when the P picture 623 replaced with the skip P picture similarly refers to the I or P picture 621 replaced with the skip P picture (reference 671), it is not necessary to separate and expand the reduced compressed image stream.

  In this case, the reproduction processing unit 502 reproduces the reduced compressed image stream recorded on the recording medium 10 and outputs it to the recording processing unit 508. On the other hand, from the compression multiplexing unit 507, a post-reproduction reduced compressed image stream generated by further reducing and compressing the high-resolution image reproduced and synthesized from the divided compressed image stream shown in FIG. Is output.

  The recording processing unit 508 generates a converted compressed image stream by replacing each picture (picture 621, picture 623) of the reduced compressed image stream after reproduction with a skip P picture (picture 611, picture 613) of the reduced compressed image stream. To be recorded on the recording medium 20.

  In this image conversion method, since it is not necessary to separate and expand the reduced compressed image stream, it is possible to keep the processing amount very small. However, it is necessary to devise when generating a reduced compressed image stream. Details of this will be described in Embodiment 4.

(Image conversion method 2)
In the reduced compressed image stream in FIG. 14B, when the P picture 612 refers to the skip P picture 611 (reference 652), or in the converted compressed image stream in FIG. 14C, the skip P picture is replaced. When the P picture 623 refers to the P picture 622 of the reduced compressed image stream (reference 672), it is necessary to separate and expand the reduced compressed image stream.

  In this case, the reproduction processing unit 502 reproduces the reduced compressed image stream recorded on the recording medium 10 and outputs it to the separation / decompression unit 503. The separation / decompression unit 503 separates and decompresses the reduced compressed image stream and outputs the reduced image to the memory control unit 501. At this time, the skip P picture (picture 611, picture 613) is not output. The memory control unit 501 once records the reduced image in the external memory 300, reads it, and outputs it to the compression multiplexing unit 507.

  On the other hand, the image reduction unit 506 outputs a post-reproduction reduced image obtained by further reducing the high-resolution image reproduced and synthesized from the divided compressed image stream shown in FIG.

  The compression multiplexing unit 507 combines the reduced image input from the memory control unit 501 and each picture of the reduced image after reproduction input from the image reduction unit 506 in the order shown in FIG. An image stream is generated and output to the recording processing unit 508. The recording processing unit 508 records the converted compressed image stream on the recording medium 20.

  In this image conversion method, since it is necessary to separate and expand the reduced compressed image stream, it is not necessary to expand the skip P picture, but the processing amount is increased as compared with the image conversion method described above. However, when the reduced compressed image stream is generated, it is sufficient that the reduced compressed image stream alone conforms to the standard of compression encoding.

  As described above, according to the image conversion apparatus 1001 according to the third embodiment, the reduced compressed image stream recorded on the recording medium 10 by the image recording / reproducing apparatus 1000 using the image encoding apparatus according to the first embodiment, In addition, a HDTV size image having a conventional resolution is generated from the divided compressed image stream at 24 frames per second and is recorded on the recording medium 20, thereby displaying a smooth motion image on an image reproducing device that supports only the conventional resolution. 20 can be created.

  At this time, since it is not necessary to separate and decompress all the data in the reduced compressed image stream, the processing amount required for image conversion is reduced, and encoding and recording can be performed with a small circuit scale and arithmetic processing amount. It is said.

  The image conversion apparatus 1001 according to the third embodiment has been described above, but the present invention is not limited to this embodiment.

  For example, in the above description, the reproduction processing unit 502 and the recording processing unit 508, the separation / decompression unit 503, and the compression multiplexing unit 507 are described as separate configurations, but the common configuration such as the recording / reproduction processing unit and the compression / decompression unit may be used. It is feasible. Further, the conversion compressed image stream is described as being recorded on the recording medium 20 different from the recording medium 10, but the configuration in which the converted compressed image stream is written back to the recording medium 10 when the external memory 300 has a sufficiently large capacity. It can also be.

  Furthermore, in the above description, the image conversion apparatus has been described. However, for example, an image reduction unit 506 is added to the image recording / playback apparatus 1000 using the image encoding apparatus according to Embodiment 1, and an image conversion function is provided. It can also be realized as an on-board image recording / reproducing apparatus.

(Embodiment 4)
Next, the fourth embodiment will be described.

  The image coding apparatus according to the fourth embodiment is a modification of the image coding apparatus according to the first embodiment, that is, the compression / decompression unit 205, and the image recording apparatus, that is, the image recording / reproducing apparatus 1000.

  In the fourth embodiment, the converted compressed image obtained by processing the recording medium 10 on which the high-definition image is recorded by the image conversion apparatus 1001 according to the third embodiment by devising the buffer control when compressing the reduced image. An image encoding apparatus and an image recording / reproducing apparatus that enable a stream to comply with the compression encoding standard will be described.

  The image recording / reproducing apparatus using the image encoding apparatus according to the fourth embodiment has the same configuration and operation at the block diagram level as those of the image recording / reproducing apparatus 1000 shown in FIG. The details of the processing are different from those of the first embodiment.

  FIG. 15 is a diagram illustrating an image of buffer control in the compression / decompression unit 205 of the image recording / reproducing device using the image coding device according to the fourth embodiment.

  In the standard of image compression coding, a model of the separation / expansion unit 503 that operates ideally is assumed in order to generate an image bitstream that can be normally decoded even if it is exchanged between apparatuses. Hereinafter, H.C. The model of the separation / decompression unit will be described assuming compression coding according to the H.264 standard. The image encoding apparatus controls the code amount of a code string generated from each picture so that a stream buffer of this model (hereinafter referred to as CPB (Coded Picture Buffer)) does not cause overflow or underflow.

  FIG. 15A is a diagram showing the picture order of the reduced-compression image stream recorded on the recording medium 10.

  FIG. 15B is a diagram showing the buffer occupancy rate of the CPB model in the reduced compressed image stream recorded on the recording medium 10.

  FIG. 15C is a diagram illustrating the buffer occupancy rate of the CPB model in the converted compressed image stream after the conversion processing is performed by the image conversion apparatus 1001 according to the third embodiment.

  As shown in FIG. 15B, the buffer occupancy rate of the CPB model gradually increases in accordance with the bit rate of the image bitstream, and at the moment when the decoding process of each picture is started, the code amount of that picture is increased. Decrease (ie read from the stream buffer at infinite speed).

  In general, the I picture has the largest code amount of the code string, the P picture has the next largest code quantity, and the B picture has the smallest code quantity. Since this code amount varies depending on the degree of difficulty of compression processing of each picture, the CPB occupation ratio varies with time. However, the encoding unit does not perform full or empty with respect to the capacity of the CPB. Perform quantity control. FIG. 15B is a graph showing this state.

  Here, the target recording medium 10 is not the skip P picture 611 but the P picture before the P picture 612 in the reduced compressed image stream of FIG. 14B in the image conversion processing described in the third embodiment. In the case of referring to the picture 610 (reference 651) and referring to the I or P picture 621 in which the P picture 623 replaced with the skip P picture is also replaced with the skip P picture in the converted compressed image stream of FIG. This corresponds to the image conversion processing in the case of performing (reference 671).

  In this case, since each picture of the post-reproduction reduced compressed image stream replaced by the image conversion apparatus 1001 according to Embodiment 3 is data obtained by compressing the actual image, the data amount per picture is larger than the skip P picture. . In the compression / decompression unit 205 of the image recording apparatus according to the fourth embodiment, a buffer having a size smaller than the standard is assumed as the CPB buffer in anticipation of the increase in the data amount, and rate control is performed when the reduced compressed image stream is generated. As a result, even if the skip P picture is replaced with each picture of the reduced image after reproduction, as shown in FIG. 15C, code conversion is possible in which the CPB occupancy does not exceed the standard range.

  It should be noted that an increase in the amount of data expected in the compression / decompression unit 205 and an increase in the compression multiplexing unit 507 are determined by an agreement between the image encoding device according to the fourth embodiment and the image conversion device according to the third embodiment according to standards and the like. It becomes possible to match the CPB occupancy rate when compressing the reduced image after reproduction, and to keep the CPB occupancy rate in the converted compressed image stream within the standard range.

  As described above, according to the image encoding device according to the fourth embodiment, the image conversion device 1001 according to the third embodiment can generate 24 frames per second from the reduced compressed image stream and the divided compressed image stream recorded on the recording medium 10. Thus, when a converted compressed image stream having a conventional resolution is generated and recorded on the recording medium 20, a compressed image stream complying with the buffer capacity defined by the standard can be generated.

  The image coding apparatus according to Embodiment 4 has been described above, but the present invention is not limited to this embodiment.

  For example, in the above description, H.C. Although the description has been given using the CPB buffer on the assumption of the image compression based on the H.264 encoding, the VBV buffer may be used based on the MPEG-2 encoding, and the decoding may be performed based on another encoding scheme. Any coding method may be used as long as a model for the buffer of the device input stage is used.

(Embodiment 5)
Next, the fifth embodiment will be described.

  The image coding apparatus according to the fifth embodiment is a modification of the image coding apparatus according to the fourth embodiment, and a recording medium 10 on which a high-definition image is recorded as in the fourth embodiment. This is an image encoding device that enables the converted compressed image stream processed by the image conversion device 1001 according to the third aspect to conform to the compression encoding standard.

  FIG. 16 is a diagram showing an image of buffer control in the compression / decompression unit 205 of the image recording / playback apparatus using the image coding apparatus according to the fifth embodiment.

  FIG. 16A is a diagram showing the picture order of the reduced-compression image stream recorded on the recording medium 10.

  FIG. 16B is a diagram showing the buffer occupancy rate of the CPB model in the reduced compressed image stream on the recording medium 10 recorded by the image recording / reproducing apparatus 1000 according to the first embodiment.

  FIG. 16C shows the buffer occupancy rate of the CPB model in the reduced compressed image stream on the recording medium 10 recorded by the image recording / reproducing apparatus according to the fifth embodiment.

  FIG. 16D shows the buffer occupancy rate of the CPB model in the converted compressed image stream after the conversion processing by the image conversion apparatus 1001 according to the third embodiment.

  In the compression / decompression unit 205 of the image recording apparatus according to the fifth embodiment, the data amount of each picture of the post-reproduction reduced compressed image stream replaced by the image conversion apparatus 1001 according to the third embodiment when a skip P picture is inserted. Stuffing data having a data amount corresponding to the difference between the data amount and the skip P picture data amount is inserted into the reduced image compressed data, and the stuffing data amount is included in the rate control when the reduced compressed image stream is generated. I do.

  In this case, the data amount of the divided compressed image stream is substantially the same as the data amount of the reduced compressed image stream because it is originally a continuous image with the reduced compressed image stream. Therefore, the difference between the data amount and the skipped P picture data amount is included in the rate control while being inserted into the reduced image compressed data as stuffing data, so that the conversion compressed image stream follows the change in the compression difficulty of the input image. The code conversion can be performed without the CPB occupancy rate exceeding the standard range.

  As described above, according to the image encoding device according to the fifth embodiment, the image conversion device 1001 according to the third embodiment can generate 24 frames per second from the reduced compressed image stream and the divided compressed image stream recorded on the recording medium 10. Thus, when a converted compressed image stream having a conventional resolution is generated and recorded on the recording medium 20, a compressed image stream complying with the buffer capacity defined by the standard can be generated.

  Further, by adjusting the amount of stuffing data to be multiplexed on the reduced compressed image stream, it is possible to follow the time change of the compression difficulty level of the input image.

  The image coding apparatus according to Embodiment 5 has been described above, but the present invention is not limited to this embodiment.

  For example, in the above description, H.C. Although the description has been given using the CPB buffer on the assumption of the image compression based on the H.264 encoding, the VBV buffer may be used based on the MPEG-2 encoding, and the decoding may be performed based on another encoding scheme. Any coding method may be used as long as a model for the buffer of the device input stage is used.

(Embodiment 6)
Next, the sixth embodiment will be described. The image recording / reproducing apparatus using the image encoding apparatus according to the sixth embodiment is a modification of the image recording / reproducing apparatus using the image encoding apparatus according to the first embodiment.

  FIG. 17 is a diagram showing a configuration of an image recording / reproducing device 1002 using the image encoding device according to the sixth embodiment. As illustrated in FIG. 17, the image recording / reproducing device 1002 includes a camera unit 100, a recording / reproducing unit 200, an external memory 300, an output terminal 301, and a display device 302. In FIG. 17, the same components as those in FIG.

  FIG. 18 is a diagram showing signals at the time of recording in the image recording / reproducing apparatus 1002 using the image encoding apparatus according to the sixth embodiment.

  FIG. 18A is a diagram illustrating a 4K2K image signal generated by the camera image signal processing unit 104.

  FIG. 18B is a diagram illustrating an image signal output from the camera image signal processing unit 104 to the memory control unit 201.

  FIG. 18C is a diagram illustrating a reduced and compressed image stream generated by compressing the reduced image in the compression / decompression unit 205.

  FIG. 18D is a diagram showing a divided compressed image stream generated by compressing the divided images in the compression / decompression unit 205.

  FIG. 18E is a diagram showing a divided compressed image stream output from the compression / expansion unit 205 by inserting a skip P picture into the reduced compression image stream in the compression / expansion unit 205.

  The image reduction unit 110 and the image division unit 111 are simultaneously supplied with the 4K2K image signal shown in FIG. In FIG. 3 (a), 4K2K image signals of 24 frames per second are taken and alternately supplied every other frame. In the sixth embodiment, the camera unit 100 takes 4K2K image signals of 12 frames per second. Are supplied at the same time. That is, the reduced image and the divided image in the HDTV size multiplexed image output from the camera image signal processing unit 104 have the same timing. Here, the camera image signal processing unit 104 multiplexes the reduced image and the divided image at the same timing so that the reduced image is output first.

  The memory control unit 201 extracts a reduced image from the HDTV size multiplexed image output from the camera image signal processing unit 104, temporarily stores the reduced image in the reduced image area 310 on the external memory 300, and reads and compresses it as necessary. The data is output to the decompression unit 205. Further, the memory control unit 201 extracts a divided image from the HDTV size multiplexed image output from the camera image signal processing unit 104, temporarily stores it in the divided image area 311 on the external memory 300, and reads it as necessary. Are output to the compression / expansion unit 205 and are combined and output to the image output unit 202 and the image reduction unit 203 as a 4K2K image. The memory control unit 201 outputs the reduced image and the divided image to the compression / decompression unit 205 in the HDTV size multiplexed image format.

  The image output unit 202 outputs the 4K2K image output from the memory control unit 201 to an external display device or the like via the output terminal 301 after performing image processing such as edge enhancement and noise cancellation. The image reduction unit 203 further reduces the 4K2K image output from the memory control unit 201 according to the resolution of the display device 302 and outputs the reduced image to the GUI multiplexing unit 204.

  The compression / decompression unit 205 converts the HDTV size multiplexed image output from the memory control unit 201 to MPEG-2 or H.264. Based on an encoding method such as H.264, the data is compressed and multiplexed. At this time, the compression / decompression unit 205 distinguishes between the reduced image and the divided image in the input HDTV size multiplexed image under the control of the system control unit 207, and performs internal processing so that the compression processing becomes independent from each other. Switch and compress.

  Regarding the details of the compression processing in the compression / decompression unit 205, the same processing as the compression / decompression unit 205 in the first embodiment is performed. However, using the fact that the reduced image is output first out of the reduced image and the divided image at the same timing, the entire motion vector is detected from the reduced image, and the divided image is compressed by using this to compress the divided image. Improve image coding efficiency.

  The subsequent processing is the same as that of the image recording / reproducing apparatus using the image encoding apparatus according to the first embodiment.

  FIG. 19 is a diagram showing processing when reproducing the recording medium 10 recorded by the image recording / reproducing apparatus 1002 using the image encoding apparatus according to the sixth embodiment.

  FIG. 19A is a diagram showing a divided compressed image stream read from the recording medium 10 by an image reproducing apparatus compatible with high-resolution recording.

  FIG. 19B is a diagram showing a high-resolution image signal that is reproduced and output by an image reproducing apparatus that supports high-resolution recording.

  FIG. 19C is a diagram illustrating a reduced and compressed image stream read from the recording medium 10 by an image reproducing apparatus that supports only the conventional resolution.

  FIG. 19D is a diagram illustrating a conventional resolution image signal that is reproduced and output by an image reproducing apparatus that supports only the conventional resolution.

  In the image recording / playback apparatus 1002 using the image coding apparatus according to the sixth embodiment, the recording / playback processing unit 206 starts from the HIGHRESO main folder of the recording medium 10 and displays the divided compressed image stream shown in FIG. Is output to the compression / decompression unit 205. The compression / decompression unit 205 separates and decompresses the divided compressed image stream, and outputs the high resolution image shown in FIG. 19B to the memory control unit 201. The memory control unit 201 once records the high resolution image in the external memory 300, reads it, and outputs it to the image output unit 202 and the image reduction unit 203.

  On the other hand, in the image reproducing apparatus that supports only the conventional resolution, the reduced compressed image stream shown in FIG. 19C is read from the BDMV main folder of the recording medium 10 and output to the compression / decompression unit. The compression / decompression unit separates and decompresses the reduced compressed image stream, and outputs the conventional resolution image shown in FIG.

  As described above, according to the image recording / reproducing apparatus 1002 using the image encoding apparatus according to the sixth embodiment, a high-resolution image having a resolution exceeding the conventional standard can be recorded and reproduced on the recording medium 10.

  At this time, in the recording / reproducing unit 200, the control of the arrangement of the image in the external memory 300 in the memory control unit 201 and the generation, recording and reproduction of the compressed image stream in the compression / decompression unit 205 and the recording / reproduction processing unit 206 However, since the image itself processed by these circuits handles an image having a resolution size, encoding and recording can be performed with a small circuit scale and a calculation processing amount.

  Furthermore, since it is necessary to reproduce only one of the reduced compressed image stream and the divided compressed image stream at the time of reproduction, the calculation processing amount can be further reduced.

  Further, when the recording medium 10 recorded by the image recording / reproducing apparatus 1002 according to Embodiment 6 is attached to an image reproducing apparatus conforming to a conventional standard that does not support high-resolution image reproduction, The image reproducing apparatus can reproduce an image having a resolution of 1920 × 1080 conforming to the conventional standard by reproducing only the reduced image. That is, the image recording / reproducing apparatus 1002 can ensure compatibility with the image reproducing apparatus of the conventional standard.

  Further, the image recording / reproducing apparatus 1002 according to the sixth embodiment records the compressed image stream of the conventional resolution corresponding to the conventional standard and the divided compressed image stream in one file system of the recording medium 10, so that the management is possible. Easy. That is, the image recording / reproducing apparatus 1002 according to the first embodiment can record a high-resolution image that can be easily managed.

  Also, the compression / decompression unit 205 of the image recording / playback apparatus 1002 according to Embodiment 6 uses the fact that the reduced image is output first among the reduced image and the divided image at the same timing, and the entire image is reduced from the reduced image. By detecting the motion vector and compressing the divided image using the detected motion vector, the coding efficiency of the divided image can be improved.

  The image recording / reproducing apparatus using the image encoding apparatus according to the sixth embodiment has been described above, but the present invention is not limited to this embodiment.

  For example, in the above description, one main folder of the divided compressed image stream corresponds to one main folder of the reduced compressed image stream. However, the divided compressed image stream may be divided into a plurality of divided compressed image streams. These may be recorded separately in a plurality of main folders.

  The image coding apparatus, the image recording apparatus, and the image conversion apparatus according to the first, second, third, fourth, and fifth embodiments have been described above. However, the present invention is not limited to this embodiment. Unless it deviates from the gist of the present invention, various modifications conceived by those skilled in the art have been made in the present embodiment, and forms constructed by combining components in different embodiments are also included in the scope of the present invention. .

  For example, in the above description, an image having a resolution of 1920 × 1080 has been described as an example of a conventional standard image, but the conventional standard image may have a resolution other than 1920 × 1080. For example, the conventional standard image may be an image having a resolution of 1440 × 1080.

  In the above description, the high-resolution image has a resolution of 4096 × 2160 or 3840 × 2160, but the image size of the high-resolution image is not limited to this, and may be a resolution higher than the resolution of the conventional standard. . Therefore, for example, a conventional standard image may be an image having a resolution of 720 × 480, and a high-resolution image may be an image having a resolution of 1920 × 1080. Alternatively, for example, a conventional standard image may be a 3840 × 2160 image, and a high-resolution image may be a 7680 × 4320 image.

  Further, in the above description, the GOP structure using only the I picture and the P picture has been described, but a GOP structure using the B picture may be used. Also, the interval between I pictures can be arbitrarily determined.

  In the above description, an image of 24 frames per second is described as an example of the input image, but the frame rate may be other than 24 frames per second. For example, an image having a frame rate of 30 frames per second may be used. Further, it may be a progressive scanning image or an interlaced scanning image.

  Further, some or all of the components constituting the image encoding device, the image recording / playback device, and the image conversion device may be configured by one system LSI (Large Scale Integration). Good. The system LSI is an ultra-multifunctional LSI manufactured by integrating a plurality of components on one chip. Specifically, a microprocessor, a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. It is a computer system comprised including. Specifically, for example, as shown in FIG. 2, the camera image signal processing unit 104, the recording / reproducing unit 200, and the super-resolution processing unit 320 may each be configured by one LSI. May be configured as a single system LSI.

  The present invention can be applied to an image encoding device, a code conversion device, an image recording device, and an image reproduction device, and in particular, a digital video camera, a digital camera, a digital recorder, a digital TV, a game machine, which encodes a high resolution image, The present invention can be applied to image recording / reproducing apparatuses such as IP telephones and mobile telephones, and image encoding apparatuses mounted on video transmission apparatuses such as digital broadcasting and cable television.

DESCRIPTION OF SYMBOLS 100 Camera part 101 Lens group 102 Image pick-up part 103 A / D conversion part 104 Camera image signal process part 105 Camera control part 200 Recording / reproducing part 201 Memory control part 202 Image output part 203 Image reduction part 204 GUI multiplexing part 205 Compression / decompression part 206 Recording / playback processing unit 207 System control unit 208 Recording control unit 300 External memory 301 Output terminal 302 Display device 310 Reduced image area 311 Divided image area 320 Super-resolution processing unit 330 External memory 500 Recording / playback unit 501 Memory control unit 502 Playback processing unit 503 Separation / decompression unit 504 Playback control unit 505 System control unit 506 Image reduction unit 507 Compression multiplexing unit 508 Recording processing unit 509 Recording control unit

Claims (24)

An image signal having a first screen resolution input continuously in time is encoded using an encoding unit capable of encoding an image signal having a second screen resolution smaller than the first resolution. An encoding device that performs encoding,
An image reduction unit that reduces a frame included in an image signal having the first screen resolution that is continuously input in time to generate a reduced image of the second screen resolution;
An image dividing unit for dividing a frame included in the image signal into a plurality of divided images having the second screen resolution;
The image reduction unit and the image division unit are controlled to generate a reduced image based on M (M is an integer) frames that are temporally continuous among the frames included in the image signal, and in time series By alternately repeating the first process to be output and the second process to generate a divided image based on N (N is an integer) consecutive frames of the M frames and output them in time series An image signal control unit that generates a converted image and outputs the converted image;
The converted image output from the image signal control unit is input, the divided image and the reduced image are compressed and encoded into separate streams, and the divided compressed image stream obtained by compressing the divided image and the reduced image are compressed. And an encoding unit that outputs a reduced compressed image stream obtained in this manner. The image encoding device according to claim 1, wherein the image signal control unit outputs the number M of frames processed in the image reduction unit and the number N of frames processed in the image dividing unit as frame number information. The image encoding device according to claim 1, wherein the image signal control unit outputs pixel position relationship auxiliary data indicating a correspondence relationship between a pixel position of the reduced image and a pixel position of the divided image. When encoding the reduced image, the encoding unit inserts the reduced compressed image stream having an increased frame rate by inserting a picture composed of skipped macroblocks between the compressed image streams of the reduced image. The image encoding device according to claim 1 to generate. The image encoding according to claim 4, wherein the encoding unit performs code amount control using a buffer size smaller than a buffer size in a standard of an encoding method in code amount control when encoding the reduced image. apparatus. The encoding unit adds a stuffing to a picture composed of skipped macroblocks when encoding the reduced image, so that the data amount is approximately the same as that of other interframe prediction encoded pictures. The image encoding device according to claim 4, wherein the reduced compressed image stream is generated. The image reduction unit reduces each of the N frames to generate a divided-part reduced image having a resolution smaller than the first resolution;
The encoding unit performs compression encoding so that the reduced compressed image stream includes the reduced image and the divided unit reduced image.
The image encoding device according to claim 1. The image encoding according to claim 1, wherein the encoding unit performs an inter-frame encoding process when encoding the reduced image, and performs an intra-frame encoding process when encoding the divided image. apparatus. The image coding apparatus according to claim 1, further comprising a multiplexing unit that multiplexes the divided compressed image stream as user data of the reduced compressed image stream. The image coding apparatus according to claim 1, further comprising a multiplexing unit that multiplexes the divided compressed image stream as a packet having another PID in the reduced compressed image stream. The image coding apparatus according to claim 2, further comprising a multiplexing unit that multiplexes the frame number information as user data on at least one of the divided compressed image stream and the reduced compressed image stream. The image coding apparatus according to claim 2, further comprising a multiplexing unit that multiplexes the frame number information as a packet having another PID in at least one of the divided compressed image stream and the reduced compressed image stream. The image coding apparatus according to claim 2, further comprising: a multiplexing unit that multiplexes the frame number information as a descriptor in MPEG Systems on at least one of the divided compressed image stream and the reduced compressed image stream. The image coding apparatus according to claim 3, further comprising a multiplexing unit that multiplexes the pixel position relationship auxiliary data as user data on at least one of the divided compressed image stream and the reduced compressed image stream. The image coding apparatus according to claim 3, further comprising a multiplexing unit that multiplexes the pixel position relationship auxiliary data as a packet having another PID in at least one of the divided compressed image stream and the reduced compressed image stream. The image coding apparatus according to claim 3, further comprising a multiplexing unit that multiplexes the pixel position relationship auxiliary data as a descriptor in MPEG Systems on at least one of the divided compressed image stream and the reduced compressed image stream. A reduced compressed image stream in which a reduced image having a resolution smaller than the first resolution and an image signal having a resolution smaller than the first resolution and composed of skipped macroblocks are compression-encoded;
An input unit that receives, as an input, a divided compressed image stream obtained by compressing and encoding a divided image obtained by dividing an image signal having the first resolution into a plurality of frames having a resolution smaller than the first resolution; ,
An expansion unit that restores an image signal having the first resolution before division based on a plurality of divided images included in the divided compressed image stream;
A reduction unit that reduces the image signal restored in the extension unit to the resolution of the divided image and generates a reduced image after reproduction;
A compression unit that compresses and encodes the reduced image after reproduction to generate a reduced compressed image stream after reproduction;
A code processing unit that generates and outputs a converted compressed image stream by replacing each picture included in the reduced image stream after reproduction with an image signal composed of skipped macroblocks included in the reduced compressed image stream;
A coding conversion apparatus comprising: The image encoding device further includes a recording unit that records the divided compressed image stream and the reduced compressed image stream on a recording medium having a predetermined file system,
The recording unit generates a divided image folder and a reduced image folder on the file system, and records the divided compressed image stream in the divided image folder, the reduced compressed image stream, and the encoding Recording reduced image control information used when reproducing the reduced compressed image stream generated in the recording unit in the reduced image folder,
The image encoding device according to claim 1. The image signal control unit causes the image reduction unit to generate a first reduced image group based on temporally continuous M (M is an integer) frames among frames included in the image signal. And generating a second reduced image group based on N (N is an integer) frames continuous to the M frames,
The encoding unit generates the reduced compressed image stream by compressing and encoding the first reduced image and the second reduced image in time series, and reproduces the reduced compressed image stream. Generate reduced image control information used for
The recording unit generates a divided image folder and a reduced image folder on the file system, and records the divided compressed image stream in the divided image folder, the reduced compressed image stream, and the encoding Recording reduced image control information used when reproducing the reduced compressed image stream generated by the recording unit in the reduced image folder,
The image encoding device according to claim 1. The encoding unit compresses and encodes a divided image of an image signal having the first resolution before the reduction of the second reduced image, using a motion vector detected when the second reduced image is encoded. It is characterized by
The image encoding device according to claim 19. A compressed image having a resolution smaller than the first resolution recorded on the removable recording medium and an image signal having a resolution smaller than the first resolution and composed of skipped macroblocks are compressed and encoded. A reduced compressed image stream, and a divided compressed image stream obtained by compression-encoding a divided image obtained by dividing an image signal having the first resolution into a plurality of frames having a resolution smaller than the first resolution; , Reading from the recording medium,
A decompression unit that restores an image signal before reduction based on a reduced image included in the reduced compressed image stream, and restores an image signal before division from the divided image included in the divided compressed image stream;
An output unit for outputting the restored image signal in time series;
An image reproduction apparatus comprising:   The decompression unit uses the image signal restored from the divided image among the image signals output before and after when the reduced image related to restoration is output in time series, and the reduced image related to the restoration is The image reproducing device according to claim 21, wherein the image reproducing device is restored. An image signal having a first screen resolution input continuously in time is encoded using an encoding unit capable of encoding an image signal having a second screen resolution smaller than the first resolution. An encoding method for performing
An image reduction step of generating a reduced image of the second screen resolution by reducing a frame included in the image signal having the first screen resolution inputted continuously in time;
An image dividing step of dividing the frame included in the image signal into a plurality of divided images having the second screen resolution;
The image reduction step and the image division step are controlled, and a reduced image is generated based on M (M is an integer) frames that are temporally continuous among the frames included in the image signal. By alternately repeating the first process to be output and the second process to generate a divided image based on N (N is an integer) consecutive frames of the M frames and output them in time series An image signal control step of generating a converted image and outputting the converted image;
The converted image output from the image signal control unit is input, the divided image and the reduced image are compressed and encoded into separate streams, and the divided compressed image stream obtained by compressing the divided image and the reduced image are compressed. And an encoding step for outputting a reduced compressed image stream obtained in this manner. An image signal having a first screen resolution input continuously in time is encoded using an encoding unit capable of encoding an image signal having a second screen resolution smaller than the first resolution. An integrated circuit that performs
An image reduction unit that reduces a frame included in an image signal having the first screen resolution that is continuously input in time to generate a reduced image of the second screen resolution;
An image dividing unit that divides a frame included in the image signal into a divided image having the second screen resolution;
The image reduction unit and the image division unit are controlled to generate a reduced image based on M (M is an integer) frames that are temporally continuous among the frames included in the image signal, and in time series By alternately repeating the first process to be output and the second process to generate a divided image based on N (N is an integer) consecutive frames of the M frames and output them in time series An image signal control unit that generates a converted image and outputs the converted image;
The converted image output from the image signal control unit is input, the divided image and the reduced image are compressed and encoded into separate streams, and the divided compressed image stream obtained by compressing the divided image and the reduced image are compressed. And an encoding unit that outputs a reduced compressed image stream obtained in this way.

Images