JP2004242250A - Image playback device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image playback device which smoothly and rapidly played back even an image in motion by simple processing. <P>SOLUTION: The image playback device outputs image data recorded as being segmented into a plurality of frequency band components by composing them for respective band. When the image is played back at a high speed, image data (Fig.(b)) obtained by composing frequency components of a plurality of images (Fig.(a)) are decoded and played back. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image reproduction technique, and more particularly to an image reproduction technique capable of reproducing an image at high speed.
[0002]
[Prior art]
Conventionally, when reproducing temporally continuous image data such as moving image data, for example, a desired content can be searched in a short time, or the overall flow can be roughly grasped. There is a demand for performing regeneration.
[0003]
An image reproducing apparatus having such a high-speed reproducing function has also been realized. In general, a moving image is a set of images for one screen called a frame. Therefore, a high-speed reproduction function can be realized by performing thinning-out reproduction such as reproducing recorded frames every few frames.
[0004]
However, when such thinning-out reproduction is performed, as the reproduction speed increases (that is, as the number of frames to be thinned out increases), the reproduced moving image becomes jerky, and the visual characteristics cannot be said to be good.
[0005]
As a high-speed reproduction method with improved visual characteristics, for example, Patent Literature 1 discloses that only intra-coded frames among encoded image data in which one intra-coded frame and a plurality of inter-coded frames alternately appear. A high-speed playback method is disclosed. Patent Document 1 describes that an intra-coded frame is decomposed into partial images called slices obtained by dividing the frame in the horizontal direction, and reproduction is performed at a higher speed by reproducing one slice / frame. (FIGS. 1-3).
[0006]
Patent Document 2 proposes a method for generating a one-frame composite image from partial images of a plurality of continuous frames when reproducing a moving image encoded in the MPEG format at a high speed, and reproducing the composite image. (FIG. 6).
[0007]
[Patent Document 1]
JP-A-7-162851
[Patent Document 2]
JP 2001-352524 A
[0008]
[Problems to be solved by the invention]
The high-speed reproduction method proposed in these patent documents can visually and smoothly display the motion of an image even during high-speed reproduction, as compared with simple frame thinning. However, in each of these proposals, a plurality of frame images are sliced (divided) into a rectangular shape, and a slice image of each image is combined to form a composite image for one frame. The mixed images are mixed.
[0009]
For this reason, when a composite image is generated from four frames of images having motion between frames as shown in FIG. 12, the composite image has a shift as shown in FIG. 13, and the reproduced image is visually unfavorable.
[0010]
Further, in order to perform high-speed reproduction while generating such a composite image, it is necessary to read, combine, and reproduce a plurality of images within a normal reproduction processing time for one frame. In some cases, it has increased or required a large amount of resources such as a memory. For example, in the method proposed in Patent Document 2, it is necessary to read data at twice the speed at the time of double speed reproduction.
[0011]
SUMMARY OF THE INVENTION The present invention has been made in view of such problems of the related art, and an object of the present invention is to provide an image reproducing apparatus capable of performing a simple process and performing a smooth display even at a high-speed reproduction regardless of the size of the motion. To provide.
[0012]
[Means for Solving the Problems]
In order to achieve the above object, an image reproducing apparatus according to the present invention is an image reproducing apparatus which performs band synthesis on image data recorded by being divided into a plurality of frequency band components and outputs the image data. An acquisition unit, a reproduction output unit configured to perform band synthesis on the acquired image data, and a control unit configured to control the data acquisition unit and the reproduction output unit in accordance with a reproduction condition. Is an integer of 2 or more). When one image is band-synthesized and reproduced and output from the image data constituting the two images, the data acquisition unit performs the processing from each of the image data constituting the (m-1) images. The data of some frequency band components among the plurality of frequency band components, and the data of at least some of the frequency band components of the plurality of frequency band components are obtained from the image data constituting the remaining one image. Each is acquired, the reproduction output means, based on those acquired data, one image and band synthesis, and controls to output.
[0013]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, the present invention will be described in detail based on preferred embodiments with reference to the accompanying drawings.
■ (First embodiment)
FIG. 1 is a block diagram showing a configuration example of a recording system 100 in a recording / reproducing apparatus as an example of an image reproducing apparatus of the present invention, and FIG. 4 is a block diagram showing a configuration example of a reproducing system 400 of the same apparatus. .
[0014]
■ (Configuration of the recording system 100)
First, the configuration and operation of the recording system 100 will be described with reference to FIG. In FIG. 1, reference numeral 101 denotes a lens unit including a lens and a diaphragm, and reference numeral 102 denotes an imaging unit including an imaging element such as a CCD or a CMOS sensor. A memory unit 103 temporarily stores data output from the imaging unit 102. Reference numeral 104 denotes an image data creation unit that performs a predetermined process on the image data temporarily stored in the memory unit 103. Reference numeral 105 denotes a camera control unit that integrally controls the lens unit 101 to the image data creation unit 104.
[0015]
■ (Operation of the recording system 100)
Light incident through the lens unit 101 is imaged on the image sensor of the image capturing unit 102. Then, in the imaging unit 102, the imaging device is driven by a drive control signal from the camera control unit 105. The output signal of the image sensor is A / D-converted in the image capturing unit 102 and then stored in the memory unit 103. The stored image data is input to the image data creation unit 104 by a memory read control signal from the camera control unit 105.
[0016]
The image data creation unit 104 performs processing such as pixel interpolation processing, color calculation processing, and gamma processing to create baseband image data for one screen. The image data created by the image data creation unit 104 is input to an image compression unit 120 described later.
[0017]
In the present embodiment, as the image compression / encoding processing performed by the image compression unit 120, a description will be given assuming that image data is divided into a plurality of bands and compression / encoding processing conforming to the so-called JPEG2000 system for performing compression / decompression is performed. Since the JPEG2000 compression encoding method is described in detail in ISO / IEC15444, only the parts necessary for understanding the present embodiment will be described here.
[0018]
The image data created by the image data creating unit 104 is input to the discrete wavelet transform unit 106 of the image compressing unit 120 and subjected to two-dimensional discrete wavelet transform.
[0019]
FIG. 2A is a diagram illustrating a configuration example of the discrete wavelet transform unit 106 that divides an input signal into bands. In the figure, H0 and H1 are FIR filters, H0 has a low-pass characteristic, and H1 has a high-pass characteristic. Further, a circular portion having a downward arrow symbol represents a downsampler. The input multi-level image signal is processed by filters H0 and H1, divided into signals of different frequency bands, and then down-sampled 2: 1. Since this configuration is equivalent to a two-channel filter bank and is already well known, a detailed description is omitted here.
[0020]
In FIG. 2A, the conversion processing in the horizontal direction and the vertical direction is performed on the input multi-valued image signal of the baseband as one set. Further, after the first set of processing is completed, the same processing is repeatedly performed on the signal (LL1) having the lowest frequency band, thereby finally outputting a series of data belonging to seven different frequency bands.
[0021]
FIG. 2B illustrates a state in which the input multi-valued image signal is divided into different frequency bands (sub-bands) as a result of performing the conversion processing in the discrete wavelet transform unit 106 having the configuration illustrated in FIG. , And shows an example of labeling each frequency band as HH1, HL1, LH1,..., LL2. In the following description, one set of conversion processing in the horizontal and vertical directions is considered as one level of decomposition, and each frequency band HH1, HL1, LH1,..., LL2 is called a subband conversion coefficient.
[0022]
The subband transform coefficients output from the discrete wavelet transform unit 106 are output to the quantization unit 107 in FIG. The quantization unit 107 quantizes the sub-band transform coefficients output from the discrete wavelet transform unit 106. FIG. 3 is a diagram illustrating a relationship between an input value and an output value in the quantization unit 107. As described above, the quantization unit 107 linearly quantizes the subband transform coefficients, converts the coefficients into a quantization index, and outputs the result to the entropy encoding unit 108 that follows.
[0023]
The entropy encoding unit 108 decomposes the input quantization index into bit planes, performs entropy encoding in units of pit planes, and outputs a code stream. The code stream is processed into a predetermined file format by adding header information and the like by the image encoded data output unit 109, and then output to the recording device 110. The recording device 110 includes a memory card, an optical disk, a magnetic disk, and the like. Further, a computer or a server connected via a network may be used as the recording device 110.
[0024]
■ (Configuration of reproduction system 400)
Next, the reproduction system 400 of the recording / reproduction device according to the present embodiment will be described. FIG. 4 is a block diagram showing a configuration example of a reproduction system 400 that expands and reproduces image data encoded by the recording system 100 shown in FIG.
[0025]
4, reference numeral 110 denotes the recording apparatus shown in FIG. 1, 401 denotes a code reading unit, 402 denotes an entropy decoding unit, 403 denotes an inverse quantization unit, 404 denotes a buffer memory for decoding, and 405 denotes an inverse discrete wavelet transform unit. Yes, 402 to 405 constitute an image decompression unit 410 for decompressing the compressed image data. Reference numeral 406 denotes a display buffer memory unit; 407, an image output unit such as an LCD and a CRT; and 408, a reproduction control unit that comprehensively controls the reproduction system 400. An operation switch 409 for the user to specify various playback operations is connected to the playback control unit 408.
[0026]
■ (Operation during normal playback)
First, the operation during normal reproduction (constant speed reproduction) will be described. The image data compressed and encoded by the recording system 100 and recorded in the recording device 110 is read by the code reading unit 401. The code reading unit 401 analyzes the header included in the data stream and extracts parameters necessary for the subsequent decoding processing.
[0027]
After that, the entropy decoding unit 402 performs a decoding process reverse to that of the entropy coding unit 108 to decode the quantized coefficient values. The decoded coefficient value is output to the subsequent inverse quantization unit 403. The inverse quantization unit 403 inversely quantizes the input coefficient value using the same quantization characteristics as the quantization unit 107, and stores the resulting discrete wavelet transform coefficient in the decoding buffer memory unit 404.
[0028]
Next, with reference to FIG. 5 showing a detailed configuration example of the inverse discrete wavelet transform unit 405, a procedure of performing an inverse discrete wavelet transform on the transform coefficients stored in the decoding memory 404 in the inverse discrete wavelet transform unit 405 will be described. .
[0029]
The inverse discrete wavelet transform unit 405 has a symmetric configuration with the discrete wavelet transform unit 106 described with reference to FIG. 2, and is configured by a 1: 2 upsampler and filters G0 and G1, and performs an inverse transform process. That is, the upsampler performs the reverse process of the downsampler, the filter G0 performs the reverse process of the filter H0, and the filter G1 performs the reverse process of the filter H1.
[0030]
The inverse discrete wavelet transform unit 405 first reads out the subband transform coefficients of LL2 and LH2 stored in the decoding buffer memory unit 404 in the vertical direction and upsamples them by 1: 2, and then G0, LH2 for LL2. Is subjected to the filtering process by G1 and added. The same processing is performed for HL2 and HH2. Next, the above processing results are read out in the horizontal direction, and G0 after up-sampling is applied to the results obtained from LL2 and LH2, and G1 after up-sampling is applied to the results obtained from HL2 and HH2. And add. With the above processing, one-level synthesis is completed.
[0031]
By performing the above processing at all levels, baseband image data can be decoded. The baseband image data combined by the inverse discrete wavelet transform unit 405 is input to the display buffer memory unit 406. The image output unit 407 outputs the data from the display buffer memory unit 406 as a visible image to a display device such as an LCD or a CRT monitor.
[0032]
As described above, at the time of normal reproduction, the reproduction control unit 408 decodes all the sub-band transform coefficients of each frame as shown in FIG. 6A, and decodes and displays them in chronological order as shown in FIG. 6B. Perform control. In FIG. 6, n, n + 1,... Indicate the order of continuous frame images. In this example, the image of the (n + 1) th frame is spaced 1/30 second from the image of the nth frame. Is displayed.
[0033]
■ (Operation during high-speed playback)
Next, an operation for performing high-speed reproduction (search) using the reproduction system 400 of FIG. 4 will be described.
For example, when an instruction for high-speed reproduction (here, double-speed reproduction) is given from the operation switch 409, the reproduction control unit 408 notifies the code reading unit 401 that double-speed reproduction is to be performed. The code reading unit 401 reads the subband conversion coefficient for each frame from the recording device 110 according to the correspondence between the notified playback speed and a predetermined frame reading method during high-speed playback.
[0034]
That is, at the time of normal reproduction, the conversion coefficients of all the sub-bands are read out for each frame and demodulated and reproduced, but at the time of high-speed reproduction, the sub-band conversion coefficients of all the sub-bands and the sub-band conversion coefficients of some sub-bands are read. There is a frame to read only. In the present embodiment, when the reproduction at the m-times speed (m is an integer of 2 or more) is instructed, the subband conversion coefficients of all the subbands are read out only for one frame every m frames, and the remaining (m-1) frames are read. For, only the sub-band transform coefficients of the LL sub-band at the lowest level (LL 2 sub-band in this embodiment because two-level discrete wavelet transform is performed) are read out.
[0035]
Since the playback is now double speed, as shown in FIG. 7A, a frame (n, n + 2, n + 4) for reading only the LL2 subband conversion coefficient and a frame (n + 1, n + 3) for reading all the subband conversion coefficients are used. n + 5) are present alternately. Since the time required to read only the LL2 sub-band transform coefficients is sufficiently shorter than reading the transform coefficients of all the sub-bands, m-times playback as described in Patent Document 2 performs m times the normal playback. It is not necessary to perform high-speed reading such as reading at a high speed, and the code reading unit 401 may perform reading at a slightly faster speed than during normal reproduction even during high-speed reproduction.
[0036]
In order to read only the LL2 sub-band conversion coefficient, the header of the recorded image data is analyzed, and the sub-band conversion coefficient data of the sub-band (shaded area) other than the LL 2 sub-band in FIG. Processing not to be read from 110 is performed.
[0037]
The read data of each subband transform coefficient decodes the coefficient value quantized by the entropy decoding unit 402. The decoded coefficient value is output to the subsequent inverse quantization unit 403. The inverse quantization unit 403 inversely quantizes the input coefficient value, and stores the obtained discrete wavelet transform coefficient in the decoding buffer memory unit 404.
[0038]
After that, the reproduction control unit 408 combines m consecutive frames in the decoding buffer memory unit 404 to generate one combined frame. That is, the (m-1) frame from which only the transform coefficients of the LL2 subband are read and one frame from which the transform coefficients of all the subbands are read are combined to generate one combined frame.
[0039]
In the present embodiment, the synthesized frame is generated by weighted averaging the LL2 subband conversion coefficient of the frame from which the conversion coefficients of all the subbands have been read and the LL2 subband conversion coefficient of the remaining (m-1) frames.
[0040]
That is, as shown in FIG. 7B, the LL2 component LL2 (n + 1) of the (n + 1) th frame and the LL2 component LL2 (n) of the nth frame are multiplied by predetermined weighting factors α and β, respectively, and added. Then, LL (n + 1) ′, which is the LL subband transform coefficient after synthesis, is calculated. That is,
LL (n + 1) '= α × LL (n + 1) + β × LL (n)
Is performed.
[0041]
In the present embodiment, the coefficients α and β satisfy the relationship of α + β = 1.0. Therefore, for example, when combining by taking a simple average value, α = 0.5 and β = 0.5, and when it is desired to emphasize the LL component of the (n + 1) th frame, for example, α = 0. 7. A coefficient value is determined so that α> β such as β = 0.3, and a weighted average calculation is performed. The sub-band transform coefficient values for one frame in which the LL sub-band transform coefficients are weighted and averaged are converted by the inverse discrete wavelet transform unit 405 into baseband image data. Thereafter, the image data is input to the display buffer memory unit 406 and output to the image output unit 407 as a visible image as shown in FIG.
[0042]
As described above, according to the control of the playback control unit 408, the playback system 400 uses only the conversion coefficient of a specific subband (LL subband in the present embodiment) as shown in FIG. A frame to be read and a frame from which the transform coefficients of all the subbands are read out are alternately provided, and as shown in FIG. Generate Then, the synthesized frame is decoded into a baseband image signal and displayed. By reading, combining, and decoding the subband coefficients for two consecutive frames in one frame period (1/30), double speed reproduction can be realized as shown in FIG. 7C.
[0043]
As shown in FIG. 7C, the image displayed by the high-speed reproduction according to the present embodiment includes the image components of the nth frame and the (n + 2) th frame in which the information is completely lost by the simple frame thinning method. Since the after-image is displayed on the reproduced images of the (n + 1) -th and (n + 3) -th frames, respectively, a natural display effect can be obtained even when the double-speed reproduction is displayed.
[0044]
Further, as described above, in the present embodiment, when reading out the sub-band conversion coefficients recorded in the recording apparatus, one of the m successive frames recorded during m-times speed reproduction is one sub-band of all sub-bands. The transform coefficients are read out, and only some of the sub-band transform coefficients (the LL2 sub-band transform coefficients in the present embodiment) need to be read out for the (m-1) thinned-out frames. Therefore, it is not necessary to read out all the conversion coefficients for all of the m frames, which has the effect of reducing the memory capacity used and the power consumption.
[0045]
It is clear that such a high-speed reproduction method is applicable not only to 2 × -speed reproduction but also to m × -speed (m is an integer of 2 or more) reproduction.
For example, in the case of performing triple speed reproduction, the processing may be performed as shown in FIG. As can be seen from comparison with the processing at the time of 2 × speed reproduction shown in FIG. 7, at the time of 3 × speed reproduction, two out of three consecutive frames are frames from which only the LL2 subband conversion coefficient is read, and the LL2 subband of three frames A common process is performed except that the transform coefficients are weighted and averaged.
[0046]
As shown in FIG. 8B, the weighted averaging operation in the decoding buffer memory unit 404 includes the LL2 component LL2 (n + 2) of the (n + 2) -th frame and the LL2 component LL2 (n + 1) of the (n + 1) -th frame. The LL2 component LL2 (n) of n frames is multiplied by a predetermined coefficient α, coefficient β, and coefficient γ, respectively, and added. That is, the combined LL2 subband coefficient LL2 (n + 2) ′ is
LL2 (n + 2) '= α × LL2 (n + 2) + β × LL2 (n + 1) + γ × LL2 (n)
Is required.
[0047]
In this case, α + β + γ = 1.0. For example, when a simple average value is calculated, α = 0.33, β = 0.33, and γ = 0.33. To emphasize the LL component of the (n + 2) th frame, for example, α = 0.5 , Β = 0.3 and γ = 0.2 to perform the calculation of the weighted average. Each of the weighted averaged subband transform coefficient values is converted into baseband image data by the inverse discrete wavelet transform unit 405. Thereafter, the image data is input to the display buffer memory unit 406 and output to the image output unit 407 as a visible image as shown in FIG.
[0048]
By reading, combining, and decoding the subband coefficients for three consecutive frames in one frame period (1/30), triple-speed reproduction can be realized.
[0049]
Also in the display image shown in FIG. 8C, the image component of the nth frame or the (n + 1) th frame, for which the information has been completely lost by the conventional simple frame thinning method, is the decoded image of the (n + 2) th frame. In this case, the image is displayed as an afterimage, and even when 3 × speed reproduction is displayed, the reproduction display can be performed with a natural feeling.
[0050]
Thereafter, similarly, in the case of m-times speed reproduction, a weighted average calculation of LL components for m frames is performed using m weighted average coefficients, and the obtained subband coefficients are converted into baseband coefficients by the inverse discrete wavelet transform unit 405. By converting the image data into image data, an afterimage effect can be obtained, and m-times speed reproduction with little visual discomfort can be performed even when the motion between frames is large.
[0051]
(1) (Modification of the first embodiment)
In the above-described example, for a frame (thinned-out frame) from which only a part of the sub-band transform coefficients are read, only the LL2 sub-band transform coefficient is read, and a composite frame is generated by weighted averaging of the LL2 sub-band transform coefficients. .
However, the type and number of subband conversion coefficients to be read out for a thinned frame can be arbitrarily set as long as they are not all the subband coefficients.
[0052]
That is, as shown in FIGS. 9A and 9B, in addition to the LL2 subband conversion coefficients, the subband conversion coefficients HL2, LH2, and HH2 are also read out, and these subband conversion coefficients are weighted average. Alternatively, a composite frame may be generated and reproduced.
In this case, the weighted average calculation is performed in subband units. Although FIG. 9 shows the case of 2 × speed reproduction, the same processing may be performed in 3 × or higher speed reproduction.
[0053]
As described above, according to the present embodiment, when high-speed playback of an image that has been compression-encoded for each of a plurality of frequency band components, a part of the sub-band transform coefficient representing the image of one frame to be played back is used. By generating from the subband transform coefficients of a plurality of consecutive frames, smooth high-speed reproduction with a simple process and with little visual discomfort can be performed regardless of the movement between frames.
[0054]
■ (Second embodiment)
In the first embodiment, a combined frame is generated by weighted averaging of subband transform coefficients common to each frame to be combined. On the other hand, the present embodiment is characterized in that different subband transform coefficients are read from each frame to be synthesized and one synthesized frame is generated.
[0055]
Note that the high-speed reproduction method described in the present embodiment can also be performed by the recording / reproducing apparatus described in the first embodiment, and a description of the configuration of the recording / reproducing apparatus will be omitted. Also, the processing at the time of reproduction at the normal speed is common to that of the first embodiment, and therefore, only the operation at the time of high-speed reproduction will be described below.
[0056]
As in the first embodiment, first, double speed reproduction will be described. For example, when there is an instruction to perform double-speed playback from the operation switch 409 in FIG. 4, the playback control unit 408 notifies the code reading unit that double-speed playback is to be performed. The code reading unit 401 reads the subband conversion coefficient for each frame from the recording device 110 according to the correspondence between the notified playback speed and a predetermined frame reading method during high-speed playback.
[0057]
That is, at the time of normal reproduction, the conversion coefficients of all sub-bands are read out for each frame and demodulated and reproduced. At the time of high-speed reproduction, the sub-band conversion coefficients are read out for sub-bands different from each other in the frame to be synthesized. In the present embodiment, when m-times speed reproduction (m is an integer of 2 or more) is instructed, subband conversion coefficients of subbands different from each other from consecutive m frames, and Reading is performed so that sub-band conversion coefficients for all types of sub-bands can be obtained by the band conversion coefficients.
[0058]
Therefore, at the time of 2 × speed reproduction, as shown in FIGS. 10A and 10B, for example, as shown in FIGS. 10A and 10B, the second level (LL2, HL2, HH2, LH2) starts from one frame of the coded image data of two consecutive frames. The sub-band conversion coefficients are required to read out the sub-band conversion coefficients of the first level (HL1, HH1, LH1) from the other frame to synthesize one frame from the image data of two consecutive frames. Sub-band transform coefficients of all the sub-bands are obtained.
[0059]
That is, with respect to the n-th frame and the (n + 1) -th frame, the HL1 (n), HH1 (n), and LH1 (n) sub-band conversion coefficient data are read for the image data of the n-th frame, and the following (n + 1) -th ) With respect to the image data of the frame, the subband conversion coefficient data of LL2 (n + 1), HL2 (n + 1), HH2 (n + 1), LH2 (n + 1) is read.
[0060]
As described above, data of different subband conversion coefficients are read from image data of two consecutive frames so that all the subband conversion coefficients for one screen are combined as shown in FIG. 10B.
[0061]
The read data of each subband transform coefficient decodes the coefficient value quantized by the entropy decoding unit 402. The decoded coefficient value is output to the subsequent inverse quantization unit 403. The inverse quantization unit 403 inversely quantizes the input coefficient value, and stores the obtained discrete wavelet transform coefficient in the decoding buffer memory unit 404.
[0062]
Thereafter, the subband transform coefficient values for one screen combined as shown in FIG. 10B are converted into baseband image data by the inverse discrete wavelet transform unit 405. The baseband image data is input to the display buffer memory unit 406 and output to the image output unit 407 as a visible image as shown in FIG.
[0063]
As described above, under the control of the playback control unit 408, the playback system 400 converts all the sub-bands from two consecutive frames of each frame shown in FIG. Are obtained, and the subband conversion coefficients of different subbands are read from each frame. Then, a combined frame shown in FIG. 10B is created, decoded into a baseband image signal, and displayed. By reading, combining and decoding these subband coefficients for two consecutive frames in one frame period (1/30), double speed reproduction can be realized as shown in FIG.
[0064]
As shown in FIG. 10C, the image displayed by the high-speed reproduction according to the present embodiment includes the image components of the nth frame and the (n + 2) th frame in which the information is completely lost by the simple frame thinning method. Since the after-image is displayed on the reproduced images of the (n + 1) -th and (n + 3) -th frames, respectively, a natural display effect can be obtained even when the double-speed reproduction is displayed.
[0065]
In this embodiment, at the time of m-times speed reproduction, when reading out the sub-band transform coefficients recorded in the recording device, the header of the data is analyzed, and sub-bands different from each other are selected from the m successive image data. The sub-band transform coefficients of all the sub-bands for one frame are read by combining the transform coefficients and the different sub-band transform coefficients read from the m frames. Therefore, since it is only necessary to read out some sub-band transform coefficients for each frame, it is not necessary to read out all the transform coefficients for all of the m frames at a speed of m times as shown in the related art. As in the first embodiment, there is also an effect of reducing the used memory capacity and power consumption.
[0066]
It is clear that the method of the present embodiment can be applied not only to 2 × speed reproduction but also to m × speed (m is an integer of 2 or more) reproduction as in the first embodiment.
For example, in the case of performing the triple speed reproduction, the processing may be performed as shown in FIG. As can be seen from a comparison with the processing at the time of 2 × speed reproduction shown in FIG. 10, at the time of 3 × speed reproduction, different subband conversion coefficients are used for each frame from among three consecutive frames of image data. The common processing is performed except that the sub-band transform coefficients read from each are read out so as to form the sub-band transform coefficients of all the sub-bands for one frame.
[0067]
In this example, as shown in FIG. 11A, the frame from which the sub-band transform coefficients of the first-level (HL1, HH1, LH1) sub-band transform coefficients are read, and the second levels (HL2, HH2, The frame for reading out the subband conversion coefficient of LH2) and the frame for reading out only the LL2 subband conversion coefficient are sequentially repeated. This makes it possible to obtain the subband conversion coefficients of one set of all subbands from the subband conversion coefficients read from three consecutive frames.
[0068]
The data (FIG. 11A) of the sub-band transform coefficient read from each frame is decoded into a coefficient value quantized by the entropy decoding unit 402, and is output to the subsequent inverse quantization unit 403. The inverse quantization unit 403 inversely quantizes the input coefficient value, and stores the obtained discrete wavelet transform coefficient in the decoding buffer memory unit 404. After that, the subband transform coefficient values for one screen combined as shown in FIG. 11B are converted into baseband image data by the inverse discrete wavelet transform unit 405. Thereafter, the image data is input to the display buffer memory unit 406 and output to the image output unit 407 as a visible image as shown in FIG.
[0069]
As described above, according to the control of the playback control unit 408, the playback system 400 performs three times speed playback from three consecutive frames of each frame shown in FIG. Are obtained, and the subband conversion coefficients of different subbands are read from each frame. Then, a combined frame shown in FIG. 11B is created, decoded into a baseband image signal, and displayed. By reading, combining, and decoding the subband coefficients for three consecutive frames in one frame interval (1/30), triple-speed reproduction can be realized as shown in FIG.
[0070]
As shown in FIG. 11C, the image displayed by the high-speed reproduction according to the present embodiment has image components of the n-th frame and the (n + 1) -th frame in which information is completely lost by a simple frame thinning method. Since the reconstructed image of the (n + 2) -th frame is displayed as an afterimage, a natural display effect can be obtained even when 3 × speed reproduction is displayed.
[0071]
Thereafter, in the case of m-times speed reproduction, control for reading out data of subband conversion coefficients different for each frame from image data of m frames is performed in the same manner. By converting the obtained subband coefficients into baseband image data by the inverse discrete wavelet transform unit 405, an afterimage effect of a high-frequency component is obtained. Double speed playback can be performed.
[0072]
As described above, according to the present embodiment, when an image compressed and encoded for each of a plurality of frequency band components is reproduced at high speed, a subband transform coefficient representing an image for one frame to be reproduced is converted into a plurality of subband transform coefficients. By generating without duplication from the subband transform coefficients of consecutive frames, smooth high-speed reproduction with simple processing and with little visual discomfort can be performed regardless of the movement between frames.
[0073]
[Other embodiments]
It is also possible to combine the first and second embodiments described above. In this case, for example, in the second embodiment, read control is performed so that a part of subband transform coefficients read from data of continuous m frames at the time of m-times speed reproduction is overlapped, and synthesis in the decoding memory buffer unit 404 is performed. In the assembling process, one combined frame may be generated by performing the weighted averaging process performed in the first embodiment on the overlapping subband transform coefficients.
[0074]
In the present embodiment, the JPEG2000 system using discrete wavelet transform has been described as an example of a process of dividing an image into a plurality of frequency bands and compressing the image. However, the image signal is divided into a plurality of frequency bands and compressed. The present invention can be applied to any compression method, and there is no particular limitation on the compression method itself.
[0075]
Furthermore, in the above-described embodiment, for ease of explanation and understanding of the present invention, image data compressed and encoded using a two-level two-dimensional discrete wavelet transform has been described as an example. May be converted.
[0076]
In the above-described embodiment, the description has been given only of the image reproducing apparatus including one device. However, the same function may be realized by a system including a plurality of devices.
[0077]
In addition, a software program for realizing the functions of the above-described embodiments is supplied to a system or an apparatus having a computer capable of executing the program directly from a recording medium or by using wired / wireless communication. The present invention includes a case where a computer achieves the same function by executing the supplied program.
[0078]
Therefore, the program code itself supplied and installed in the computer to implement the functional processing of the present invention by the computer also implements the present invention. That is, the present invention includes the computer program itself for implementing the functional processing of the present invention.
[0079]
In this case, any form of the program, such as an object code, a program executed by an interpreter, and script data to be supplied to the OS, may be used as long as the program has a function.
[0080]
As a recording medium for supplying the program, for example, a magnetic recording medium such as a flexible disk, a hard disk, a magnetic tape, an MO, a CD-ROM, a CD-R, a CD-RW, a DVD-ROM, a DVD-R, a DVD- There are optical / magneto-optical storage media such as RW, and nonvolatile semiconductor memories.
[0081]
As a method for supplying a program using wired / wireless communication, a computer program itself that forms the present invention on a server on a computer network, or a computer that forms the present invention on a client computer, such as a compressed file having an automatic installation function, etc. A method of storing a data file (program data file) that can be a program and downloading the program data file to a connected client computer may be used. In this case, the program data file can be divided into a plurality of segment files, and the segment files can be arranged on different servers.
[0082]
That is, the present invention also includes a server device that allows a plurality of users to download a program data file for implementing the functional processing of the present invention on a computer.
[0083]
Further, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM and distributed to users, and key information for decrypting the encryption for a user who satisfies predetermined conditions is transmitted to, for example, a homepage via the Internet. It is also possible to realize the program by supplying it by downloading it from, and using the key information to execute an encrypted program and install it on a computer.
[0084]
The functions of the above-described embodiments are implemented when the computer executes the read program, and an OS or the like running on the computer executes a part of the actual processing based on the instructions of the program. Alternatively, all the operations are performed, and the functions of the above-described embodiments can be realized by the processing.
[0085]
Further, after the program read from the recording medium is written into a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion board or the The CPU or the like provided in the function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments can also be realized by the processing.
[0086]
【The invention's effect】
As described above, according to the present invention, when reproducing an image divided into a plurality of frequency bands and recorded at a high speed, at least a part of the frame is read out of only a part of the subband transform coefficients. By doing so, high-speed reproduction can be performed without frames in which information is lost, and even in the case of high-speed reproduction of an image having motion between frames, smooth reproduction output with little visual discomfort can be realized.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a configuration example of a recording system of a recording / reproducing apparatus according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating a configuration example of a discrete wavelet transform unit 106 in FIG.
FIG. 3 is a diagram illustrating an example of input / output characteristics of a quantization unit 107 in FIG. 1;
FIG. 4 is a block diagram showing a configuration example of a reproducing system of the recording / reproducing apparatus according to the embodiment of the present invention.
FIG. 5 is a diagram illustrating a configuration example of an inverse discrete wavelet transform unit 405 in FIG. 4;
FIG. 6 is a diagram illustrating a relationship between subband transform coefficients of each frame and an image to be decoded / reproduced during normal reproduction according to the embodiment of the present invention.
FIG. 7 is a diagram illustrating the relationship between subband transform coefficients read from each frame during double-speed playback and images to be decoded and played back in the first embodiment.
FIG. 8 is a diagram illustrating a relationship between subband transform coefficients read from each frame during triple-speed playback and decoded / reproduced images in the first embodiment.
FIG. 9 is a diagram illustrating the relationship between subband transform coefficients read from each frame during double-speed playback and decoded / reproduced images in a modification of the first embodiment.
FIG. 10 is a diagram illustrating a relationship between subband transform coefficients read from each frame during double-speed playback and decoded / reproduced images in the second embodiment.
FIG. 11 is a diagram illustrating the relationship between subband transform coefficients read from each frame during triple-speed playback and decoded / reproduced images in the second embodiment.
FIG. 12 is a diagram illustrating an example of an image having motion between frames.
FIG. 13 is a diagram schematically showing a state in which the image of FIG. 12 is reproduced at a high speed by a conventional method.
[Explanation of symbols]
101: Lens unit
102: Image change
103: Memory section
104: Image data creation unit
105: Camera control unit
106: discrete wavelet transform unit
107: Quantization unit
108: Entropy coding unit
109: Image encoded data output unit
110: Recording device
401: code reading unit
402: Entropy compounding unit
403: inverse quantization unit
404: decoding buffer memory unit
405: inverse discrete wavelet transform unit
406: display buffer memory unit
407: Image output unit
408: Playback control unit
409: Operation switch

Claims (11)

An image reproducing apparatus for performing band synthesis on image data recorded by being divided into a plurality of frequency band components and outputting the data,
Data acquisition means for acquiring the image data,
A reproduction output unit that outputs the acquired image data by performing band synthesis,
Control means for controlling the data acquisition means and the reproduction output means according to reproduction conditions,
The control means, when performing band reproduction of one image from image data constituting continuous m (m is an integer of 2 or more) images and reproducing and outputting,
The data acquisition unit forms, from each of the image data constituting the (m−1) images, data of a part of the plurality of frequency band components and data of the remaining one image. From the image data to be obtained, the data of at least a part of the frequency band components of the plurality of frequency band components, respectively,
An image reproducing apparatus, wherein the reproduction output means controls the band synthesis of the one image based on the acquired data and outputs the resultant image. In the case where the control means performs band reproduction of one image from image data constituting the continuous m (m is an integer of 2 or more) images and reproduces and outputs the image,
The data acquisition unit converts the data of the same frequency band component from each of the image data constituting the (m-1) images, and converts the data of the plurality of 2. The image reproducing apparatus according to claim 1, wherein data of all the frequency band components of the frequency band components is acquired. The image reproducing apparatus according to claim 2, wherein the same frequency band component is a lowest frequency band component. The image reproducing apparatus according to claim 2, wherein the same frequency band component is a plurality of frequency band components including a lowest frequency band component. The reproduction output unit performs predetermined processing on each of the data of the same frequency band component and data of a frequency band component equal to the same frequency band component included in the image data of the remaining one image. The image reproducing apparatus according to any one of claims 2 to 4, wherein the one image is band-synthesized and output after performing calculation and synthesis. The image reproducing apparatus according to claim 5, wherein the predetermined operation is a weighted average operation. The data of the frequency band component obtained from each of the image data forming the (m-1) images and the data of the frequency band component obtained from the image data forming the remaining one image are all 2. The image reproducing apparatus according to claim 1, wherein data of different frequency band components and a combination of acquired data of frequency band components is data of all frequency band components of the plurality of frequency band components. . 8. The image reproducing apparatus according to claim 1, wherein the image data is divided into the plurality of frequency band components by a two-dimensional discrete wavelet transform of a plurality of levels. Further, means for taking an image,
The image reproducing apparatus according to claim 1, further comprising a conversion unit configured to divide the photographed image into a plurality of frequency components and record the divided image. When the reproduction condition is a reproduction at a speed higher than a normal reproduction speed, the control unit is configured to output one image from the image data constituting the continuous m (m is an integer of 2 or more) images. 10. The image reproducing apparatus according to claim 1, wherein the data acquisition unit and the reproduction output unit are controlled so as to synthesize and reproduce and output. The image reproducing apparatus according to claim 10, wherein the reproduction condition is input by an operation unit operable by a user.

Images