JP4143322B2 - Image reproducing apparatus and image reproducing method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an apparatus capable of recording and reproducing images, and more particularly to an image reproducing apparatus and a reproducing method having a high-speed moving image reproduction function.
[0002]
[Prior art]
2. Description of the Related Art In recent years, recording / playback apparatuses such as digital still cameras and digital video movies have become mainstream products that use Motion JPEG, MPEG, or DV as a moving image recording format.
[0003]
In these products, when a moving image recorded on a recording medium is reproduced in a time shorter than the actual time of the recorded moving image (high-speed reproduction), the time given for the decoding process of each frame in the moving image To reduce the amount of data, the video processing rate can be reduced by reducing the amount of data by reducing the frame rate of the moving picture, reducing the number of frames to be decoded within a unit time, or by reducing the resolution of the image for each frame. Shortened and realized high-speed playback display.
[0004]
[Problems to be solved by the invention]
However, if the frame rate is reduced and playback is performed at a high speed to reduce the decoding processing time as in the prior art described above, it is difficult to see the frame-by-frame video display, and the images between the frames are lost and you want to see them. The scene was sometimes overlooked.
[0005]
In addition, when the resolution was lowered uniformly, the image quality of the entire screen was lowered, resulting in a blurred image, and the details of the subject of interest could not be confirmed at all.
[0006]
Further, in any case, the higher the playback speed, the worse the above-mentioned problem, and thus the playback speed may be limited.
[0007]
[Means for Solving the Problems]
As means for solving such a problem, the present invention has means having the following configuration.
[0008]
The image reproduction apparatus of the present invention is a reproduction means for reproducing image data composed of a plurality of screens, and is encoded by being shifted up on a bit plane from the other areas in the screen included in the image data. Detection means for detecting a processing region, decoding means for decoding the reproduced image data, reproduction speed selection means capable of selecting a reproduction speed in the reproduction means, and faster than normal reproduction by the reproduction speed selection means When a predetermined playback speed is selected, the decoding is performed such that a part of the processing region detected by the detection unit is shifted down on the bit plane, so that the area of the processing region is reduced and decoding is performed. And a control means for controlling the means.
[0009]
The image reproduction method of the present invention includes a reproduction step of reproducing image data composed of a plurality of screens, and an encoding process by shifting up on a bit plane with respect to other areas in the screen included in the image data. A detection step for detecting the processed region, a decoding step for decoding the reproduced image data, a reproduction speed selection step for selecting a reproduction speed in the reproduction step, and a normal reproduction in the reproduction speed selection step. When a fast predetermined reproduction speed is selected, the processing area detected in the detection step is shifted down on the bit plane, so that the area of the processing area is reduced and decoded. And a control step for controlling the decoding process in the decoding step.
[0010]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.
[0011]
FIG. 1 is a schematic block diagram of an image reproducing apparatus according to the present invention.
[0012]
Reference numeral 100 is an image input unit, 101 is an encoding circuit for compressing / encoding an input image, 102 is a recording processing circuit for encoded image data, 103 is a writing / reading means, and a recording medium such as a hard disk or a magnetic disk A recording / reproducing unit, 104 a reproduction processing circuit for image data, 105 a decoding circuit for expanding / decoding reproduced image data, 106 an image display unit, 107 an external output terminal for reproduced image, 108 Is a region-of-interest detection circuit that detects the presence / absence and region (form) of the region of interest in the reproduced image data, 109 is an operation unit, 110 is a reproduction speed setting that can be set to any speed according to an operation from the operation unit Circuit 111, a servo circuit for controlling the operation (rotation) of the recording medium in the recording / reproducing unit and the operation of the writing / reading means, and 112 for each unit of the reproducing apparatus. It is a Gosuru system controller.
[0013]
Next, the operation of the block diagram of FIG. 1 will be described according to the operation state while referring to the configuration of the encoding circuit 101 and the configuration of the decoding circuit 105 shown in more detail.
[0014]
FIG. 2 shows the configuration of the encoding circuit 101 in detail. In the present embodiment, an encoding method using wavelet transform is adopted.
[0015]
During the recording operation, the component conversion unit 201 performs color space conversion on the input image data input from the image input unit 100 to the encoding circuit 101. The converted data of each color component is output after being subjected to predetermined thinning processing as necessary. When the input image data is a monochrome grayscale image, there is no need to perform component conversion. In the following description, the processing performed for each color component of the image data subjected to color space conversion as described above will be described.
[0016]
Next, the tile division unit 202 divides the input image data into a plurality of rectangular tiles having a predetermined size and outputs them. The size of the tile can be the maximum size of the entire data of each color component, and in this case, the tile division is not substantially performed.
[0017]
The discrete wavelet transform unit 203 performs a two-dimensional discrete wavelet transform on the input image data for each tile of each color component to decompose it into frequency components, and transform coefficient groups (hereinafter, referred to as a plurality of frequency bands). (Referred to as subband).
[0018]
FIG. 3 is a diagram showing the configuration of subbands output by the discrete wavelet transform unit 203, in which two-dimensional wavelet transform is recursively performed at two levels with respect to a low frequency band.
[0019]
In the discrete wavelet transform unit 203, lossless encoding and lossy encoding can be selected, and when performing lossy encoding, a real type filter whose coefficient after the wavelet transform is a real number is used as a lossless code. When it is desired to perform the conversion, an integer type filter in which the coefficient after wavelet transform is an integer is used.
[0020]
Next, the quantization unit 204 performs quantization using a quantization step set by a predetermined method for each input subband, and generates and outputs a quantization index. If it is desired to perform the above-described lossless encoding, the quantization unit 204 does not perform quantization but outputs the input transform coefficient itself.
[0021]
In the present embodiment, by providing the region of interest setting unit 205, a special region of interest (hereinafter referred to as a region of interest) is arbitrarily set for a partial region of the image. It is possible to perform encoding by changing the image quality depending on the region, and it is possible to process the region of interest with high image quality (that is, high resolution and high gradation). The region of interest here is the same as the ROI (Region Of Interest) in Jpeg2000, which is a typical encoding method using wavelet transform.
[0022]
When the region of interest is set by the region-of-interest setting unit 205, as shown in FIG. 4, a mask for specifying a transform coefficient related to the region of interest is generated for each subband. In FIG. 4, the white portion corresponds to the region of interest, and the coefficients included in this portion are predetermined bits so that the coefficients related to regions other than the region of interest are completely separated on the bit plane prior to encoding. It is shifted up by several minutes. Thus, by checking the presence of bits 1 and 0 in all the bit planes, it is possible to determine whether there is a region of interest and which region is the region of interest. The number of upshifts is added as a parameter to a predetermined marker of a code string to be described later. In addition, the set region of interest is encoded so as to have higher image quality than the other regions by the amount shifted up by the subsequent encoding process.
[0023]
Next, as shown in FIG. 5, the entropy encoding unit 206 further divides the input subband into a plurality of rectangular blocks (hereinafter referred to as code blocks) (when the subband and the rectangular block have the same size, the entropy encoding unit 206 does not divide the subband. Then, entropy encoding is independently performed with this code block as a unit to generate encoded data. At this time, the bits representing the quantization index are arithmetically encoded in order from the upper bit plane to generate encoded data.
[0024]
The code string forming unit 207 forms a code string based on the progressive form set by a predetermined method and outputs the code string. As the code string forming method here, the code string forming unit 207 selects an appropriate amount of encoded data in order from the upper bit plane of the encoded data of each code block in accordance with the progressive form to be adopted, and one or more encoded data Configure layers.
[0025]
For example, when the set progressive form is SNR scalable, the code string forming unit 207 arranges the encoded data in the order from the upper layer to the lower layer in units of layers as shown in FIG. Form. At this time, it is possible to omit the latter layer and select not to include the encoded data related to the lower bit plane. In this way, the data amount of the code string can be optimized, and the image quality of the image that is decoded and reproduced can be changed according to the data amount.
[0026]
On the other hand, when the set progressive form is spatial resolution scalable, the code string forming unit 207 arranges the encoded data in the order from the low frequency subband to the high frequency subband as shown in FIG. Form. At this time, it is also possible to select not to include the encoded data of the second half subband in the code string. In this way, the data amount of the code string can be optimized, and the resolution of an image reproduced by decoding the code string can be changed according to the data amount.
[0027]
Further, the code string forming unit 207 outputs a final code string in which a header composed of various markers is added to the code string formed according to each progressive form set as described above.
[0028]
FIG. 8 is an example illustrating the configuration of the code string of the encoded data output from the code string forming unit 207. In FIG. 8, the main header MH includes the resolution of the image to be compression-encoded, the number of color components, the bit accuracy of each component (the number of bits representing each component), the size of tiles constituting the image, and the discrete wavelet transform filter. , The number of the region of interest shifted up (number of shift bits), a parameter relating to compression encoding such as a quantization step, and a marker designating information relating to a code string configuration such as a progressive form.
[0029]
The tile header TH_i includes a marker indicating the start of the i-th tile. Further, when a parameter related to encoding in the tile is changed from a previously encoded tile, a marker for specifying the parameter is also included. BS_i is the encoded data of the i-th tile, and its arrangement is configured based on the progressive form described above.
[0030]
The code string formed as described above is recorded on the recording medium by the recording / reproducing unit 103 after being recorded by the recording processing circuit 102 in FIG.
[0031]
Each operation of the recording / reproducing unit 103 related to the recording operation and the reproducing operation is controlled by the servo circuit 111 to be appropriate.
[0032]
Subsequently, during the normal reproduction operation, the code string reproduced from the recording / reproducing unit 103 in FIG. 1 is reproduced by the reproduction processing circuit 104 and decoded by the decoding circuit 105.
[0033]
FIG. 9 shows the configuration of the decoding circuit 105 in detail. The code string input unit 901 inputs the reproduced code string, and extracts various parameters necessary for the decoding process, such as an image or tile size, a progressive form, and a quantization step. The extracted various parameters are appropriately used in the subsequent decoding process.
[0034]
The code string input unit 901 detects the shift bit number of the region of interest written in the marker by reading the code string and the distribution state of the region of interest in the image data, and sends it to the region of interest detection circuit 108 as information on the region of interest. To do.
[0035]
The actual code string (data portion) is then output to the entropy decoding unit 902.
[0036]
Note that the entire code sequence to be decoded includes code sequences for a plurality of tiles having the form of FIG. 8 described above, as many as the number of color components obtained by the component conversion unit 201 described above. In this embodiment, the decoding process is performed independently for each color component, and the code sequence of each tile constituting the color component to be decoded is sequentially decoded.
[0037]
The entropy decoding unit 902 performs a decoding process on the input code string and outputs a quantization index. In this decoding process, the quantization index in the code block is sequentially decoded from the upper bit plane, and the quantization index is restored.
[0038]
For example, at this time, when the progressive form of the code string is SNR scalable and only a predetermined number of upper layers are input, the decoding process is terminated at the input layer, and the restored value at that time Is output as a quantization index.
[0039]
The inverse quantization unit 903 performs inverse quantization on the input quantization index based on the quantization step previously read from the code string, restores the transform coefficient, and outputs it.
[0040]
The inverse discrete wavelet transform unit 904 restores the corresponding color component data (image density data when the encoding target image is a monochrome image) by performing a two-dimensional inverse discrete wavelet transform from the input transform coefficient. And output.
[0041]
At this time, when the progressive form of the code string is spatial resolution scalable and only the sub-bands of the levels encoded in the first half (for example, only LL, only LL, HL2, LH2, and HH2) are restored, The resolution of the restored color component data changes according to the level of the restored subband.
[0042]
FIG. 10 shows this state. When only the coefficients of the subband LL are decoded in the same figure, the inverse discrete wavelet transform is not substantially performed so that the coefficients of the LL are within the original data range. Output after adjustment. In this case, the restored color component data has a size of 1/4 in the horizontal and vertical directions with respect to the original resolution, as indicated by r = 0 in FIG.
[0043]
Further, when decoding is performed up to LL, HL2, LH2, and HH2 subbands, by performing inverse transformation at one level, as shown in r = 1 in the figure, the original resolution is ½ in the horizontal and vertical directions. Is restored.
[0044]
The above processing is performed in units of tiles, and the image configuration unit 905 again configures each color component data of each restored tile as color component data that constitutes one original encoding target image, and performs component inverse conversion. Output to the unit 906.
[0045]
The component inverse transform unit 906 restores and outputs image data having the color space of the original encoding target image by performing predetermined conversion on each input color component data. At this time, when the original color component data has been thinned out by the component conversion unit 201, the original color component data is converted (data interpolation) to a necessary resolution before reverse conversion.
[0046]
In the above description, when the progressive form is spatial resolution scalable, the image quality of the restored image can be controlled by limiting the layers to be decoded. In the case of SNR scalable, the resolution of the restored image can be controlled by limiting the number of subband levels to be subjected to inverse discrete wavelet transform.
[0047]
Further, regarding the region of interest, the region masked and shifted up at the time of encoding is reproduced with high image quality, and the other regions are reproduced with low image quality.
[0048]
In addition, the entropy decoding unit 902, the inverse quantization unit 903, the inverse discrete wavelet transform unit 904, and the like can intentionally control the decoding process performed by each unit by inputting a control signal from the system controller. Specifically, a control signal is input when controlling the reproduction mode of the region of interest during the high-speed reproduction operation, which will be described in detail later.
[0049]
The decoded image output from the component inverse transform unit 906 can be displayed on the display unit 106 incorporated in the apparatus, and can be output from the external output terminal 107 to an external display device, recording device, or the like. ing.
[0050]
The above is the detailed description of FIG. 1, and the recording operation and the normal reproduction operation have been described with respect to the reproduction apparatus of the present invention.
[0051]
The configuration described above is a description of processing performed in units of still images (one screen). When recording / reproducing a moving image, the above-described processing for each screen is streamed and continuously processed so as to conform to a predetermined frame rate. Alternatively, a moving image format such as Motion JPEG 2000 may be used.
[0052]
Next, the high speed reproduction operation will be described. High-speed playback is a function that performs double-speed playback in a shorter time than the recording time required when a moving image consisting of a plurality of screens is recorded.
[0053]
The high-speed playback operation can be started in response to an instruction input from the operation unit 109 by the user from a normal playback (1 × speed playback) state or a stopped state. The operation unit 109 is provided with an operation member such as a selection button or a jog shuttle as a user interface so that a desired reproduction speed can be selected or an instruction can be input.
[0054]
The reproduction speed setting circuit 110 detects an input from the operation unit 109, sets a reproduction speed according to the instruction input value, and outputs instruction information to the system controller 112.
[0055]
Based on the instruction information from the playback speed setting circuit 110, the system controller 112 controls the operation of each part of the playback device, and shifts to a high speed playback operation. Specifically, the servo 111 speeds up the recording medium rotation operation of the recording / reproducing unit 103, moves the reading means at high speed or skips, and further reads out each part of the reproduction processing circuit 104 and the decoding circuit 105 at high speed. It controls so as to correspond to the playback data to be played.
[0056]
When information on the region of interest is obtained from the code string input unit 901 in the decoding circuit 105 during the high-speed playback operation, the region of interest detection circuit 108 analyzes the presence / absence and region (form) of the region of interest, Output analysis results.
[0057]
Based on the analysis result output from the region-of-interest detection circuit 108 and the instruction information instructed from the playback speed setting circuit 110, the system controller 112 performs decoding so as to execute a decoding process according to the speed of the high-speed playback operation. The processing of each part of the circuit 105 is controlled. Specifically, when the image (screen) in which the region of interest exists is played back at high speed, the system controller 112 performs processing for the processing in the entropy decoding unit 902, the inverse quantization unit 903, the inverse discrete wavelet transform unit 904, and the like. A part that is controlled by transmitting a control signal in real time and changes the marker (parameter) that indicates the existence of the region of interest, or is shifted up on the bit plane included in the region of interest while corresponding to the screen period to be played back Is changed so that all or part of it is shifted down before decoding, so that at least part of the decoding process on the screen including the region of interest becomes invalid, or part of the high-quality region becomes invalid. The decoding process other than the area is changed to reduce the load required for the decoding process and to change the priority of the region of interest.
[0058]
Each figure of FIG. 11 is an example of a display image displayed on the display unit 106 when the decoding process of the region of interest is controlled according to the reproduction speed.
[0059]
When the video displayed as shown in FIG. 11A in normal playback is played back at double speed, the video is displayed as shown in FIG. 11B. FIG. 11B shows an example in which the region of interest in the screen maintains the same level of resolution as that during normal playback, and the other peripheral regions display a lower resolution than during normal playback. That is, both the region of interest and the peripheral region are relatively lowered in resolution, and decoding processing is performed at a frame rate corresponding to double-speed playback.
[0060]
As a result, even if double-speed playback is performed, the resolution of both the region of interest and the peripheral region is reduced. In particular, the region of interest can be reproduced and displayed with a visually recognizable resolution, and the decoding processing time can be shortened. Each screen can be decoded at a speed that can support the frame rate of double-speed playback.
[0061]
Further, when the video displayed as shown in FIG. 11 (a) is further increased in playback speed, such as triple speed playback, it is displayed as shown in FIG. 11 (c). FIG. 11 (c), by changing the parameters of the region of interest shifts down so that the outer peripheral region of interest equal to the peripheral region of the predetermined range, the decoding process as reduce the area of the region of interest as a result of It is an example. The range to be reduced is determined according to the playback speed, and the faster the playback speed, the smaller the range of the region of interest.
[0062]
As a result, even if high-speed playback such as triple-speed playback is performed, the decoding processing time of each screen can be shortened to support the frame rate of triple-speed playback, and a part of the region of interest has high resolution. High-speed playback display is possible while maintaining. Note that the decoding process of reducing the region of interest as shown in FIG. 11C may be performed regardless of the playback speed during high-speed playback, or the decoding process for lowering the resolution relatively in FIG. 11B. This may be performed only during high-speed playback that is not possible to deal with, and that is higher than a predetermined threshold.
[0063]
Furthermore, when the video displayed as shown in FIG. 11A is played back at high speed, it can also be displayed as shown in FIG. FIG. 11D shows an example in which the parameters of the region of interest are changed, the portion shifted up as the region of interest is ignored, and the decoding process is performed with the same resolution as the surrounding region.
[0064]
Thus, by ignoring the decoding process of the region of interest during high-speed playback, the decoding process can be simplified, and it is possible to cope with as high-speed playback as possible.
[0065]
As another display mode for high-speed playback, only the region of interest in the screen maintains a high resolution, and other peripheral regions have a lower resolution than usual and are decoded and displayed. It is feasible from the form.
[0066]
Further, as another display mode at the time of high-speed playback, an example in which the range of the region of interest remains the same and the update interval is changed only in the peripheral region and decoding processing is also considered. For example, in the normal playback shown in FIG. 10A, the update interval of the entire screen is every screen, but when the 2 × speed playback is performed, both the region of interest and the peripheral region are set every 4 screens. Each screen is maintained, and the update interval is increased only in the peripheral area, such as every 8 screens or every 12 screens in the peripheral area. This makes it possible to maintain a high frame rate for the region of interest of the subject of interest while shortening the decoding processing time.
[0067]
The above is the description of the present embodiment.
[0068]
In the present embodiment, the high-speed playback speed has been described as being doubled, tripled, and + integer times. However, of course, −integer multiples, that is, high-speed playback in the negative direction can be similarly realized, and Both negative and negative are not limited to integer multiples, and the speed can be set steplessly using a jog shuttle or the like.
[0069]
【The invention's effect】
According to the present invention, when image data including a processing region encoded with high image quality is reproduced at high speed, the processing in the decoded reproduced image is performed by reducing the area of the processing region and decoding the image data. A part of the area can be displayed with high image quality, and the load generated by the decoding process during the high-speed playback operation can be reduced.
[Brief description of the drawings]
FIG. 1 is a block diagram showing the configuration of an image playback apparatus according to the present invention.
FIG. 2 is a block diagram for explaining an encoding circuit of the present invention.
FIG. 3 is a diagram showing a configuration of subbands of wavelet transform.
FIG. 4 is a diagram for explaining a region of interest.
FIG. 5 is a diagram for explaining a code block;
FIG. 6 is a diagram illustrating SNR scalable.
FIG. 7 is a diagram illustrating spatial resolution scalable.
FIG. 8 is a diagram illustrating a configuration of a final code string.
FIG. 9 is a block diagram for explaining a decoding circuit of the present invention.
FIG. 10 is a diagram illustrating a state of subbands during decoding.
FIG. 11 is an example of a display screen in the present invention.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 100 Input terminal 101 Encoding circuit 102 Recording processing circuit 103 Recording / reproduction | regeneration part 104 Reproduction | regeneration processing circuit 105 Decoding circuit 106 Display part 107 External output terminal 108 Area of interest detection circuit 109 Operation part 110 Reproduction speed setting circuit 111 Servo circuit 112 System controller

Claims (5)

Playback means for playing back image data comprising a plurality of screens;
Detection means for detecting a processing region that has been encoded by being shifted up on a bit plane from the other region in the screen included in the image data;
Decoding means for decoding the reproduced image data;
Reproduction speed selection means capable of selecting a reproduction speed in the reproduction means;
When a predetermined playback speed faster than normal playback is selected by the playback speed selection means, the processing area is shifted down on the bit plane by detecting a part of the processing area detected by the detection means. And a control means for controlling the decoding means so as to reduce the area of the image.   2. The image according to claim 1, further comprising: a predetermined area setting unit that sets addition of the processing area to the input image; and a recording unit that records the input image to which the processing area is added. Playback device.   3. The image processing apparatus according to claim 2, further comprising encoding means for encoding the input image, wherein the predetermined area setting means sets addition of the processing area in the course of encoding processing by the encoding means. Image playback device.   2. The image reproducing apparatus according to claim 1, further comprising display means for displaying an image related to the image data decoded by the decoding means. A playback step for playing back image data comprising a plurality of screens;
A detection step of detecting a processing region that has been encoded by being shifted up on a bit plane from the other region in the screen included in the image data;
A decoding step of decoding the reproduced image data;
A playback speed selection step of selecting a playback speed in the playback step;
When a predetermined playback speed higher than that during normal playback is selected in the playback speed selection step, a part of the processing area detected in the detection step is shifted down on the bit plane, whereby the processing area And a control step for controlling the decoding process in the decoding step so as to reduce the area of the image.

Images