JPH10229560A - Image processing unit and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To display simultaneously plural moving images stored in the same storage medium on a single display screen. SOLUTION: A memory 6 stores coded data FL as to plural input signals and a memory IF7 reads the coded data FL from the memory 6 corresponding to one or plural bands having a data amount corresponding to a screen division number M as to M sets of input signals among the coded data FL corresponding to the plural bands based on the screen division number M inputted from an outside under the control of a reproduction control part 9 to output the data to a decoder 8. The decoder 8 applies decoding processing based on the read coded data FL and stores decoded data DDG as to the M sets of the input signals. A video interface part 4 reads the decoded data DDG to output the data as reproduced image data SDG. Thus, multi-screen moving image reproduction is conducted easily based on the coded data FL stored in the memory 6.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to an image processing apparatus and an image processing method, and more particularly, to an image processing apparatus and an image processing method for performing image processing based on compressed image data obtained by compressing image data.

[0002]

2. Description of the Related Art Conventionally, magnetic tapes and optical disks have been used as storage media for recording image signals corresponding to moving images. In such a storage medium, a plurality of image signals corresponding to a plurality of moving images may be recorded.

[0003] When a plurality of image signals corresponding to a plurality of moving images are recorded on a storage medium, the recorded image is stored in order to confirm what kind of moving image is recorded on the storage medium. Generally, fast forward reproduction is sequentially performed from the head side of a medium, and cueing reproduction is sequentially performed based on index information (recording start position information of each image, time information, etc.) stored in a storage medium in advance. Was.

[0004]

In the above-mentioned conventional method for checking the storage state using an image storage medium, the storage medium is mechanically driven to sequentially fast-forward (forward jump) or rewind (reverse jump). Has to be performed to confirm the contents, and there is a problem that it takes time and effort for confirmation.

If a plurality of moving images of different storage media can be simultaneously displayed on the same display screen, it is suitable for search processing and the like. However, a tape-shaped recording medium such as a magnetic tape has a problem that random access is restricted due to its structure.

On the other hand, in a disk-shaped recording medium such as an optical disk, the transfer rate is low due to the presence of a drive mechanism.
For this reason, there is a problem that it is difficult to simultaneously display a plurality of moving images stored on the same storage medium. Therefore, an object of the present invention is to provide an image processing apparatus and an image processing method capable of simultaneously displaying a plurality of moving images stored in the same storage medium on one display screen.

[0007]

In order to solve the above-mentioned problems, the invention according to claim 1 is capable of being written or read electrically or optically, divides an input signal into a plurality of bands, and performs quantization. And coded data storage means for storing coded data obtained by coding for a plurality of the input signals, based on a screen division number M input from the outside, and storing the coded data corresponding to the plurality of bands. The screen division number M for the M input signals
Reading means for reading the coded data corresponding to one or a plurality of bands having a data amount corresponding to the data amount from the storage means, and performing decoding processing based on the coded data read by the reading means to perform decoding. Decoding means for outputting encoded data, decoded data storage means having a storage capacity capable of storing at least one frame of the encoded data, and storing the decoded data for the M input signals; Output means for reading out the decoded data and outputting it as reproduced image data.

According to the first aspect of the present invention, the encoded data storage means stores the encoded data for a plurality of input signals. The readout means includes, based on the number of screen divisions M input from the outside, one or a plurality of encoded data corresponding to the plurality of bands having a data amount corresponding to the number of screen divisions M for M input signals. The encoded data corresponding to the band is read out from the storage means and output to the decoding means.

The decoding means performs a decoding process based on the encoded data read by the reading means, and outputs the decoded data to the decoded data storage means. The decoded data storage means stores decoded data for the M input signals.

[0010] The output means reads the decoded data from the decoded data storage means and outputs it as reproduced image data. According to a second aspect of the present invention, in the first aspect of the present invention, when the number N of the input signals is smaller than the data division number M, an M divided storage area obtained by dividing the storage area of the decoded data storage means by M. And a dummy data writing unit that stores the decoded data in the N M divided storage areas and writes the dummy decoded data in the (M−N) M divided storage areas.

According to the second aspect of the present invention, in addition to the operation of the first aspect of the invention, the dummy data writing means is provided for decoding the decoded data when the number N of the input signals is smaller than the data division number M. The storage area of the storage means is an M-divided storage area divided into M, and the decoded data is stored in the N M-divided storage areas and the dummy decoded data is written in the (M-N) M-divided storage areas.

According to a third aspect of the present invention, in the first or second aspect of the present invention, there is provided a display unit for dividing one screen based on the reproduced image data and displaying a plurality of images. I do. According to the invention described in claim 3,
In addition to the function of the invention described in claim 1 or 2,
The display unit divides one screen based on the reproduced image data and displays a plurality of images.

[0013] According to a fourth aspect of the present invention, in the first aspect of the present invention, the encoding includes:
X-layer two-dimensional wavelet transform processing (X: an integer of 2 or more), and the reading means sets the screen division number M to 4 ^Z ≧ M> 4 ^(Z−1) (where Z is X ≧ Z ≧ In the case where the number is (an integer satisfying 0), from the first band, which is the lowest band, to the (3 · (X−
It is configured to read out the coded data constituting (3 · (X−Z) +1) bands up to (Z) +1) band.

According to the fourth aspect of the present invention, in addition to the operation of the first aspect of the present invention, the reading means may be arranged so that the screen division number M is 4 ^Z ≧ M> 4 ^{(Z -1)} (where Z is an integer that satisfies X ≧ Z ≧ 0), from the first band, which is the lowest band, to the (3 · (X−
The encoded data forming (3 · (X−Z) +1) bands up to the (Z) +1) band is read.

[0015] The invention according to claim 5 is the invention according to claims 1 to
In the invention according to any one of Items 3, the encoding includes:
(2^X× 2^X) Pixels are taken as one block (X: natural number),
(2 ^X× 2^XDiscrete generating the discrete cosine transform coefficients
A cosine transform process, wherein the storage means performs the encoding
The discrete cosine transform coefficients of one frame obtained by
Divided into (3 · X + 1) bands set and stored,
The reading means sets the screen division number M to 4^Z≧ M> 4^(Z-1) (Where Z is an integer satisfying X ≧ Z ≧ 0)
From the first band, which is the lowest band, to (3 · (X−
(3) (X−Z) +1) bands up to Z) +1) band
Is configured to read the encoded data.
You.

According to a fifth aspect of the present invention, in addition to the operation of any one of the first to third aspects, the storage means stores the discrete cosine transform coefficients of one frame obtained by encoding. Is divided into preset (3 · X + 1) bands and stored, and the reading means determines that the screen division number M is 4 ^Z ≧ M> 4 ^(Z−1) (where Z is X ≧ Z ≧ 0) , The first band, which is the lowest band, from the first band (3 · (X−
The encoded data forming (3 · (X−Z) +1) bands up to the (Z) +1) band is read.

According to a sixth aspect of the present invention, in the first aspect of the present invention, the encoded data storage means and the decoded data storage means are constituted by semiconductor memories. According to the sixth aspect of the present invention, in addition to the operation of the first aspect of the present invention, the encoded data storage means and the decoded data storage means are constituted by the semiconductor memory, and the high speed operation is realized. Can read and write arbitrary data.

According to a seventh aspect of the present invention, an input image is divided into a plurality of bands, quantized, encoded, electrically or optically written, and stored as encoded data in a readable encoded data storage medium. Based on an encoded data storage step and an externally input screen division number M, the encoded data corresponding to the plurality of bands corresponds to the data division number M for M input videos. A reading step of reading the coded data corresponding to one or a plurality of bands having a data amount from the coded data storage medium, and a decoding step of performing a decoding process based on the coded data read in the reading step And decoding the M input signals on a decoded data storage medium having a storage capacity capable of storing at least one frame of the encoded data. And decoding data storage step of storing the No. of data, configure and an output step of outputting as the reproduced image data by reading the decoded data from the decoding data storage medium.

According to the seventh aspect of the present invention, in the encoded data storage step, the encoded data is stored in the encoded data storage medium. In the reading step, based on the screen division number M input from the outside, one or a plurality of encoded data corresponding to the plurality of bands having a data amount corresponding to the data division number M for the M input videos is included. The encoded data corresponding to the band of the data is read out from the encoded data storage medium.

In the decoding step, a decoding process is performed based on the encoded data read in the reading step. In the decoded data storage step, the decoded data for the M input signals is stored in the decoded data storage medium.

In the output step, the decoded data is read from the decoded data storage medium and output as reproduced image data. According to an eighth aspect of the present invention, in the invention of the seventh aspect, when the number N of the input videos is smaller than the number M of data divisions, the storage area of the decoded data storage medium is divided into M divided storage areas. , And a dummy data writing step of storing the decoded data in the N M divided storage areas and writing the dummy decoded data in the (M−N) M divided storage areas.

According to the eighth aspect of the present invention, in addition to the operation of the seventh aspect, in the dummy data writing step, when the number N of input pictures is smaller than the number M of data divisions, the decoded data is stored. The storage area of the medium is divided into M divided M storage areas, the decoded data is stored in N M divided storage areas, and the dummy decoded data is written in (M−N) M divided storage areas.

According to a ninth aspect of the present invention, in accordance with the seventh or eighth aspect, there is provided a display step of dividing one screen based on the reproduced image data and displaying a plurality of images. I do. According to the ninth aspect, in addition to the effect of the seventh or eighth aspect, the display step divides one screen based on the reproduced image data and displays a plurality of images. .

According to a tenth aspect of the present invention, in the invention according to any one of the seventh to ninth aspects, the encoding is an X-layer two-dimensional wavelet transform process (X:
The reading step is performed when the number M of screen divisions is 4 ^Z ≧ M> 4 ^(Z−1) (where Z is an integer satisfying X ≧ Z ≧ 0). From the first band, which is the band of (3 · (X−
It is configured to read out the coded data constituting (3 · (X−Z) +1) bands up to (Z) +1) band.

According to the tenth aspect, the seventh aspect is provided.
In addition to the effects of the invention described in any one of claims 9 to 9,
In the reading step, when the screen division number M is 4 ^Z ≧ M> 4 ^(Z-1) (where Z is an integer satisfying X ≧ Z ≧ 0), the first band which is the lowest band is used. The (3. (X-
The encoded data forming (3 · (X−Z) +1) bands up to the (Z) +1) band is read.

According to an eleventh aspect of the present invention, in the invention according to any one of the seventh to ninth aspects, the encoding is performed by setting (2 ^X × 2 ^X ) pixels as one block (X: natural number) , (2 ^X × 2 ^X ) discrete cosine transform coefficients, and in the storing step, the discrete cosine transform coefficients of one frame obtained by the encoding are set in advance (3 · In the reading step, the screen division number M is 4 ^Z ≧ M> 4 ^(Z−1) (where Z is an integer satisfying X ≧ Z ≧ 0). In this case, from the first band, which is the lowest band, to (3 · (X−
It is configured to read out the coded data constituting (3 · (X−Z) +1) bands up to (Z) +1) band.

According to the eleventh aspect of the present invention, the seventh aspect is provided.
In addition to the effects of the invention described in any one of claims 9 to 9,
In the storing step, the discrete cosine transform coefficient of one frame obtained by the encoding is divided into (3 · X + 1) bands that are set in advance and stored, and in the reading step, the screen division number M is 4 ^Z ≧ M> 4 ^(Z-1) (where Z is an integer satisfying X ≧ Z ≧ 0), from the first band, which is the lowest band, to (3 · (X−
The encoded data forming (3 · (X−Z) +1) bands up to the (Z) +1) band is read.

[0028]

Preferred embodiments of the present invention will now be described with reference to the drawings. First Embodiment 1) Configuration of Video Recording / Reproducing Apparatus FIG. 1 shows a schematic block diagram of a video recording / reproducing apparatus according to a first embodiment using a wavelet transform as a subband encoding method.

The video recording / reproducing apparatus 1 can be roughly classified into a video camera 2 for capturing an object to be imaged and outputting it as an image signal SG, a display 3 for performing various displays based on the decoded image signal SDG, and a video camera 2 The image signal SG input from the ITU-R Rec. 601 is converted to image data DG having a 4: 2: 2 format and output to an encoder 5 described later, and decoded image data DDG output from a decoder 8 described later is converted to a decoded image signal SDG and displayed. 3, a video interface 4 for outputting the frame data FL through a wavelet transform of the image data input through the video interface 4, and a memory 6 for storing the frame data FL. While writing the frame data FL into the memory 6,
The memory interface unit 7 outputs the frame data FL read from the memory 6 to a decoder 8 described later, and decodes the frame data FL read from the memory 6 via the memory interface unit 7 to decode the decoded image data D
It is provided with a decoder 8 for outputting DG and a reproduction control section 9 for performing various reproduction controls based on external instructions.

In this case, the memory 6 is configured using a memory having no mechanically readable or electrically readable or mechanically driving portion. For example, a semiconductor memory such as a flash memory, an SRAM, or a DRAM is used. It is configured using. It is also possible to use a PC card or the like.

The encoder 5 receives the input image data D
Wavelet transform G to sub-band image data DSB
, A quantizer 12 that quantizes the sub-band image data DSB and outputs a plurality of quantized sub-band data DQSB, and encodes the quantized sub-band data DQSB by two-dimensional Huffman coding. A variable-length encoding unit 13 that outputs the image data DVLG,
And a formatter unit 14 that collectively outputs a plurality of pieces of encoded image data DVLG as frame data FL having a predetermined format (see FIG. 5).

The decoder 8 performs inverse formatting of the input frame data FL and outputs a plurality of inverse format variable length encoded image data DVLG ', and a plurality of inverse format variable length encoded image data DVLG. And a variable-length decoding unit 16 that two-dimensionally Huffman-decodes each of them and outputs a plurality of quantized decoded image data DQSB ', and dequantizes the plurality of quantized decoded image data and outputs them as dequantized image data DSB' And an inverse wavelet transform unit 18 that performs inverse wavelet transform on the inversely quantized image data DSB ′ and outputs decoded image data DDG.

2) Operation of Encoder Now, a more specific operation of the encoder 5 will be described. First, the general operation of the wavelet transform unit 11 (3
2 and 3 for hierarchical two-dimensional wavelet transform)
This will be described with reference to FIG.

The three-layer two-dimensional wavelet transform is shown in FIG.
As shown in FIG. 3, a process of performing one-dimensional subband division in a first direction (horizontal direction in FIG. 2) and further performing one-dimensional subband division in a second direction (vertical direction in FIG. 2) is performed. Can be realized by recursively applying to the lowest sub-band data LL1 (LL2).

In FIG. 2, reference numerals "L" and "H"
Is a quadrature mirror filter (QMF) designed based on the wavelet theory, where the symbol “L” represents a low-pass filter and the symbol “H” represents a high-pass filter. In this case, the low-pass filter L
And the impulse response of the high-pass filter H
Assuming that (n) and h (n), h (n) = (-1) ^(1-n) l (1-n).

The symbol “↓ 2” indicates サ ブ sub-sampling. Furthermore, a pair of “L ↓ 2” and “H ↓”
"2" constitutes a split filter pair. Next, a detailed operation of the wavelet transform unit 11 will be described. a) First Hierarchy The input image data DG is divided into sub-bands in the horizontal direction, and is divided into a low-frequency signal and a high-frequency signal on a first frame memory as shown in FIG.

Next, subband division is performed in the vertical direction based on the data in the first frame memory, and as shown in FIG. 3B, the subband data LL1, DSB7, DSB8, It is divided into four subband data of DSB9. b) Second layer Subsequently, the sub-band data LL1, DSB7, DSB8, DSB9
Out of the sub-band data LL1 in the lowest band is divided into sub-bands in the horizontal direction and, as shown in FIG. 3C, is divided into a low-band signal and a high-band signal on the first frame memory.

Next, subband division is performed in the vertical direction based on the subband data LL1 in the first frame memory, and as shown in FIG. 3D, the subband data LL1 corresponding to the subband data LL1 in the second frame memory. The area is divided into four sub-band data LL2, DSB4, DSB5, and DSB6. c) Third layer Similarly, subband data LL2, DSB4, DSB5, DSB6
The subband data LL2 of the lowest band is divided into subbands in the horizontal direction, and is divided into a lowband signal and a highband signal on the first frame memory as shown in FIG.

Next, subband division is performed in the vertical direction based on the data in the area corresponding to the subband data LL2 on the first frame memory, and as shown in FIG.
In the area corresponding to the sub-band data LL2 on the second frame memory, the sub-band data DSB0, DSB1, DSB2, DS
It is divided into four sub-band data of B3.

The image data DG input by performing the two-dimensional wavelet transform of the first to third layers is divided into ten sub-band data DSB0 to DSB9. These sub-band data DSB0 to DSB9 constitute sub-band image data DSB.

The sub-band image data DSB (= sub-band data DSB0 to DSB9) obtained by performing the wavelet transform on the image data DG in this manner is output to the quantization unit 12. The sub-band image data DSB is quantized by the quantization unit 12, and each sub-band data DSB0,
, DSB8, and DSB9 are output to the variable-length encoding unit 13 as a plurality of quantized sub-band data DQSB.

The variable-length coding unit 13 performs two-dimensional Huffman coding on the plurality of quantized sub-band data DQSB and converts the plurality of quantized sub-band data DQSB into a plurality of variable-length coded image data DVLG.
4 is output. The formatter unit 14 collectively outputs the plurality of input variable-length coded image data DVLG to the memory 6 via the memory interface unit 7 as frame data FL having a predetermined format.

3) Physical Format of Memory FIG. 4 shows a physical format of the memory. The memory 6
A directory for storing directory information such as a file name of an image sequence, a start sector number of a file corresponding to the image sequence, an end sector number of a file corresponding to the image sequence, a file size of the file, and a recording time. Area 6
A and program area 6 for storing frame data FL
B, by referring to the directory area 6A, the number of image sequences recorded in the memory 6, the start position of each image sequence, the recording time, and the like can be found.

More specifically, the memory 6 is composed of N sectors (for example, each sector is composed of 2048 bytes), and the directory area 6A is the 0th sector (in the figure,
SC0 is assigned, and the program area 6B is assigned to the first sector (sector 1 in the figure) SC1 to SC1.
N sectors of the Nth sector (denoted as sector N in the figure) are allocated.

Further, the actual coded image data DEG is stored in the program area 6B in units of frame data corresponding to the frames, and each of the frame data FL1 to FLL facilitates detection of the head of each of the frame data FL1,..., FLL. Therefore, data is always written from the beginning of each of the sectors SC1 to SCN, and zero data is written as dummy data in an area where data of the last sector corresponding to the frame data does not exist.

More specifically, the first frame data FL
1, data is recorded from the beginning of the first sector SC1,
The data is recorded up to the middle of the n-th sector SCn, and zero data is written as dummy data in the remaining portion of the n-th sector. And the next (n + 1) th sector SC
The second frame data is written from the beginning of (n + 1).

4) Configuration of Frame Data FIG. 5 shows a data configuration diagram of the frame data FL. The frame data FL is roughly divided into an index information section 3
0 and an image data section 31.

The index information section 30 has SOF (Start Of Frame) data 32 representing the head of frame data.
, Frame number data 33 representing a frame number (Frame No.), frame byte count data 34 representing the total number of bytes of the frame, and subband SB0 byte count representing the number of bytes of the subband SB0 in the frame. Data 35, subband SB3 byte count data 36 representing the total number of bytes of four subband data from subband SB0 to subband SB3 in the frame, and 7 from subband SB0 to subband SB6 in the frame. And sub-band SB6 byte count data 37 representing the total number of bytes of one sub-band data.

The image data section 31 includes luminance signal (Y) component data, RY color difference signal component data, and BY color difference signal component data for each sub-band.
More specifically, luminance signal (Y) component data SB0Y corresponding to the luminance signal component of subband SB0, RY color difference signal component data SB0R corresponding to the RY color difference signal component of subband SB0, subband SB0 , The BY color difference signal component data SB0B corresponding to the BY color difference signal component, the luminance signal (Y) component data SB1Y of the sub-band SB1,.
..., luminance signal (Y) component data SB9Y corresponding to the luminance signal component of sub-band SB9, RY of sub-band SB9
RY color difference signal component data SB corresponding to the color difference signal component
9R and BY color difference signal component data SB9B corresponding to the BY color difference signal components of the sub-band SB9.

Here, the luminance signal (Y) component data SB1 of the sub-band SB1 is used as the luminance signal (Y) component data.
A description will be given using Y as an example. Luminance signal (Y) component data SB
1Y is S representing the start of the sub-band SB1.
OS (Start Of Subband) data 40, Q value data 41 representing the quantization step width (Q Value) of the subband SB1, subband byte count data 42 representing the number of bytes of the subband SB1, and 2 And Huffman encoded data 43 which is sub-band data subjected to dimensional Huffman encoding.

5) Operation of Decoder Here, a specific operation of the decoder 8 will be described.
First, when a reproduction command is input to the reproduction control unit 9, the memory interface unit 7 reads out the frame data FL corresponding to the reproduction command from the memory 6 and sequentially outputs the frame data FL to the inverse formatter unit 15 of the decoder 8.

The inverse formatter 15 generates a plurality of coded image data DVLG ′ corresponding to each subband from the frame data FL and outputs the coded image data DVLG ′ to the variable length decoder 16.
The variable length decoding unit 16 calculates the encoded image data DVLG '
Performs dimensional Huffman decoding to obtain each subband data D
A plurality of quantized sub-band data DQSB 'corresponding to SB0, DSB1,..., DSB8, and DSB9 are output to the inverse quantization unit.

The inverse quantization unit 17 performs inverse quantization of the quantized sub-band data DQSB ', and outputs the inverse-quantized image data DS
B ′ is output to the inverse wavelet transform unit 18. The inverse wavelet transform unit 18 performs inverse wavelet transform on the input inverse quantized image data DSB 'and outputs the result to the video interface unit 4 as decoded image data DDG.

6) Operation of Wavelet Inverse Transformation Unit Next, the schematic operation of the wavelet inverse transformation unit 18 (three layers 2
The two-dimensional wavelet inverse transform will be described with reference to FIGS. In the three-layer two-dimensional inverse wavelet transform, as shown in FIG. 6, one-dimensional subband synthesis is performed in a first direction (vertical direction in FIG. 6) and further in a second direction (horizontal direction in FIG. 6). This can be realized by performing a process of performing one-dimensional subband synthesis, and sequentially resynthesizing the two synthesis results.

In FIG. 6, reference numerals "L" and "H"
Is a quadrature mirror filter (QMF) designed based on the wavelet theory, where the symbol “L” represents a low-pass filter and the symbol “H” represents a high-pass filter. Also in this case, assuming that the impulse responses of the low-pass filter L and the high-pass filter H are l (n) and h (n), respectively, h (n) = (− 1) ^(1-n) l (1 ⁾ -N).

The symbol “$ 2” indicates double upsampling. Further, a pair of “$ 2L” and “$ 2
"H" constitutes a synthesis filter pair. Next, the detailed operation of the inverse wavelet transform unit 18 will be described. a) The third-layer sub-band data DSB0 'and the sub-band data DSB1' are converted to the second layer by the third-layer first vertical synthesis filter pair.
The low frequency signal is synthesized on the frame memory FM2.

On the other hand, the sub-band data DSB2 'and the sub-band data DSB3' are synthesized into a high-frequency signal on the second frame memory FM2 by the third hierarchical second vertical synthesizing filter pair. A low-frequency signal as a result of the vertical synthesis of the sub-band data DSB0 'and the sub-band data DSB1', and the sub-band data DSB2 'and the sub-band data DSB
The high-frequency signal, which is the result of vertical synthesis of 3 ', is synthesized as sub-band data LL2' by a third layer horizontal synthesis filter pair in a corresponding area on the first frame memory FM1.

B) The second-layer sub-band data LL2 'and sub-band data DSB4' are converted to the second-layer first vertical synthesis filter pair by the second
The low frequency signal is synthesized on the frame memory FM2.

On the other hand, the sub-band data DSB 5 ′ and the sub-band data DSB 6 ′ are synthesized into a high-frequency signal on the second frame memory FM 2 by the second hierarchical second vertical-direction synthesis filter pair. A low-frequency signal as a result of vertical synthesis of the sub-band data LL2 and the sub-band data DSB4 ', and the sub-band data DSB5' and the sub-band data DS
The high-frequency signal, which is the result of the vertical synthesis of B6 ', is synthesized as sub-band data LL1' by a second layer horizontal synthesis filter pair in a corresponding area on the first frame memory FM1.

C) First-layer sub-band data LL1 'and sub-band data DSB7' are converted to second-layer data by the first-layer first vertical-direction synthesis filter pair.
The low frequency signal is synthesized on the frame memory FM2.

On the other hand, the sub-band data DSB 8 ′ and the sub-band data DSB 9 ′ are synthesized into a high-frequency signal on the second frame memory FM 2 by the first hierarchy second vertical synthesis filter pair. The low band signal which is the vertical synthesis result of the subband data LL1 'and the subband data DSB7' and the high band signal which is the vertical synthesis result of the subband data DSB8 'and the subband data DSB9' are the first layer horizontal synthesis filter The pair is combined as decoded image data DDG on the first frame memory FM1.

The decoded image data DDG is subjected to D / A conversion via the video interface unit 4 to obtain the image signal SDG.
Is output to the display 3. As a result, an image is displayed on the screen of the display 3. 7) Multi-screen moving image reproduction processing Next, a description will be given of a plurality of moving image reproduction based on a plurality of moving image data stored in different areas of the memory 6 on one screen, that is, a so-called multi-screen moving image reproduction processing.

FIG. 8 shows the relationship between the data storage state of the memory 6 and the physical format in the multi-screen moving image reproduction processing (in the figure, at the time of 4-screen moving image reproduction). As in the case of normal playback, the directory area 6A stores the file name of the image sequence, the start sector number of the file corresponding to the image sequence, the end sector number of the file corresponding to the image sequence, the file size of the file, Directory information such as time is stored.

In the program area 6B, four files corresponding to four image sequences are stored. The first image sequence is stored in the range from sector 1 to sector K, the second image sequence is stored in the range from sector (K + 1) to sector L,
The image sequence is stored in a range from sector (L + 1) to sector M, and the fourth image sequence is stored in a range from sector (M + 1) to sector N.

In this case, sectors K, L, M, N
In, zero data is stored in a portion where no data is stored. In the multi-screen moving image reproduction, the sub-band synthesis is discontinued in the wavelet inverse transform shown in FIG. 6 to reduce the number of pixels (1/4, 1/16, 1/64 of the total number of pixels of the original screen). ) Can be realized by directly displaying the reproduced image on the screen of the display 3.

The number of pixels of the reproduced image obtained by the number of sub-band data used for sub-band synthesis is as follows. When using the sub-band data DSB0 to DSB9: the total number of pixels When using the sub-band data DSB0 to DSB6: 1/4 of the total number of pixels When using the sub-band data DSB0 to DSB3: 1/16 of the total number of pixels When only DSB0 is used: 1/64 of the total number of pixels.
If display is performed by combining all three layers of the second layer and the first layer, all-pixel display can be performed.

If the display is performed by synthesizing only the third and second hierarchies, it is possible to display 1/4 of the number of pixels, and the display screen capable of displaying all pixels has four screens. Simultaneous display (4-split display) can be performed. Further, if the display is performed by combining only the third layer, a display of 1/16 of the total number of pixels can be performed, and a 16-screen simultaneous display (16-split display) can be performed on a display screen capable of displaying all pixels. I can do it.

Furthermore, if the display is performed using only the sub-band data DSB0 without performing the synthesis, 1/6 of all the pixels can be obtained.
4 pixels can be displayed, and 64 screens can be simultaneously displayed (64-split display) on a display screen capable of displaying all pixels. 8) Multi-screen video playback process Next, the multi-screen video playback process will be described.

FIG. 9 is a flowchart of a multi-screen moving image reproduction process. The reproduction control unit 9 determines whether or not the file name of the video sequence to be reproduced by the user has been input (step S1). If the file name of the video sequence has not been input (step S1; No),
The reproduction processing ends, and the apparatus enters a standby state.

If it is determined in step S1 that the file name of the video sequence (image to be reproduced) is input (step S1; Yes), the reproduction control unit 9
Acquires the video sequence number N based on the input file name (step S2), and shifts the processing to the multi-screen reproduction display processing of step S3.

FIG. 10 is a flowchart showing a multi-screen moving image reproduction / display process. The reproduction control unit 9 determines whether or not the number N of video sequences is equal to or greater than the maximum number of screen divisions = 64 (screen) (step S21). In the determination in step S21, the number of video sequences N is equal to the maximum number of screen divisions = 6.
If it is less than 4 (screen) (step S21; N
o), the number of screen divisions M is determined according to N (step S2)
2) While notifying the memory interface unit 7 and the decoder 8, the process proceeds to step S24.

More specifically, in order to prevent the processing time required for performing the pixel number conversion processing and the like and to simplify the processing, the preset screen division number is set to 1 (screen), 4 (screen), 16 (screen). (Screens) and 64 (screens), and the number M of screen divisions is set to the minimum number of screens exceeding the number N of video sequences.

For example, when the video sequence number N = 3, the screen division number M = 4 (screen) and the video sequence number N
= 32, the number of screen divisions M is set to 64.
If it is determined in step S21 that the video sequence number N is equal to or greater than the maximum screen division number 64 (step S2).
1; Yes), the number M of screen divisions is set to M = 64, the memory interface unit 7 and the decoder 8 are notified, and the number N of video sequences is set to N = 64 (step S23).

Next, the display counter X is set to X = 1 (step S24). Subsequently, subband data necessary for the X-th image sequence is extracted according to the number of screen divisions (step S25).

More specifically, when the number of screen divisions M is notified from the reproduction control unit 9, the memory interface unit 7 refers to the start sector number of the image sequence from the directory area 6A on the memory 6 and outputs the frame data FL. (Or a part of the frame data FL).

More specifically, SOF (Start Of Frame) data 32 representing the start of frame data, frame number data 33 representing a frame number (Frame No.), and frame byte number representing the total byte number of the frame Referring to any one of the count data 34 and the sub-band SB0 byte count data 35, the sub-band SB3 byte count data 36, or the sub-band SB6 byte count data 37, the data corresponding to the screen division number M is referred to. Read the necessary sub-band data.

For example, when the screen division number M = 4, the memory interface unit 7 reads and refers to the sub-band SB6 byte count data 37 corresponding to the screen division number M = 4, and refers to the corresponding frame data FL. From the sub-band SB0 to SBSB6, and sequentially outputs the data to the inverse formatter unit 15 of the decoder 8.

Next, the decoder 8 comprises an inverse formatter 15
, Two-dimensional Huffman decoding by the variable length decoding unit 16, inverse quantization by the inverse quantization unit 17, and inverse wavelet transform by the inverse wavelet transform unit 18 (step S26).

More specifically, the inverse formatter unit 15
A plurality of encoded image data DVLG 'corresponding to subbands SB0 to SBSB6 are generated and output to variable length decoding section 16. The variable length decoding unit 16 performs two-dimensional Huffman decoding of the encoded image data DVLG ′, and outputs a plurality of quantized decoded image data DQSB ′ to the inverse quantization unit 17.

The inverse quantization unit 17 inversely quantizes the plurality of quantized decoded image data and decodes the decoded subband image data DSB '.
Is output to the inverse wavelet transform unit 18. The inverse wavelet transform unit 18 performs an inverse wavelet transform on the decoded subband image data DSB 'corresponding to the seven subband data from the subband SB0 to the subband SB6, and writes the decoded image data DDG into the frame memory (step S27). .

9) Inverse Wavelet Transform Operation During Multi-Screen Moving Image Reproduction Processing Here, the inverse wavelet transformation operation during multi-screen moving image reproduction processing will be described with reference to FIG. As shown in FIG. 11, the three-layer two-dimensional wavelet inverse transform at the time of multi-screen reproduction that can display from the screen division number M = 1 (no division) to the screen division number M = 64 is performed in the first direction (in FIG. One-dimensional subband synthesis is performed in the vertical direction (vertical direction), one-dimensional subband synthesis is performed in the second direction (horizontal direction in FIG. 11), and two more synthesis results are sequentially re-synthesized. The processing can be realized by performing a predetermined number of layers according to the number of screen divisions.

In FIG. 11, the symbols “L” and “H” are quadrature mirror filters (QMF) designed based on the wavelet theory, and the symbol “L” is low-pass as in FIG. The symbol “H” represents a high-pass filter.

Also in this case, the low-pass filter L
And the impulse response of the high-pass filter H
Assuming that (n) and h (n), h (n) = (-1) ^(1-n) l (1-n).

The symbol “$ 2” indicates double upsampling. Further, a pair of “$ 2L” and “$ 2
"H" constitutes a synthesis filter pair. 10) Detailed Operation of Wavelet Inverse Transformation Unit at the Time of Playing Multi-Screen Moving Image Next, the detailed operation of the wavelet inverse transform unit 18 at the time of playing a multi-screen moving image will be described with reference to FIGS. Here, FIG. 12 shows a case where the screen division number M = 4.

The decoded sub-band image data DS read out from the memory by the number of sub-bands corresponding to the screen division number M and subjected to inverse formatting, variable length decoding and inverse quantization.
B ′ is written in the area ARX (= AR1) corresponding to the image sequence number X = 1 on the first frame memory FM1. a) When the number M of screen divisions is 64, the inverse wavelet transform unit 18 does not perform any processing, and the decoded image data D
DG is the sub-band data DSB0 'itself written at the corresponding position on the first frame memory FM1 (step S27). When the inverse wavelet transform is terminated at this stage, the decoded image data DDG corresponds to an image having 1/64 of the total number of pixels. b) Third layer (processing is required when the number of screen divisions M = 1, 4, and 16) The sub-band data DSB0 'and the sub-band data DSB1' are converted to the second layer by the third vertical first synthesis filter pair in the third layer.
The low frequency signal is synthesized on the frame memory FM2.

On the other hand, the sub-band data DSB2 'and the sub-band data DSB3' are synthesized into a high-frequency signal on the second frame memory FM2 by the third hierarchical second vertical synthesizing filter pair. A low-frequency signal as a result of the vertical synthesis of the sub-band data DSB0 'and the sub-band data DSB1', and the sub-band data DSB2 'and the sub-band data DSB
The high-frequency signal, which is the result of vertical synthesis of 3 ', is synthesized and written as sub-band data LL2' in the corresponding area on the first frame memory FM1 by the third hierarchical horizontal synthesis filter pair (step S27).

When the inverse wavelet transform is terminated at this stage, the decoded image data DDG corresponds to an image having 1/16 of the total number of pixels. c) The second hierarchy (processing is required when the number of screen divisions M = 1, 4) After the processing of the third hierarchy, the sub-band data LL2 'and the sub-band data DSB4' are combined in the second hierarchy in the first vertical direction. The low-frequency signal is synthesized on the second frame memory FM2 by the filter pair.

On the other hand, the sub-band data DSB5 'and the sub-band data DSB6' are synthesized into a high-frequency signal on the second frame memory FM2 by the second hierarchical second vertical synthesis filter pair. The low band signal as a result of the vertical synthesis of the subband data LL2 'and the subband data DSB4' and the high band signal as the result of the vertical synthesis of the subband data DSB5 'and the subband data DSB6' are the second-layer horizontal synthesis filter. Subband data LL1 'by pair
Is synthesized and written in the corresponding area on the first frame memory FM1 (step S2).
7).

When the inverse wavelet transform is terminated at this stage, the decoded image data DDG corresponds to an image whose number of pixels is 1/4 of the total number of pixels. d) The first layer (processing is required when the number of screen divisions M = 1) After the processing of the second layer, the sub-band data LL1 'and the sub-band data DSB7' are combined with the first-layer first vertical synthesis filter pair. Is combined with the low-frequency signal on the second frame memory FM2.

On the other hand, the sub-band data DSB 8 ′ and the sub-band data DSB 9 ′ are synthesized into a high-frequency signal on the second frame memory FM 2 by the first hierarchy second vertical synthesis filter pair. The low band signal which is the vertical synthesis result of the subband data LL1 'and the subband data DSB7' and the high band signal which is the vertical synthesis result of the subband data DSB8 'and the subband data DSB9' are the first layer horizontal synthesis filter The decoded image data DDG is combined and written on the first frame memory FM1 by the pair (step S27).

At this stage, the decoded image data DDG
Is equivalent to an image in which the number of pixels is 1/1 of the total number of pixels. Next, it is determined whether the value of the display counter X is equal to the video sequence number N (step S28).

If it is determined in step S28 that X <N (step S28; No), the display counter X
Is counted up, that is, X = X + 1 is set (step S30), and the process returns to step S25 to repeat the processes of steps S25 to S27.

If it is determined in step S28 that X = N (step S28; Yes), "0" is set in all the P [number] and P = M-N [number] areas where no image is displayed. Write data (zero data = black display).

Subsequently, the decoded image data DDG on the first frame memory FM1 is output to the video interface unit 4. As a result, the video interface unit 4 performs D / A conversion of the input decoded image data DDG and outputs the image signal SDG.
Is output to the display 3, and an image having the number of pixels corresponding to the screen division number M is displayed on the screen of the display 3.

For example, when a four-screen moving image is reproduced, four images (moving images) are simultaneously displayed on the display screen of the display 3 as shown in FIG. Next, the reproduction control unit 9 proceeds to step S4.
Then, it is determined whether or not the display is completed based on the information of the video sequence to be displayed (step S).
4).

If it is determined in step S4 that the multi-screen moving image display processing has not been completed (step S4; N
o), the processing is repeated at step S3 (= steps S21 to S3)
The process shifts to 30), and the same processing is repeated to continue the multi-screen moving image reproduction processing. If it is determined in step S4 that the multi-screen moving image display process has ended (step S4; Yes), the process ends.

As described above, according to the first embodiment, a multi-screen moving image can be reproduced in real time. In the above-described first embodiment, the case where the three-layer two-dimensional wavelet transform process is performed has been described. However, the present invention is also applicable to the case where the X-layer two-dimensional wavelet transform process (X: an integer of 2 or more) is performed. It is.

That is, when the screen division number M is 4 ^Z ≧ M> 4 ^(Z−1) (where Z is an integer satisfying X ≧ Z ≧ 0), 0 subband (first
Band (equivalent to the band) to the (3 · (X−Z)) sub-band (the third
(3 · (X−Z)) up to (X−Z) +1) band
+1) subband data DSB constituting the subbands
0 to DSB (3 · (XZ)) may be read from the memory 6 and processed. Second Embodiment In the first embodiment described above, the wavelet transform is used as the encoding method.
PEG (Joint Photographic Experts Group) and MPE
This is an embodiment in which DCT (discrete cosine transform) used in G (Moving Picture Experts Group) is used.

In the case of using DCT, the wavelet transform unit 11 of FIG. 1 is arranged with a two-dimensional DCT unit 51 for performing a two-dimensional discrete cosine transform and a DCT coefficient obtained by the discrete cosine transform as shown in FIG. The DCT coefficient rearranging unit 53 for performing the reverse rearrangement of the DCT coefficients shown in FIG. 14B is replaced by the DCT coefficient rearranging unit 53 and the inversely rearranged D shown in FIG.
Two-dimensional inverse DCT (2
(Dimensional IDCT) unit 54.

If the block to be transformed in the two-dimensional DCT has an 8 [pixel] × 8 [pixel] configuration, 64 DCT coefficients K0 to K63 are obtained as shown in FIG. First
The DCT coefficient K0 represents the DC component, and is shown in FIG.
The DCT coefficients on the middle and right sides represent high frequency components in the horizontal direction, and the DCT coefficients on the lower side represent high frequency components in the vertical direction in FIG.

Therefore, as shown in FIG. 15B, the DCT coefficient rearranging section 52 shown in FIG. 14A converts 64 DCT coefficients K0 to K63 into 10 DCT coefficient groups (bands) G0 to G9. And the DCT coefficient groups for one frame are grouped for each band, and mapped (rearranged) as shown in FIG. 16, the subband division using the wavelet transform as in the first embodiment described above. It is possible to perform band division similar to the above.

As a result, from the quantization operation of the quantization unit 12 in the encoder 5 of the first embodiment to the recording operation to the memory 6 and the reading operation from the memory 6 to the inverse quantization operation of the inverse quantization unit 17 in the decoder 8 Until this, the same processing as in the first embodiment can be performed.

More specifically, as shown in FIG. 16, the first DCT coefficient group G0 of each frame is mapped to the first frame DCT coefficient group (band) FG0 based on the original arrangement of the blocks to be transformed. 2DCT coefficient group G1 is mapped to second frame DCT coefficient group FG1 based on the original arrangement of the blocks to be transformed,
.., The ninth DCT coefficient group G8 is mapped to the ninth frame DCT coefficient group FG8 based on the original arrangement of the transform target blocks, and the tenth DCT coefficient group G9 is mapped to the tenth frame DCT based on the original arrangement of the transform target blocks. Mapping to coefficient group FG9.

As a result, the same band division can be performed as in the case where the wavelet transform is used. Here, the number of divided screens = 4
A description will be given of a reproduction operation in the case of performing four-screen moving image reproduction of a first video sequence to a fourth video sequence on one screen of a display.

First frame DC of first video sequence
The T coefficient group FG0 to the seventh frame DCT coefficient group FG6 are read from the memory and subjected to variable length decoding and inverse quantization. The inversely quantized DCT coefficient is DC
In the T coefficient reverse rearrangement unit 53, the DCT coefficient reverse rearrangement is performed by the reverse operation of the mapping in FIG.
As shown in (a), the block is reconstructed into a block composed of 8 × 8 = 64 [DCT coefficients K0 to K63].

In this case, the eighth DCT group G
Since the DCT coefficients corresponding to the seventh to tenth DCT groups G9 have not been generated, all the DCT coefficients are set to “0”.
Insert The two-dimensional IDCT unit performs a two-dimensional IDCT on the obtained block composed of 64 DCT coefficients.
CT is performed, サ ブ sub-sampling is performed in the vertical direction and the horizontal direction, and as shown in FIG.
× 4 [pixels] (the original number of pixels (= 64 [pixels]; FIG. 17)
(Refer to (a))), a block composed of (1/4 of the number of pixels) is formed, and the frame memory is written in the first area of the first to fourth areas divided into four. In this case, D
Since the CT coefficient is band-limited to a low band, it is not necessary to perform band limitation by a filter before subsampling.

The above processing is performed in the first video sequence in the first video sequence.
Frame DCT coefficient group FG0 to seventh frame DC
By performing the processing on all DCT coefficients constituting the T coefficient group FG6, a decoded image having a screen size of 1/4 is displayed in the first area on the frame memory.

Similarly, the second video sequence to the fourth video sequence
The same processing is performed for the video sequence, and by writing to the corresponding area of the frame memory, the first
As a result, a decoded image in a state where the fourth video sequence can be displayed in a four-divided manner on one screen is obtained.

[0109] By performing D / A conversion of the decoded image obtained in this way by the video interface unit, a four-divided moving image is reproduced on the display screen. Next, multi-screen moving image reproduction will be described. FIG. 18 shows a flowchart of the multi-screen moving image reproduction process. In the following description, the device configuration will be described with reference to FIG.

The reproduction control section 9 obtains the number N of video sequences based on the number of file names that the user intends to reproduce (step S2). More specifically, as shown in FIG. 18, it is determined whether or not the video sequence number N is equal to or greater than the maximum screen division number = 64 (screen) (step S42).

If it is determined in step S42 that the number N of video sequences is less than the maximum number of screen divisions = 64 (screen) (step S42; No), the number M of screen divisions is determined according to N (step S44). ), And notifies the memory interface unit 7 and the decoder 8 and shifts the processing to step S45.

More specifically, in order to prevent the processing time required for performing the pixel number conversion processing and the like and to simplify the processing, the preset screen division number is set to 1 (screen), 4 (screen), 16 (screen). (Screens) and 64 (screens), and the number M of screen divisions is set to the minimum number of screens exceeding the number N of video sequences.

For example, when the video sequence number N = 3, the screen division number M = 4 (screen) and the video sequence number N
= 32, the number of screen divisions M is set to 64.
If it is determined in step S42 that the number N of video sequences is equal to or greater than the maximum number of screen divisions 64 (step S4).
2; Yes), the screen division number M is set to M = 64, the memory interface unit 7 and the decoder 8 are notified, and the video sequence number N is set to N = 64 (step S43).

Next, the display counter X is set to X = 1 (step S45). Then, the DCT required for the X-th image sequence according to the screen division number M
A coefficient is extracted (step S46).

More specifically, when the number of screen divisions M is notified from the reproduction control unit 9, the memory interface unit 7 refers to the start sector number of the image sequence from the directory area 6A on the memory 6 to refer to the frame data FL. (Or a part of the frame data FL).

More specifically, similarly to the first embodiment,
SOF (Start Of Frame) indicating the start of frame data
Data, frame number data indicating a frame number (Frame No.), frame byte count data indicating the total number of bytes of the frame, first frame DCT coefficient group FG0 byte count data, fourth frame DCT coefficient group FG3 bytes Number count data (F
Any one of data corresponding to the screen division number M from the total number of bytes of G0 to FG3) or the seventh frame DCT coefficient group FG6 byte count data (representing the total number of bytes of FG0 to FG6) See
Read the necessary frame DCT coefficient group.

For example, when the screen division number M = 4, the memory interface unit 7 reads and refers to the seventh frame DCT coefficient group FG6 byte number count data corresponding to the screen division number M = 4, and responds. From the frame data FL, seven frame DCT coefficient groups from the first frame DCT coefficient group FG0 to the seventh frame DCT coefficient group FG6 are read, and sequentially output to the inverse formatter unit 15 of the decoder 8.

Next, the decoder 8 comprises an inverse formatter 15
, The two-dimensional Huffman decoding by the variable length decoding unit 16, the inverse quantization by the inverse quantization unit 17, the inverse DCT coefficient rearrangement by the DCT coefficient inverse rearrangement unit 53, and the inverse DCT by the two-dimensional IDCT ( Step S47).

More specifically, the inverse formatter unit 15
A plurality of encoded image data DVLG ′ corresponding to the first frame DCT coefficient group FG0 to the seventh frame DCT coefficient group FG6 are generated and output to the variable length decoding unit 16. The variable length decoding unit 16 outputs the encoded image data DVLG '
, And outputs a plurality of quantized decoded image data DQSB ′ to the inverse quantization unit 17.

The inverse quantization unit 17 inversely quantizes the plurality of quantized decoded image data and decodes the decoded subband image data DSB '.
And outputs it to the DCT coefficient reverse rearrangement unit 53. DCT
The coefficient reverse rearrangement unit 53 performs reverse rearrangement of the frame DCT coefficients corresponding to the seven frame DCT coefficient groups from the first frame DCT coefficient group FG0 to the seventh frame DCT coefficient group FG6, and Output.

The two-dimensional IDCT unit 54 receives the input DC
The inverse DCT of the T coefficient is performed, and after subsampling, the decoded image data DDG is output to the video interface unit 4. Here, the inverse DCT operation at the time of reproducing a multi-screen moving image will be described. a) When the number of screen divisions M = 64, two-dimensional IDCT
The unit 54 performs the first reordering of the DCT coefficients.
In addition to the DCT coefficient group G0, 0 was inserted from the second DCT coefficient group G1 to the tenth DCT coefficient group G9 to construct an 8 × 8 DCT coefficient block, and two-dimensional inverse DCT was performed on this block. Thereafter, 1/8 sub-sampling is performed in each of the horizontal and vertical directions, and the data is written to the corresponding position on the first frame memory FM1 (step S48).

When the above processing is performed on the DCT coefficients for one frame of the X (= 1) -th image sequence, an area ARX (= AR1) corresponding to the image sequence number X on the first frame memory FM1 is obtained. The decoded image data DDG whose number of pixels is 1/64 of the total number of pixels is obtained.

B) When the screen division number M = 16, 2
The dimension IDCT unit 54 includes a fifth DCT coefficient group G4 in addition to the first DCT coefficient group G0 to the fourth DCT coefficient group G3 obtained by the reverse permutation of the DCT coefficients.
From the 10th DCT coefficient group G9 to 0
A DCT coefficient block of x8 is constructed, two-dimensional inverse DCT is performed on this block, and then 1/4 sub-sampling is performed in each of the horizontal and vertical directions to obtain a corresponding position on the first frame memory FM1. (Step S48).

When the above processing is performed on the DCT coefficients for one frame of the X (= 1) -th image sequence, an area ARX (= AR1) corresponding to the image sequence number X on the first frame memory FM1 is obtained. The decoded image data DDG whose number of pixels is 1/16 of the total number of pixels is obtained.

C) If the number of screen divisions is M = 4, the two-dimensional IDCT unit 54 adds the eighth DCT to the first DCT coefficient group G0 to the seventh DCT coefficient group G6 obtained by the reverse permutation of the DCT coefficients. By inserting 0 from the coefficient group G7 to the tenth DCT coefficient group G9, 8 ×
8 DCT coefficient blocks, two-dimensional inverse DCT is performed on this block, and then 1/2 of the sub-sampling is performed in the horizontal and vertical directions. Write (Step S
48).

When the above processing is performed on the DCT coefficients for one frame of the X (= 1) -th image sequence, an area ARX (= AR1) corresponding to the image sequence number X on the first frame memory FM1 is obtained. The decoded image data DDG whose number of pixels is 1/4 of the total number of pixels is obtained.

D) Further, when the number of screen divisions M = 1, the two-dimensional IDCT unit 54 converts the first DCT coefficient group G0 obtained by the DCT coefficient reverse rearrangement into the
The two-dimensional inverse DCT is performed on the 8 × 8 DCT coefficient block constructed from the CT coefficient group G9, and written to the corresponding position on the first frame memory FM1 (step S48).

When the above processing is performed on the DCT coefficients for one frame, decoded image data DDG whose number of pixels is 1/1 of the total number of pixels is obtained on the first frame memory FM1.
Next, it is determined whether or not the value of the display counter X is equal to the video sequence number N (step S49).

If it is determined in step S49 that X <N (step S49; No), the display counter X is counted up, that is, X = X + 1 (step S51), and the process returns to step S46. Steps S46 to S48 are repeated.

If it is determined in step S49 that X = N (step S49; Yes), “0” is set in all the P [number] and P = M−N [number] areas where no image is displayed. Data (zero data = black display) is written (step S50). Subsequently, it outputs the decoded image data DDG on the first frame memory FM1 to the video interface unit 4.

As a result, the video interface unit 4
Converts the input decoded image data DDG into a digital signal and outputs it to the display 3 as an image signal SDG. On the screen of the display 3, an image having the number of pixels corresponding to the screen division number M is displayed. It will be.

Next, the reproduction control section 9 shifts the processing to step S4, and determines whether or not the display is completed based on the information of the video sequence to be displayed (step S4). If the multi-screen moving image display process has not been completed in the determination of step S4 (step S4; No),
The process is repeated at step S3 (= steps S42 to S51).
Then, the same process is repeated to continue the multi-screen moving image reproduction process.

If it is determined in step S4 that the multi-screen moving image display processing has been completed (step S4; Ye
s), end the process. As described above, also according to the second embodiment, it is possible to easily reproduce a multi-screen moving image in real time.

In the above second embodiment, 8 × 8
Although the DCT transform coefficient is generated by using pixels as blocks, (2 ^X × 2 ^X ) pixels are taken as one block (X: natural number), and (2 ^X × 2 ^X ) discrete cosine transform coefficients are used. The present invention can also be applied to a case in which a discrete cosine transform process for generating is performed.

That is, the obtained discrete cosine transform coefficient of one frame is divided and stored in a preset (3 × X + 1) band, and the screen division number M is 4 ^Z ≧ M> 4 ^(Z−1). (Where Z is an integer satisfying X ≧ Z ≧ 0), the first DC corresponding to the first band which is the lowest band.
(3 · (X−Z) +1) DCT coefficient groups from the T coefficient group to the (3 · (X−Z) +1) DCT coefficient group corresponding to the (3 · (X−Z) +1) band May be read from the memory 6 and processed.

In each of the above embodiments, reproduction is performed at a normal reproduction speed (1 × speed).
…… Fast forward playback (or rewind playback)
The present invention can also be applied to the case where

[0137]

According to the first aspect of the present invention, the coded data storage means stores coded data for a plurality of input signals, and the read means stores the coded data based on the externally input screen division number M. Out of the encoded data corresponding to the plurality of bands, the encoded data corresponding to one or a plurality of bands having a data amount corresponding to the screen division number M for the M input signals is read out from the storage means and decoded. Means for performing decoding on the basis of the encoded data read by the reading means, and outputs the decoded data to the decoded data storage means. Since the decoded data is read out from the decoded data storage means and output as reproduced image data, the output means reads out the decoded data from the decoded data storage means. It can be carried out easily the multi-screen video playback on the basis of the data.

According to the second aspect of the present invention, in addition to the effect of the first aspect of the invention, when the number N of input signals is smaller than the number M of data divisions, the dummy data Since the storage area of the storage means is an M-divided storage area obtained by dividing M, the decoded data is stored in the N M-divided storage areas, and the dummy decoded data is written in the (M-N) M-divided storage areas. Even when the number of input signals is small and the moving image to be displayed does not cover the entire screen, multi-screen display can be easily performed without changing the processing.

According to the invention set forth in claim 3, according to claim 1
Alternatively, in addition to the effect of the second aspect, the display means divides one screen based on the reproduced image data and displays a plurality of images. According to the invention described in claim 4, in addition to the effect of the invention described in any one of claims 1 to 3, in addition to the reading means, the readout means may be arranged so that the screen division number M is ^4Z ≧ M> 4 ^(Z-1) (Where Z is an integer that satisfies X ≧ Z ≧ 0), the first band, which is the lowest band, is shifted to (3 · (X−
(3) (X−Z) +1) The coded data constituting the (3) (X−Z) +1) bands is read out, so that the amount of data to be read out is easily set to be substantially constant according to the number of divided screens. Even if the number of divided screens increases, the processing time required for display can be made substantially constant, and multi-screen moving image reproduction can be performed regardless of the number of divided screens.

According to the fifth aspect of the present invention, in addition to the effects of the first to third aspects, the storage means stores the discrete cosine transform coefficients of one frame obtained by encoding. Is divided into preset (3 · X + 1) bands and stored, and the reading means determines that the screen division number M is 4 ^Z ≧ M> 4 ^(Z−1) (where Z is X ≧ Z ≧ 0) , The first band, which is the lowest band, from the first band (3 · (X−
Since (3) (X−Z) +1) bands of encoded data up to (Z) +1) band are read, even if discrete cosine transform is used as an image compression method, the data should be read in accordance with the number of divided screens. You can easily set the data amount to be almost constant, and even if the number of divided screens increases,
The processing time required for display can be substantially constant, and multi-screen moving image reproduction can be performed regardless of the number of divided screens.

According to the sixth aspect of the present invention, in addition to the effects of any one of the first to fifth aspects, the encoded data storage means and the decoded data storage means are constituted by the semiconductor memory. Therefore, in order to easily and quickly display a plurality of moving images on a screen, data can be read out by random access, and a multi-screen moving image can be reproduced without dropping frames.

According to the seventh aspect of the present invention, the encoded data storing step stores the encoded data in the encoded data storage medium, and the reading step is performed based on the number of screen divisions M input from outside. Among the encoded data corresponding to the plurality of bands, the encoded data corresponding to one or a plurality of bands having a data amount corresponding to the data division number M for the M input images is read from the encoded data storage medium, The decoding step performs a decoding process based on the encoded data read by the reading step, and the decoded data storage step stores the decoded data for the M input signals in a decoded data storage medium. In the output step, the decoded data is read out from the decoded data storage medium and is output as reproduced image data. Therefore, the output step can be easily performed based on the encoded data stored in the encoded data storage medium. It is possible to perform the surface video playback.

According to the eighth aspect of the invention, in addition to the effect of the seventh aspect of the invention, the dummy data writing step includes the step of storing the decoded data when the number N of input pictures is smaller than the number M of data divisions. Since the storage area of the medium is divided into M divided M storage areas, the decoded data is stored in N M divided storage areas, and the dummy decoded data is written in (M−N) M divided storage areas. Even when the number of input images is small and the moving image to be displayed does not cover the entire screen, multi-screen display can be easily performed without changing the processing.

According to the ninth aspect of the present invention, in addition to the effect of the seventh or eighth aspect of the present invention, the display step can be performed by:
Since one screen is divided based on the reproduced image data to display a plurality of images, multi-screen reproduction can be easily performed. According to the tenth aspect, in addition to the effect of the seventh aspect, in the reading step, the number of screen divisions M is 4 ^Z ≧ M> 4 ^(Z−1) (Where Z is an integer that satisfies X ≧ Z ≧ 0), the first band, which is the lowest band, is shifted to (3 · (X−
(3) (X−Z) +1) The coded data constituting the (3) (X−Z) +1) bands is read out, so that the amount of data to be read out is easily set to be substantially constant according to the number of divided screens. Even if the number of divided screens increases, the processing time required for display can be made substantially constant, and multi-screen moving image reproduction can be performed regardless of the number of divided screens.

According to the eleventh aspect, the seventh aspect is provided.
In addition to the effects of the invention according to any one of claims 9 to 9,
In the storing step, the discrete cosine transform coefficient of one frame obtained by the encoding is divided into (3 · X + 1) bands that are set in advance and stored, and in the reading step, the screen division number M is 4 ^Z ≧ M> 4 ^(Z-1) (where Z is an integer satisfying X ≧ Z ≧ 0), from the first band, which is the lowest band, to (3 · (X−
Since (3) (X−Z) +1) bands of encoded data up to (Z) +1) band are read, even if discrete cosine transform is used as an image compression method, the data should be read in accordance with the number of divided screens. You can easily set the data amount to be almost constant, and even if the number of divided screens increases,
The processing time required for display can be substantially constant, and multi-screen moving image reproduction can be performed regardless of the number of divided screens.

[Brief description of the drawings]

FIG. 1 is a schematic block diagram of a video recording / reproducing apparatus according to an embodiment.

FIG. 2 is an explanatory diagram (1) of a wavelet transform operation.

FIG. 3 is an explanatory diagram (2) of a wavelet transform operation.

FIG. 4 is an explanatory diagram of a physical format of a memory.

FIG. 5 is an explanatory diagram of a data configuration of frame data.

FIG. 6 is an explanatory diagram (1) of an inverse wavelet transform operation;

FIG. 7 is an explanatory diagram (2) of the inverse wavelet transform operation.

FIG. 8 is an explanatory diagram of a data storage state and a physical format of a memory when reproducing a multi-screen moving image.

FIG. 9 is a flowchart of a multi-screen moving image reproduction process.

FIG. 10 is a flowchart of a multi-screen moving image display process according to the first embodiment.

FIG. 11 is a detailed operation explanatory diagram of the inverse wavelet transform operation.

FIG. 12 is an explanatory diagram of the contents of a frame memory during a wavelet inverse transform operation in multi-screen moving image display.

FIG. 13 is an explanatory diagram of a multi-screen moving image display state.

FIG. 14 is a configuration explanatory diagram of the second embodiment.

FIG. 15 is an explanatory diagram of DCT coefficients and band division of the DCT coefficients.

FIG. 16 is an explanatory diagram of a DCT coefficient rearrangement process.

FIG. 17 is an explanatory diagram of a two-dimensional inverse DCT transform.

FIG. 18 is a flowchart of a multi-screen moving image reproduction process according to the second embodiment.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Video recording / reproducing apparatus 2 Video camera 3 Display 4 Video interface unit 5 Encoder 6 Memory 7 Memory interface unit 8 Decoder 9 Reproduction control unit 11 Wavelet transform unit 12 Quantization unit 13 Variable length encoding unit 14 Formatter unit 15 Inverter formatter unit 16 Variable length decoding unit 17 Inverse quantization unit 18 Wavelet inverse transformation unit SG image signal DG image data DDG decoded image data FL frame data DSB subband image data DQSB quantized subband data DVLG Encoded image data DVLG 'Inverse format variable length Encoded image data DQSB 'Quantized decoded image data DSB' Dequantized image data SDG Decoded image signal

Claims (11)

[Claims] 1. A code that can be written or read electrically or optically, and that divides an input signal into a plurality of bands, quantizes the band, and stores coded data obtained by encoding the plurality of input signals. Data corresponding to the screen division number M for the M input signals of the encoded data corresponding to the plurality of bands, based on the screen division number M input from the outside. Reading means for reading the coded data corresponding to one or a plurality of bands having an amount from the storage means, performing decoding processing based on the coded data read by the reading means, and outputting decoded data Decoding means, and a storage capacity capable of storing at least one frame of the encoded data. An image processing apparatus comprising: decoded data storage means for storing; and output means for reading out the decoded data and outputting it as reproduced image data. 2. The image processing apparatus according to claim 1, wherein when the number N of the input signals is smaller than the number M of data divisions, the storage area of the decoded data storage means is divided into M divided storage areas. , And a dummy data writing means for storing the decoded data in the N M divided storage areas and writing the dummy decoded data in the (M−N) M divided storage areas. Image processing device. 3. The image processing apparatus according to claim 1, further comprising a display unit that divides one screen based on the reproduced image data and displays a plurality of images. Image processing device. 4. The image processing apparatus according to claim 1, wherein the encoding is an X-layer two-dimensional wavelet transform process (X: an integer equal to or greater than 2), and wherein the reading unit performs the screen division. When the number M is 4 ^Z ≧ M> 4 ^(Z−1) (where Z is an integer satisfying X ≧ Z ≧ 0), the first band, which is the lowest band, and the first band (3 · ( X-
An image processing apparatus for reading out the coded data constituting (3 · (X−Z) +1) bands up to (Z) +1) bands. 5. The image processing apparatus according to claim 1, wherein in the encoding, (2 ^X × 2 ^X ) pixels are regarded as one block (X: natural number), and (2 ^X × 2 ^X ) A discrete cosine transform process for generating a number of discrete cosine transform coefficients, wherein the storage unit previously sets a discrete cosine transform coefficient of one frame obtained by the encoding (3 · X + 1)
And the reading means stores the screen division number M when 4 ^Z ≧ M> 4 ^(Z−1) (where Z is an integer satisfying X ≧ Z ≧ 0) From the first band, which is the lowest band, to (3 · (X−
An image processing apparatus for reading out the coded data constituting (3 · (X−Z) +1) bands up to (Z) +1) bands. 6. The image processing apparatus according to claim 1, wherein said encoded data storage means and said decoded data storage means are constituted by a semiconductor memory. Image processing device. 7. An encoded data storing step of dividing an input video into a plurality of bands, quantizing, encoding, and electrically or optically writing and storing the encoded data on a readable encoded data storage medium as encoded data. Based on a screen division number M input from the outside, among the encoded data corresponding to the plurality of bands, one having a data amount corresponding to the data division number M for the M input videos. A reading step of reading the coded data corresponding to a plurality of bands from the coded data storage medium; and a decoding step of performing a decoding process based on the coded data read in the reading step. The decoded data for the M input signals is stored in a decoded data storage medium having a storage capacity capable of storing the encoded data for a frame. An image processing method comprising: reading decoded data from the decoded data storage medium; and outputting the decoded data as reproduced image data. 8. The image processing method according to claim 7, wherein when the input video number N is smaller than the data division number M, the storage area of the decoded data storage medium is divided into M divided storage areas, A dummy data writing step of storing the decoded data in the N M divided storage areas and writing the dummy decoded data in the (M−N) M divided storage areas. Image processing method. 9. The image processing method according to claim 7, further comprising a display step of dividing one screen based on the reproduced image data and displaying a plurality of images. Image processing method. 10. The image processing method according to claim 7, wherein the encoding is an X-layer two-dimensional wavelet transform process (X: an integer equal to or greater than 2). , When the screen division number M is 4 ^Z ≧ M> 4 ^(Z−1) (where Z is an integer satisfying X ≧ Z ≧ 0), the first band which is the lowest band and the second band ( 3. (X-
An image processing method for reading out the encoded data constituting (3 · (X−Z) +1) bands up to (Z) +1) bands. 11. The image processing method according to claim 7, wherein in the encoding, (2 ^X × 2 ^X ) pixels are regarded as one block (X: natural number), and (2 ^X × 2 ^x ) discrete cosine transform processing for generating discrete cosine transform coefficients, wherein the storing step previously sets a discrete cosine transform coefficient of one frame obtained by the encoding (3 · X + 1).
The readout step is performed when the screen division number M is 4 ^Z ≧ M> 4 ^(Z−1) (where Z is an integer satisfying X ≧ Z ≧ 0). , From the first band, which is the lowest band, to (3 · (X−
An image processing method for reading out the encoded data constituting (3 · (X−Z) +1) bands up to (Z) +1) bands.