CN100571389C - Be used for the compressed and decompressed method and apparatus of encoding/decoding image and expanded images - Google Patents

Abstract

The invention provides a kind of encoding/decoding image and compressed and decompressed method and apparatus of expanded images of being used for.These method for encoding images etc. were expressed as matrix with block of pixels carrying out spatial frequency transforms when for example handling JPEG etc. before, the sum data sr (j) that calculates the interior pixel value of the sum data sc (i) of the pixel value in the row i in this matrix and the capable j in this matrix (S22), the nc (i) and the nr (j) that calculate, to make sc (i) and sr (j) respectively with descending (S23), and thinking the rational predetermined condition establishment of displacement (S24, S28) time, carry out the displacement (S26 of matrix, S30), generation comprise with this matrix permutation be required nc of reset condition (i) and nr (j) header information Ph (n) (S32), and add this header information to head such as the compressed image file of JPEG etc.

Description

Be used for the compressed and decompressed method and apparatus of encoding/decoding image and expanded images

Technical field

The present invention relates to method for encoding images, picture decoding method, moving image compression method, moving image decompression method, image encoding program, picture coding device, picture decoding apparatus, encoding/decoding image system and expanded images compression/decompression compression system, these methods, program, device and system encode to compress to rest image or moving image, perhaps/and they are decoded to decompress.

Background technology

In field of information processing, various types of technology that are used for packed data have been developed.For data (particularly for view data), the example of known compress technique comprises JPEG, the JPEG 2000 etc. that are used for rest image, and the MPEG, MPEG 2, the MPEG4 etc. that are used for moving image.

For example, in the CCIT suggestion T.81, " Information technology-digitalcompression and coding of continuous-tone still images-requirements andguidelines ", ITU has described the compression of above-mentioned use JPEG in 1992.Yet, the summary that JPEG handles will be described for convenience.

With the coloured image is example, at first will be divided into 8 * 8 block of pixels respectively at every kind of color by the image that three kinds of colors (for example, brightness signal Y and two color difference signal Cb and Cr) constitutes.Then, utilize DCT (discrete cosine transform) with each block of pixels of the block of pixels of brightness signal Y and two color difference signal Cb and Cr respectively (that is, three times) transform in the frequency space.Subsequently, use luminance signal quantization table or color difference signal quantization table (Cb and Cr are shared) that the DCT coefficient of each block of pixels is quantized, and use luminance signal Huffman table or color difference signal Huffman table (Cb and Cr are shared) that the DCT coefficient of each block of pixels is carried out entropy coding.

Although traditional JPEG has been had improvement (for example, the scalability of reversible/irreversible compression/picture quality, the minimizing of mosquito noise etc.), the JPEG 2000 that is used for the rest image compression does not also popularize.JPEG is considered to be used in the future the one preferred technique of image compression.

In recent years, the quantity of the image pickup pixel that adopts in digital camera and the cell phone significantly increases.And, be that the common equipment (often continue carry equipment) of representative has been widely used with cell phone etc., thereby the chance of taking increases, and the amount of the digital picture of being handled by the individual also increase.Therefore, in order to tackle this increase of amount of information, obviously the amount of desirable image data is less.

Yet by traditional JPEG, the unique method that reduces the size of compressing image data is to adopt more sparse quantization table, promptly enlarges quantization width.Be well known that this will increase above-mentioned mosquito noise etc., thereby reduced picture quality.

Even proposed to be used for when compression ratio is provided with higherly, also can preventing to produce the multiple technologies of this noise so far.

For example, Japanese Unexamined Patent Application communique No.2000-308058 has described so a kind of technology, wherein, can reduce the mosquito noise that on the part of image border, produces easily by carrying out preliminary treatment before handling at JPEG to reduce the signal level difference of image border part.

In addition, Japanese Unexamined Patent Application communique No.8-163375 has described so a kind of technology, wherein, distribution according to pixel value is divided into a plurality of zones with block of pixels, at each zoning sparse pixel is compensated, thereby obtain the size of each block of pixels, and each block of pixels is carried out the JPEG conversion.

Yet, by the technology described in the above-mentioned Japanese Unexamined Patent Application communique No.2000-308058, the relevant information of preliminary treatment that this pretreated device side stores and before JPEG encodes the image border part carried out need carried out, therefore, can not reproduce original image from compressed image file separately.And by this preliminary treatment, the whole block of pixels of view data has been carried out the low level conversion, thereby has increased quantization error, has therefore reduced picture quality.

In addition, because the technology of describing among the above-mentioned Japanese Unexamined Patent Application communique No.8-163375 produces a plurality of block of pixels from single block of pixels,, thereby can not finish processing so the trial of using traditional jpeg decoder to carry out decoding processing will cause error.

As mentioned above, such compressed format is not also proposed, this compressed format can only briefly compressed image file decode by jpeg decoder, even thereby when being provided with carefully, quantization table also can compress by a spot of sign indicating number, even perhaps when quantization table is provided with sparsely, also can prevent mosquito noise.

At above-mentioned situation the present invention is proposed, the objective of the invention is to provides method for encoding images, picture decoding method, moving image compression method, moving image decompression method, image encoding program, picture coding device, picture decoding apparatus, encoding/decoding image system and expanded images compression/decompression compression system at the coding method that utilizes spatial frequency transforms, these methods, program, device and system have reduced the amount of sign indicating number, and/or can suppress noise, and/or can decode to coded data.

Summary of the invention

To achieve these goals, to be applied to the basic coding method according to a kind of method for encoding images of first invention, this basic coding method comprises: the spatial frequency transforms step is used to utilize spatial frequency transforms that view data or difference image data are transformed to conversion coefficient in the frequency space from image space; And head produces step, be used for adding the information relevant to based on described conversion coefficient data as the head part with described view data or difference image data, described method for encoding images comprises: the spatial alternation step is used for before the spatial frequency transforms step described view data or difference image data being carried out the conversion of reversible graph image space; And header information interpolation step, be used for partly adding the required spatial alternation parameter of inverse transformation of carrying out described image space conversion to described head.

In addition, by according to the second inventive images coding method, in according to the described first inventive images coding method, described basic coding method also comprises coding step, this coding step is used for described conversion coefficient is encoded, make the data volume of low frequency coefficient generally less than the data volume of high frequency coefficient, and the reversible graph image space conversion of carrying out in described spatial alternation step has suppressed the high frequency coefficient that caused by spatial frequency transforms.

In addition, by according to the 3rd inventive images coding method, in according to the first inventive images coding method, described basic coding method also comprises partiting step, this partiting step is used for view data or difference image data are divided into block of pixels, and the described block of pixels that described spatial frequency transforms step will be divided in this partiting step transforms in the frequency space, and described spatial alternation carried out the image space conversion of described reversible graph at least one block of pixels in the described block of pixels before the spatial frequency transforms step.

By according to the 4th inventive images coding method, in according to the 3rd inventive images coding method, in described spatial alternation step, block of pixels is expressed as matrix M, but and described inverse spatial transform comprise at least a in the line replacement (row-wise permutation) of M and the column permutation (column-wisepermutation).

By according to the 5th inventive images coding method, in according to the 3rd inventive images coding method, in described spatial alternation step, block of pixels is expressed as matrix M, but and represents described inverse spatial transform by reversible linear matrix computing

M→AMB，

Wherein, A and B represent invertible matrix.

By according to the 6th inventive images coding method, in according to the 5th inventive images coding method, a matrix in described matrix A and the matrix B is the matrix that is used for being undertaken by matrix operation the line replacement of matrix M, and another matrix is the matrix that is used for being undertaken by matrix operation the column permutation of matrix M.

By according to the 7th inventive images coding method, in according to the 5th inventive images coding method, the matrix that is used to carry out line replacement is represented as the matrix that is used for certain delegation is added to another row and the matrix product that is used for certain delegation be multiply by-1 matrix, and the matrix that is used to carry out column permutation is represented as the matrix that is used for a certain row are added to another row and the matrix product that is used for a certain row be multiply by-1 matrix.

By according to the 8th inventive images coding method, in according to any one method for encoding images in the 4th to the 6th invention, carry out the displacement between the row of matrix M, make after displacement in the ranks, the summation that follows the pixel value of direction is ascending order or descending on column direction, and carrying out the displacement between the row of matrix M, make after displacement between row, is ascending order or descending on line direction along the summation of the pixel value of column direction.

By according to the 9th inventive images coding method, in according to the first inventive images coding method, in described spatial alternation step, but carry out described inverse spatial transform by the layout that changes the pixel in described view data or the described difference image data.

By according to the tenth inventive images coding method, in according to the first or second inventive images coding method, the conversion coefficient that in described spatial frequency transforms step, obtains comprise with the mean value of described view data or with the proportional DC coefficient of the mean value of described difference image data, and the described image space conversion of carrying out in described spatial alternation step keeps described DC coefficient constant.

By according to the 11 inventive images coding method, in according to the 3rd inventive images coding method, the described conversion coefficient that obtains in described spatial frequency transforms step comprises and the proportional DC coefficient of the mean value of described block of pixels, and the described image space conversion of carrying out in described spatial alternation keeps described DC coefficient constant.

Can be applicable to the JPEG encoding process according to the 12 inventive images coding method, this method for encoding images comprises: the spatial alternation step is used for before carrying out discrete cosine transform at least one block of pixels being carried out the conversion of reversible graph image space; Spatial alternation parameter extraction step is used to extract the required spatial alternation parameter of inverse transformation of carrying out described image space conversion; And header information interpolation step, be used for partly being added on the spatial alternation parameter that described spatial alternation parameter extraction step is extracted to the head of jpeg file.

By according to the 13 inventive images coding method, in according to the 12 inventive images coding method, it is constant by the DC coefficient in the DCT coefficient that carries out the discrete cosine transform acquisition that described image space conversion keeps, and increase the amplitude of the low frequency AC coefficient in the described DCT coefficient.

According to the 14 the invention view data or difference image data are carried out the image encoded coding method, this method for encoding images comprises: the spatial alternation step, but be used in image space, described view data or difference image data being carried out inverse spatial transform; The spatial frequency transforms step is used for view data or difference image data are converted into the frequency space from image space, thereby obtains conversion coefficient; Coding step, being used for described transform coefficients encoding is coded data; And header information produces step, is used for integrally adding the required information of the inverse transformation of carrying out described spatial alternation as header information to described coded data; After described coding step, the data volume that is used for the low frequency coefficient side is generally less than the data volume that is used for the high frequency coefficient side; And described spatial alternation step has suppressed the high frequency coefficient that caused by described spatial frequency transforms generally.

According to the 15 inventive images coding/decoding method, this picture decoding method can be applicable to the elementary solution code method, this elementary solution code method comprises: contrary spatial frequency transforms step, be used to utilize contrary spatial frequency transforms to carrying out conversion from coded image data or the conversion coefficient that produces from the code differential view data, thereby obtain the image space data, be used for utilizing the coded image data or the code differential view data of having carried out encoding to decode according to the first inventive images coding method, described picture decoding method comprises: the header information reading step is used for partly reading spatial alternation parameter at least from described head; And the inverse spatial transform step, be used for the image space data that produced by described contrary spatial frequency transforms step being carried out the inverse transformation of described image space conversion according to the described spatial alternation parameter of reading at described header information reading step.

By according to the 16 inventive images coding/decoding method, in according to the 15 inventive images coding/decoding method, this picture decoding method also comprises: the head identification step, be used for identification and whether described described spatial alternation parameter in described head part, if in described head identification step, recognize described spatial alternation parameter, then export by carrying out view data or the difference image data that described inverse spatial transform step obtains based on described spatial alternation parameter, but in described head identification step, do not recognize under the situation of described spatial alternation parameter, export the view data or the difference image data that do not carry out described inverse spatial transform step and obtain.

Moving image compression method according to the 17 invention is used to utilize according to compressing in the frame of the view data in the picture frame and compressing according to the interframe of a plurality of picture frames, the motion image data that comprises a plurality of picture frames that are time series formula relation is encoded, and compression can be applicable to compression method in the basic frame in the described frame, compression method comprises in this basic frame: partiting step is used for picture frame is divided into block of pixels; The spatial frequency transforms step is used for described block of pixels is carried out spatial frequency transforms to obtain conversion coefficient; And header information produces step, be used for adding the information relevant to data based on the described conversion coefficient that in described spatial frequency transforms step, obtains with described view data, as the head part, compression also comprises in the described frame: the spatial alternation step, and at least one that was used for before described spatial frequency transforms step described block of pixels carried out the conversion of reversible graph image space; And header information interpolation step, be used for the spatial alternation parameter that the inverse transformation of carrying out described image space conversion is required and add described head part to.

By moving image compression method according to the 18 invention, in moving image compression method according to the 17 invention, this moving image compression method also comprises: spatial alternation model selection step is used for selecting whether to carry out described spatial alternation step and described header information interpolation step.

Moving image compression method according to the 19 invention is used to utilize based on compressing in the frame of the view data in the picture frame and compressing based on the interframe of a plurality of picture frames, the motion image data that comprises a plurality of picture frames that are time series formula relation is encoded, and described interframe compression can be applicable to basic interframe compression method, this basic interframe compression method comprises: partiting step is used for picture frame is divided into block of pixels; The difference calculation procedure is used for poor by between the respective pixel piece in the picture frame that obtains at least one block of pixels in the described block of pixels that this partiting step is divided and will compare, produces the differential pixel piece; The spatial frequency transforms step is used for described differential pixel piece is carried out spatial frequency transforms; And header information produces step, be used for adding the information relevant to obtain data as the head part based on described conversion coefficient with described view data, the compression of described interframe also comprises: spatial alternation is used for that at least one the differential pixel piece to described differential pixel piece carries out the conversion of reversible graph image space before described spatial frequency transforms step; And header information interpolation step, be used for the spatial alternation parameter that the inverse transformation of carrying out described image space is required and add described head part to.

Moving image compression method according to the 20 invention, in according to the moving image compression method aspect the 19, this moving image compression method also comprises: spatial alternation model selection step is used for selecting whether to carry out described spatial alternation step and described header information interpolation step.

By the moving image compression method according to the 21 invention, in the moving image compression method according to the 19 invention, described header information produces the head that step produces the group that is used to comprise a plurality of frames; And it is the step that is used for described spatial alternation parameter is added to the described head part of corresponding group that described header information adds step.

Dynamic image decoding method according to the 22 invention, this dynamic image decoding method can be applicable to the basic exercise picture decoding method, this basic exercise picture decoding method comprises: contrary spatial frequency transforms step, be used to utilize contrary spatial frequency transforms that the conversion coefficient from the compression movement view data is carried out conversion, to obtain the image space data of block of pixels, be used for the motion image data that has carried out coding by the moving image compression method according to the 17 invention is decoded, described dynamic image decoding method comprises: the header information reading step is used for partly reading described at least spatial alternation parameter from described head; And the inverse spatial transform step, be used for the described spatial alternation parameter that basis is read at described header information reading step, the block of pixels that is produced by described contrary spatial frequency transforms step is carried out the inverse transformation of described image space conversion.

Moving image decompression method according to the 23 invention, this moving image decompression method can be applicable to the basic exercise picture decoding method, this basic exercise picture decoding method comprises: contrary spatial frequency transforms step, be used to utilize contrary spatial frequency transforms that the conversion coefficient from the compression movement view data is carried out conversion, to obtain the image space data of differential pixel piece, be used for the motion image data that has carried out coding by the moving image compression method according to the 19 invention is decoded, described moving image decompression method comprises: the header information reading step is used for partly reading described at least spatial alternation parameter from described head; And the inverse spatial transform step, be used for the described spatial alternation parameter that basis is read at described header information reading step, the differential pixel piece that is produced by described contrary spatial frequency transforms step is carried out the inverse transformation of described image space conversion.

According to the 24 inventive images coded program, thereby this image encoding program is carried out by computer view data or difference image data is encoded, described image encoding program comprises: the spatial alternation step, but be used in image space, described view data or difference image data being carried out inverse spatial transform; The spatial frequency transforms step is used for view data or difference image data are converted into the frequency space from image space, thereby obtains conversion coefficient; Coding step, being used for described transform coefficients encoding is coded data; Header information produces step, is used for integrally adding the required information of the inverse transformation of carrying out described spatial alternation as header information to described coded data.

Be used for view data or difference image data are encoded according to the 25 inventive images code device, this picture coding device comprises: spatial transform unit is used for view data or difference image data are carried out reversible graph image space conversion in the image space; The spatial frequency transforms unit is used for view data or difference image data are carried out spatial frequency transforms, thereby obtains conversion coefficient; The header information generator is used for adding the information relevant with view data or difference image data to obtain based on described conversion coefficient data; And the header information adding device, be used for the spatial alternation parameter that the inverse transformation of carrying out described image space conversion is required and add described head part to.

Be used for the coded data of having carried out coding by the described picture coding device according to the 25 invention is decoded according to the 26 inventive images decoding device, described picture decoding apparatus comprises: contrary spatial frequency transforms unit, be used for the conversion coefficient that produces from described coded data is carried out contrary spatial frequency transforms, thereby obtain the pixel data in the image space; The head interpreter is used for reading described at least spatial alternation parameter from described head; And the inverse spatial transform unit, be used for the pixel data of image being carried out the inverse transformation of described image space conversion, thereby obtain view data or difference image data according to the described spatial alternation parameter of reading by described head interpreter.

Comprise according to the 27 inventive images coding/decoding system: according to the 25 inventive images code device; And according to the 26 inventive images decoding device.

Expanded images coding/decoding system according to the 28 invention comprises the first encoding/decoding image system, this first encoding/decoding image system comprises first picture coding device, first picture decoding apparatus, second picture coding device and second picture decoding apparatus, this first picture coding device comprises: division unit is used for raw image data is divided into block of pixels; The spatial frequency transforms unit is used for obtaining conversion coefficient by the described block of pixels that is produced by described division unit is carried out spatial frequency transforms; Quantizer is used for the described conversion coefficient from described spatial frequency transforms unit is quantized, thereby produces the integer value (integral value) of conversion coefficient; Entropy coder is used for these integer values of conversion coefficient are carried out entropy coding; And compressed image file generation device, the data that are used for obtaining by the result who the information relevant with described view data is added to according to described entropy coder produce first compressed image file as the head part, this first picture decoding apparatus comprises: the head interpreter is used for reading described head part from described first compressed image file; Entropy decoder is used for by the sign indicating number that obtains from described first compressed image file is carried out the integer value that the entropy decoding obtains conversion coefficient; Inverse quantizer, the integer value that is used for conversion coefficient that described entropy decoder is produced goes to quantize, thereby obtains conversion coefficient; Contrary spatial frequency transforms unit, being used for by the conversion coefficient from described inverse quantizer being carried out contrary spatial frequency transforms obtains with the block of pixels is the pixel data of unit; And reproduce (rendering) and install, be used for producing raw image data by described block of pixels is combined as raw image data, this second picture coding device is used to produce second compressed image file, this second picture coding device obtains with lower unit by adding to described first picture coding device: spatial transform unit, and being used for before carrying out spatial frequency transforms described block of pixels being carried out can be by the reversible graph image space conversion of spatial alternation parameter recognition; And header information adding device, be used for adding described spatial alternation parameter to described head part, this second picture decoding apparatus is used for obtaining raw image data from second compressed image file, this second picture decoding apparatus obtains with lower unit by adding to described first picture decoding apparatus: additional header information reading device is used for partly reading described spatial alternation parameter from described head; And the inverse spatial transform unit, be used for carrying out the inverse transformation of described image space conversion according to the spatial alternation parameter of reading by this additional header information reading device.In this extended coding/decode system, at compressed image file to be processed is under the situation of second compressed image file, and described first picture decoding apparatus can recover the data different with described raw image data by ignore the spatial alternation parameter of describing in the head part of described second compressed image file; And wherein, can not partly read under the situation of described spatial alternation parameter from described head at described additional header information reading device, described second picture decoding apparatus does not carry out recovering raw image data by the processing that described inverse spatial transform unit carries out by only carrying out the decoding of partly being carried out by first picture decoding apparatus.

In expanded images coding/decoding system, also comprise according to the 29 expanded images coding/decoding system of inventing according to the 28 invention: image collecting device, it has described second picture coding device; And computer, it has described first picture decoding apparatus.

By according to the 30 inventive images coding method, in according to the first inventive images coding method, described spatial alternation step is the step that is used to carry out to regulate by at least one the image space conversion that parameter regulates, and described header information to add step be the step that is used for described adjusting parameter is added to described head part.

By according to the 31 inventive images coding method, in according to the 30 inventive images coding method, described basic coding method also comprises the partiting step that is used for view data or difference image data are divided into block of pixels, and described spatial frequency transforms step is the step that is used for the described block of pixels of dividing at described partiting step is carried out spatial frequency transforms, and described spatial alternation step is that at least one that be used for before carrying out described spatial frequency transforms step described block of pixels carried out the step of reversible graph image space conversion.

By according to the 32 inventive images coding method, in according to the 31 inventive images coding method, in described image transform step, block of pixels is expressed as matrix M, and this matrix M is used being represented as of invertible matrix A

M→AM

The invertible matrix computing and use being represented as of invertible matrix B

M→MB

The invertible matrix computing at least a invertible matrix computing.

By according to the 33 inventive images coding method, in according to the 32 inventive images coding method, described matrix A and matrix B are the matrixes that is used for the block of pixels implementation space low-pass filtering of being represented by described matrix M.

By according to the 34 inventive images coding method, in according to the 33 inventive images coding method, described adjusting parameter comprises that the filtering of the intensity that is used to regulate low pass spatial filtering regulates parameter.

By according to the 35 inventive images coding method, in according to the 34 inventive images coding method, described basic coding method also comprises quantization step, be used for utilizing specific quantization width that the conversion coefficient that obtains in described spatial frequency transforms is quantized, it is to be used for adding the information relevant with described view data to data that the described result according to described quantization step the obtains step as the head part that described head produces step, and regulates described quantization width according to described filtering adjusting parameter.

By according to the 36 inventive images coding method, in according to the 30 inventive images coding method, the described conversion coefficient that in described spatial frequency transforms step, obtains comprise with block of pixels in the proportional DC coefficient of mean value of view data, and described spatial alternation step remains unchanged described DC coefficient.

Be used for optionally carrying out following method according to the 37 inventive images coding method: according to the 6th inventive images coding method; And according to the 33 inventive images coding method.

According to an aspect of the present invention, provide a kind of method for encoding images, this method for encoding images comprises: view data is carried out spatial frequency transforms, to obtain conversion coefficient; With described transform coefficients encoding is coded data; Produce the head part of described coded data, this head partly comprises the information relevant with described view data; Wherein, before carrying out described spatial frequency transforms, but described view data is suitable for the inverse spatial transform of described view data; And but the header information that the inverse transformation of described inverse spatial transform is required adds in the described head part, wherein, but described inverse spatial transform is selected from replacement operator and reversible low-pass filtering operation, but and described inverse spatial transform suppressed the high frequency conversion coefficient that causes owing to described spatial frequency transforms.

According to another aspect of the present invention, provide a kind of method for encoding images of the JPEG of being applied to encoding process, this method for encoding images comprises: a) be a plurality of 8 * 8 image blocks with image division; B) each image block in described 8 * 8 image blocks is carried out discrete cosine transform, thereby obtain the DCT coefficient, described DCT coefficient comprises DC coefficient, low frequency AC coefficient and the high-frequency AC coefficient of the mean value of representing described 8 * 8 image blocks; C) utilize quantization table that described DCT coefficient is quantized; D) utilize run length coding, RLC and H ǔ ffman subsequently coding to encode through the DCT coefficient that quantizes to described; And e) produces the head part, described method comprises: but before carrying out described discrete cosine transform, at least one 8 * 8 image block is carried out inverse spatial transform, but select described inverse spatial transform adaptively in response to described at least one 8 * 8 image block thus; Extract the required spatial alternation parameter of inverse transformation of described spatial alternation; And the header information of the described spatial alternation parameter of expression added in the described head part, but wherein said inverse spatial transform is selected from replacement operator and reversible low-pass filtering operation, but and described inverse spatial transform suppressed the high frequency conversion coefficient that causes owing to described spatial frequency transforms.

According to another aspect of the present invention, a kind of picture decoding method that is used for comprising that head part and coded data coded image data are partly decoded is provided, this picture decoding method comprises: read the described head part of described coded image data, to extract the information relevant with view data; Described coded data is decoded, to obtain the conversion coefficient in the spatial frequency domain; Described conversion coefficient is carried out contrary spatial frequency transforms, thereby obtain second view data; Described coding/decoding method also comprises: but the required information of inverse transformation of extraction inverse spatial transform; But described second view data is carried out the inverse transformation of described inverse spatial transform, to obtain described view data, but wherein said inverse spatial transform is selected from replacement operator and reversible low-pass filtering operation, but and described inverse spatial transform suppressed the high frequency conversion coefficient that causes owing to described spatial frequency transforms.

According to another aspect of the present invention, a kind of moving image compression method is provided, this moving image compression method is used to utilize based on compressing in the frame of the view data in the picture frame and compressing based on the interframe of a plurality of picture frames, the motion image data that comprises a plurality of picture frames that are time series formula relation is encoded, wherein, compression comprises in the described frame: picture frame is divided into a plurality of image blocks; Described image block is carried out spatial frequency transforms, to obtain conversion coefficient; And producing the head part comprise the information relevant with described picture frame, the interior compression of described frame also comprises: but before carrying out described spatial frequency transforms, in the described image block at least one carried out inverse spatial transform; And but the header information of the spatial alternation parameter that the inverse transformation that comprises described inverse spatial transform is required adds in the described head part, but wherein said inverse spatial transform is selected from replacement operator and reversible low-pass filtering operation, but and described inverse spatial transform suppressed the high frequency conversion coefficient that causes owing to described spatial frequency transforms.

According to another aspect of the present invention, a kind of moving image compression method is provided, this moving image compression method is used to utilize based on compressing in the frame of the view data in the picture frame and compressing based on the interframe of a plurality of picture frames, the motion image data that comprises a plurality of picture frames that are time series formula relation is encoded, wherein, described inter-frame compression is drawn together: picture frame is divided into a plurality of block of pixels; By obtaining the difference between at least one and the respective pixel piece relevant in the described block of pixels, produce image block with the picture frame that will compare; Described image block is carried out spatial frequency transforms to obtain conversion coefficient; And producing the head part that comprises the information relevant with described picture frame, described interframe compression also comprises: before carrying out described spatial frequency transforms, but at least one image block in the described image block is carried out inverse spatial transform; And but the header information of the spatial alternation parameter that the inverse transformation that comprises described inverse spatial transform is required adds in the described head part, but wherein said inverse spatial transform is selected from replacement operator and reversible low-pass filtering operation, but and described inverse spatial transform suppressed the high frequency conversion coefficient that causes owing to described spatial frequency transforms.

According to another aspect of the present invention, provide a kind of being used for that the encoding moving pictures data are decoded to obtain the dynamic image decoding method of motion image data, these encoding moving pictures data comprise head part and coded data part, described dynamic image decoding method comprises: read described head part, to extract the information relevant with view data, this view data is relevant with picture frame; Described coded data is partly decoded, to obtain the conversion coefficient relevant with described picture frame; Described conversion coefficient is carried out contrary spatial frequency transforms, thereby obtain second view data; Described coding/decoding method also comprises: but from the required header information of inverse transformation of described head extracting section inverse spatial transform; But described second view data is carried out the inverse transformation of described inverse spatial transform, to obtain the view data relevant with described picture frame, but wherein said inverse spatial transform is selected from replacement operator and reversible low-pass filtering operation, but and described inverse spatial transform suppressed the high frequency conversion coefficient that causes owing to described spatial frequency transforms.

According to a further aspect of the invention, a kind of picture coding device is provided, this picture coding device comprises: spatial transform unit, but be used for view data is suitable for the inverse spatial transform of described view data, but wherein said inverse spatial transform is selected from replacement operator and reversible low-pass filtering operation; The spatial frequency transforms unit, but be used for the described view data of carrying out described inverse spatial transform is carried out spatial frequency transforms, thus obtain conversion coefficient; Head part generator is used to produce the head part that comprises the information relevant with described view data; And header information adding device, the information that is used for the spatial alternation parameter that the inverse transformation that comprises described spatial alternation is required is added described head part to, wherein, but described inverse spatial transform has suppressed the high frequency conversion coefficient that causes owing to described spatial frequency transforms.

According to a further aspect of the invention, a kind of picture decoding apparatus is provided, be used for the coded image data that comprises head part and coded data part is decoded, described picture decoding apparatus comprises: contrary spatial frequency transforms unit, be used for the conversion coefficient that produces from described coded image data is carried out contrary spatial frequency transforms, thereby obtain second view data, the head interpreter is used for partly reading described spatial alternation parameter from described head; And inverse spatial transform unit, be used for described second view data being carried out inverse spatial transform according to described spatial alternation parameter, thereby acquisition view data, wherein, described inverse spatial transform is selected from the inverse transformation of replacement operator and the inverse transformation of reversible low-pass filtering operation, and described replacement operator can suppress the high frequency conversion coefficient that causes owing to spatial frequency transforms.

The present invention also provides a kind of image encoding-decode system, and this image encoding-decode system comprises: another aspect according to the present invention and provide picture coding device; And according to the present invention another aspect and the picture decoding apparatus that provides; Wherein: described picture decoding apparatus can be decoded to the coded image data that has carried out coding by described picture coding device.

Description of drawings

Fig. 1 is the block diagram of expression according to the structure of the expansion jpeg coder of first embodiment of the invention.

Fig. 2 is the block diagram of expression according to the structure of the expansion jpeg decoder of first embodiment.

Fig. 3 is the flow chart of expression according to the encoding process of first embodiment of the invention.

Fig. 4 is the flow chart of expression according to the decoding processing of first embodiment of the invention.

Fig. 5 is the flow chart of expression according to the JEX encoding process of first embodiment of the invention.

Fig. 6 is the flow chart of expression according to the JEX decoding processing of first embodiment of the invention.

Fig. 7 is the figure of a plurality of example for the expression first embodiment of the invention with the VGA image division.

Fig. 8 is the figure of the dot structure of 8 * 8 block of pixels of expression first embodiment of the invention.

Fig. 9 will carry out the figure of example of the image of 2-D shape reproduction by filling (padding) hexagon pixel for the expression first embodiment of the invention.

Figure 10 is for wherein being arranged on expression the figure that pixel data among Fig. 9 follows the example that direction and column direction arrange.

Figure 11 will obtain for arrangement nc (i) for the calculating of representing first embodiment of the invention

The chart of the example of each element that value Chc is required.

Figure 12 represents the chart how decision value of first embodiment of the invention changes along with the variation of weight w.

Figure 13 is the figure of the example of the matrix M of expression first embodiment of the invention.

Figure 14 is the chart of the interaction order of the basic matrix that is used to the basic displacement between realizing being listed as of expression first embodiment of the invention.

Figure 15 is the chart from the data instance of original 8 * 8 block of pixels of image data extraction of expression first embodiment of the invention.

Figure 16 for the expression first embodiment of the invention about the summation that follows direction of 8 * 8 block of pixels data and along the chart of each result of calculation of the summation of column direction.

Figure 17 for the carrying out of expression first embodiment of the invention the chart of block of pixels data of displacement.

Figure 18 for the carrying out of expression first embodiment of the invention the chart of data of level shift (level shift) and DCT.

Figure 19 for the carrying out of expression first embodiment of the invention the chart of 8 * 8 blocks of data that quantize.

Figure 20 for expression by in decoding processing shown in Figure 4 to carrying out shown in Figure 19 the data that quantize carry out re-quantization and inverse DCT (IDCT), and these data are carried out level shift to be rounded to the chart of the data that obtain in 0 to 255 the scope.

Figure 21 has carried out the chart of the block of pixels of displacement for expression by data shown in Figure 20 just being carried out inverse permutation.

Figure 22 deducts the result's that original pixels blocks of data shown in Figure 15 obtains chart from decoded pixel blocks of data shown in Figure 21 for expression.

Figure 23 is the figure of the basic structure of the EXIF file of the storage JPEG packed data of expression first embodiment of the invention.

Figure 24 is the figure of the basic structure of the JPEG compression data file of representing first embodiment of the invention in further detail.

Figure 25 is the chart of structure and the concrete data instance of the header information Ph (n) of expression first embodiment of the invention.

Figure 26 can handle the equipment of expansion JPEG and the figure of the system configuration of the equipment that can handle traditional JPEG for wherein having made up of expression first embodiment of the invention.

Figure 27 is the block diagram of structure that can carry out the encoder of JEX to moving image of expression first embodiment of the invention.

Figure 28 is the block diagram of the structure of the traditional jpeg coder of expression.

Figure 29 is the block diagram of the structure of the traditional jpeg decoder of expression.

Figure 30 carries out the result's that DCT obtains chart to original 8 * 8 block of pixels data shown in Figure 15 for expression.

The result's that Figure 31 quantizes to obtain to blocks of data shown in Figure 30 for expression chart.

Figure 32 carries out re-quantization and inverse DCT for expression to blocks of data shown in Figure 31, and described blocks of data is carried out the chart of level shift with the result that obtains in the data that it are rounded to scope between 0 to 255.

Figure 33 deducts the result's that original pixels blocks of data shown in Figure 15 obtains chart from decoded pixel blocks of data shown in Figure 32 for expression.

Figure 34 is the block diagram of the structure of expression conventional motion image encoder.

Figure 35 is the block diagram of expression according to the structure of the expansion jpeg coder of second embodiment of the invention.

Figure 36 is the block diagram of expression according to the structure of the expansion jpeg decoder of second embodiment of the invention.

Figure 37 is the flow chart of expression according to the summary of the JEX encoding process of second embodiment of the invention.

Figure 38 is the flow chart of the pretreated details of JEX (LPF) among the step S61 of expression Figure 37.

Figure 39 for the Control Parameter of selecting according to setting option Choice of expression second embodiment of the invention to (η, chart SF).

Figure 40 is the flow chart of the main details of handling of JEX (LPF) of the step S62 among expression Figure 37.

Figure 41 is the V among the step S85 among expression Figure 40 _ScFlow chart with the details of kc computing.

Figure 42 is the V among the step S86 of expression Figure 40 _SrFlow chart with the details of lr computing.

Figure 43 is the flow chart of JEX (LPF) decoding processing in the expression second embodiment of the invention.

Figure 44 is the chart of expression according to the example of the Mode vector sum Pos vector of second embodiment of the invention.

Figure 45 is the figure of expression according to the example of the header information Phl (n) of second embodiment of the invention.

Figure 46 is the chart according to the example of the global header of entire image of expression second embodiment of the invention.

Figure 47 is the chart of the example of the place value in the classification that will be recorded in global header of expression second embodiment of the invention.

Figure 48 is the chart of the example of first 8 place value of expression global parameter shown in Figure 46.

Figure 49 is the chart of the example of second 8 place value of expression global parameter shown in Figure 46.

Figure 50 is the chart from the data instance of original 8 * 8 block of pixels of image data extraction of expression second embodiment of the invention.

Figure 51 carries out the result's that level shift and DCT obtain chart to original 8 * 8 block of pixels data shown in Figure 50 for expression.

The result's that Figure 52 quantizes to obtain to the blocks of data shown in Figure 51 for expression chart.

Figure 53 carries out re-quantization and inverse DCT for expression to the blocks of data shown in Figure 52, and this blocks of data is carried out the chart of level shift with the result that obtains in the data that it are rounded to scope between 0 to 255.

Figure 54 for expression from the carrying out shown in Figure 53 deduct the result's that original pixels blocks of data shown in Figure 50 obtains chart the block of pixels data of decoding.

The chart of the pattern information of extracting in the information from the head of Figure 55 for the expression second embodiment of the invention and the example of filtering positional information.

Figure 56 carries out the result's that 8 low-pass filtering treatment shown in Figure 55 obtain chart to original 8 * 8 block of pixels data shown in Figure 50 for expression.

Figure 57 for expression to the carrying out shown in Figure 56 8 * 8 block of pixels data of low-pass filtering treatment carry out the result's that level shift and DCT obtain chart.

The result's that Figure 58 quantizes to obtain to the blocks of data shown in Figure 57 for expression chart.

Figure 59 carries out re-quantization and inverse DCT for expression to the blocks of data shown in Figure 58, and this blocks of data is carried out level shift and contrary LPF handle with in the data that it are rounded to scope between 0 to 255 and the result's who obtains chart.

Figure 60 is the flow chart of the example of JEX master's processing of wherein low-pass filtering and displacement being made up of expression third embodiment of the invention.

Figure 61 is the chart of the example of the expression header information that will produce in the step S137 of Figure 60.

Figure 62 is the chart of the example of the expression header information that will produce in the step S136 of Figure 60.

The chart of example of the header information of Figure 63 during for another low-pass filtering of employing of expression third embodiment of the invention.

Embodiment

For the term that adopts in the following explanation,, before being described, specific embodiments of the invention will make their definition clearer here.

At first, term " pixel " is meant the element of composing images, and it comprises the information of locus (location of pixels) that presentation video is interior and the value of locating in this locus (pixel value).Note, image described herein is not limited to by pixel being arranged on the image that limits in the 2-D Euclidean space in discrete mode, and can be by pixel being arranged on the image that limits in the 2-D Riemann space in discrete mode, more generally, can be by pixel being arranged on the image that limits in 2-D compact manifold (compact manifold) K in discrete mode.

Term " pixel groups " is meant one group of pixel.In fact, pixel groups is used as locus one group of pixel adjacent one another are, but is not limited thereto.Such as will be described below, can be aspect mathematics pixel groups be expressed as matrix or vector, and aspect information processing, pixel groups can be handled as 2-D array or 1-D array.

Term " picture element signal " and " pixel data " are meant above-mentioned pixel value.The statement of picture element signal is meant in side circuit the pixel value handled as the signal of telecommunication etc.In addition, pixel data has shown such fact, that is, pixel value information of being used as in computer etc. is handled when handling.

Term " image space " is meant by pixel being arranged on the space that constitutes in the 2-D compact manifold K in discrete mode.The concrete example of this image space comprises entire image, and as 8 * 8 block of pixels (its unit for will in the JPEG coding/decoding, handling) of the image subset that obtains by partitioned image.In addition, can adopt the minimum pixel module unit that in such as MPEG, MPEG 2, MPEG4, the coding/decoding that H.264 waits, to handle as another example.

Term " image space conversion " is meant the mapping from image space A to image space B, and it can be defined as discrete function f.In following examples, will mainly consider reversible graph image space conversion (that is the contrafunctional conversion of definable f).But the concrete example of image space conversion comprises the linearity inverse mapping, but and the concrete example of linearity inverse mapping comprise linear orthomorphism.Below the concrete mathematical notation of reversible Linear Mapping and linear orthomorphism will be described by example.In following example, respectively, when coding, before carrying out spatial frequency transforms, carry out this image space conversion (preliminary treatment), and when decoding, after inverse transformation, also carry out this image space conversion (reprocessing).

Term " spatial frequency transforms " is meant the mapping from the image space to the frequency space.

Term " orthogonal transform " is meant the conversion from the space A that defined inner product to the space B that has defined inner product, and inner product value keeps inconvenience under conversion.Example as the spatial frequency transforms of orthogonal transform comprises DCT (discrete cosine transform).In addition, the Hadamard conversion is another example as the spatial frequency transforms of orthogonal transform, and this Hadamard conversion can be expressed as the integer arithmetic computing.

Term " displacement " is meant in the interior replacement pixel of image space (as a specific example, being block of pixels), thereby becomes the interior function of image space.As mentioned above, be associated with the location of pixels of pixel by information and obtain pixel the pixel value of pixel.By pixel value in the termination image space and the former pairing between the location of pixels, and produce new pairing, realize displacement.For example, for pixel replacement, the pixel value of certain location of pixels is replaced by the pixel value of another location.Notice that the minimum pixel collection of replacing is called permute unit, it generally is equal to pixel itself.Utilize basic displacement to carry out the exchange of two permute units.

Other term will be described in following example, perhaps adopt general term definition.

Hereinafter with reference to accompanying drawing example of the present invention is described.

[first embodiment]

Fig. 1 to Figure 27 shows the first embodiment of the present invention, and Fig. 1 is the block diagram of the structure of expression expansion jpeg coder, and Figure 28 is the block diagram of the structure of the traditional jpeg coder of expression.

Note, in appropriate circumstances, will expand JPEG and be called JPEGext, JPEGe, JEX etc.

Should be noted that the present invention can be applicable to any in method for encoding images, picture decoding method, moving image compression method, moving image decompression method, image encoding program, picture coding device, picture decoding apparatus, encoding/decoding image system and the expanded images compression/decompression compression system.

This expansion jpeg coder comprises that the piece as classification apparatus forms unit 1, coding control unit 2, mark generation unit 3, row operation determination unit 4 as the spatial alternation device, line operate determining unit 5 as the spatial alternation device, head generation unit 6 as the header information adding set, orthogonal transform unit 7 as the spatial frequency transforms device, quantifying unit 8 as quantization device, quantization table generation unit 9 as quantization device, entropy coding unit 10, and the synthesis unit 13 that is used as compressed image file generation device and header information generation device.

Therefore, expansion jpeg coder shown in Figure 1 is suitable with traditional jpeg coder shown in Figure 28, but expansion jpeg coder shown in Figure 1 has added coding control unit 2, row operation determination unit 4, line operate determining unit 5 and head generation unit 6.In addition, traditional synthesis unit 13 shown in Figure 28 is used for composite coding data and mark, and the also synthetic JEX header information of synthesis unit 13 shown in Figure 1 and coded data and mark, as described below.

Entropy coding unit 10 also comprises zigzag scanning element 11 and Huffman coding unit 12.

Piece forms unit 1 and is used for input image data, and this view data is divided into the module unit that is made of 8 * 8 pixels.

Coding control unit 2 is used to read the relevant value of setting of compression level when encoding, and sends the required value of setting of row operation determination unit 4 and line operate determining unit 5 respectively.

Mark generation unit 3 is used for being created in the required mark of synthesis unit 13 composite documents.

But row operation determination unit 4 is utilized the value of setting of sending from coding control unit 2 to determine whether to tackle from piece and is formed the inverse operation that the pixel data array on the column direction of 8 * 8 block of pixels view data that read unit 1 is scheduled to, and carries out this operation should carry out this operation the time definite.

But line operate determining unit 5 utilizes the value of setting of sending from coding control unit 2 to determine whether the inverse operation that be scheduled to the pixel data array on the line direction of 8 * 8 block of pixels view data of sending from row operation determination unit 4, and carries out this operation should carry out this operation the time definite.

In above-mentioned processing sequence, after row operation determination unit 4 is carried out scheduled operation, the operation that line operate determining unit 5 is scheduled to, but as described below, with the operation of line correlation and the operation relevant with row is interchangeable, thereby can adopt such processing sequence, promptly, the operation determination unit of being expert at 5 is carried out after the scheduled operation, the operation that row operation determination unit 4 is scheduled to.

Head generation unit 6 is used to produce header information, this header information be will because the operation of row operation determination unit 4 and the variation that in image, produce and since the operation of line operate determining unit 5 and the variation that in image, produces to revert to initial condition required to carry out subsequent decoding.

7 pairs of 8 * 8 block of pixels data that send from line operate determining unit 5 of orthogonal transform unit are carried out level shift, so that the dynamic range of pixel data is the center with 0 (zero).Then, block of pixels is carried out DCT (discrete cosine transform).Note, carry out level shift and make the mean value of block of pixels data, thereby such advantage is provided, that is, when carrying out DCT, can reduce so-called DC coefficient near 0.After this conversion, orthogonal transform unit 7 is exported 64 groups of data with the form of 8 * 8 array block data once more.

Quantifying unit 8 is used for each element by 8 * 8 array block data that will send from orthogonal transform unit 7 divided by each elements of the quantization table data that are made of 8 * 8 identical array size, and ignore its fractional part, come to use different quantization widths to quantize to each pixel.This quantization table is the table that is made of 8 * 8 quantization parameters, is used for each element that has carried out in 8 * 8 of spatial frequency transforms is quantized.

Quantization table generation unit 9 produces will be by the quantization table data of quantifying unit 8 uses.As known in the art, the relative little data of 9 pairs of DC coefficients of quantization table generation unit with low frequency AC coefficient generation value, and to the big relatively data of high-frequency AC coefficient generation value.

Zigzag scanning element 11 is used in the zigzag mode high frequency side from the DC coefficient to the AC coefficient 8 * 8 array block data that send from quantifying unit 8 is scanned, and reads this data with the form of 64 groups of 1-D array datas.

Huffman coding unit 12 is used for carrying out variable length code with reference to predetermined Huffman coding schedule based on the scanning result of zigzag scanning element 11, with amount of compressed data.

Synthesis unit 13 is used for and will has been carried out the image encoded data by Huffman coding unit 12, synthesized an image file (having carried out the image file of JPEG extended coding) and exported this document by the header information of head generation unit 6 generations and the mark that is produced by mark generation unit 3.

Then, Fig. 2 is the block diagram of the structure of expression expansion jpeg decoder, and Figure 29 is the block diagram of the structure of the traditional jpeg decoder of expression.

This expansion jpeg decoder comprises the mark determining unit 21 as the header information reading device, division unit 22, entropy decoding unit 23 as the entropy decoding device, inverse quantization unit 26 as the re-quantization device, quantization table generation unit 27 as the re-quantization device, inverse orthogonal transformation unit 28 as contrary spatial frequency transforms device, head interpreter 29 as the additional header information readout device, decoding control unit 30, line operate unit 31 as the inverse spatial transform device, row operating unit 32 as the inverse spatial transform device, and the piece reproduction units 33 that is used as transcriber.

Like this, expansion jpeg decoder shown in Figure 2 is suitable with traditional jpeg decoder shown in Figure 29, but expansion jpeg decoder shown in Figure 2 has added head interpreter 29, decoding control unit 30, line operate unit 31, row operating unit 32.And traditional division unit 22 shown in Figure 29 is used for coded data is divided into and the corresponding a plurality of parts of block of pixels, but division unit 22 shown in Figure 2 also exports the JEX header information to head interpreter 29, as described below.

Entropy decoding unit 23 also comprises Huffman decoding unit 24 and contrary zigzag scanning element 25.

It is underlined that mark determining unit 21 is used for determining to be included in the institute of the image file that has carried out the JPEG extended coding.

Division unit 22 is used for extracting and the corresponding coded portion of above-mentioned 8 * 8 block of pixels according to the result that mark determining unit 21 is determined.

Huffman decoding unit 24 is used for reference to the Huffman coding schedule decoding from the decoded portion of division unit 22 outputs.

Contrary zigzag scanning element 25 is used for by the mode with zigzag the data of having been carried out decoding by Huffman decoding unit 24 being reproduced, and obtains 8 * 8 array block data.

Inverse quantization unit 26 is used for usually carrying out re-quantization by each unit that will multiply by the quantization table data that are made of 8 * 8 identical array size from each element of 8 * 8 array block data of contrary zigzag scanning element 25 outputs.

Quantization table generation unit 27 is used to produce the quantization table data that will be used by inverse quantization unit 26, and the quantization table that produces here is identical with the quantization table of quantization table generation unit 9 generations shown in Figure 1.

Inverse orthogonal transformation unit 28 is used for producing 8 * 8 block of pixels view data by 8 * 8 array block data of having been carried out re-quantization by inverse quantization unit 26 being carried out the inverse DCT as contrary spatial frequency transforms.

Head interpreter 29 is used for determining whether write down the relevant header information with expansion JPEG at head.

Decoding control unit 30 is used for realizing control by reading the required header information of recovery image, and be used for when head interpreter 29 determines to have write down header information according to expansion JPEG, to line operate unit 31 or row operating unit 32 these header informations of output.More particularly, write down to since the specific operation of row operation determination unit 4 and the variation that in image, produces recover under the situation of required header information, decoding control unit 30 is to row operating unit 32 these header informations of output, and writing down because the specific operation of line operate determining unit 5 and the variation that produces in image recover under the situation of required header information, decoding control unit 30 is to line operate unit 31 these header informations of output.

In above-mentioned processing sequence, after the operating unit 31 of being expert at is operated, row operating unit 32 is operated, but as mentioned above, with the operation of line correlation and the operation relevant with row is interchangeable, thereby can adopt such processing sequence, promptly, after row operating unit 32 was operated, line operate unit 31 was operated.

Piece reproduction units 33 is used for being reproduced as original one page view data with each piece having been carried out decoded image data.

Next, Fig. 3 is the flow chart of presentation code processing.Processing shown in Figure 3 can be to adopt the hardware type of the encoder with all structures as shown in Figure 1 to handle, but is not limited thereto, but can handle by adopt the software formula with execution coded programs such as computers.

When this handles beginning, at first pending view data is divided into some pieces (step S1).Fig. 7 represents that VGA image wherein is divided into the figure of the example of some pieces.In the example depicted in fig. 7, the VGA image that is made of 640 * 480 pixels is divided into a plurality of 8 * 8 block of pixels, thereby produces 80 * 60=4800 8 * 8 block of pixels altogether.

Fig. 8 is the figure of the dot structure of expression 8 * 8 block of pixels.For 8 * 8 block of pixels that produced, the row number by specifying the 0th to the 7th row and the row of the 0th to the 7th row number come remarked pixel coordinate (row number, row number).At this moment, the coordinate representation of pixel that is positioned at the upper left corner is for (0,0), and the coordinate representation that is positioned at the pixel in the upper right corner is (7,0), and the coordinate representation that is positioned at the pixel in the lower left corner is (0,7), and the coordinate representation that is positioned at the pixel in the lower right corner is (7,7).With coordinate (k, l) pixel data in be expressed as f (k, l).

Notice that although show the image that vertically is arranged with square or rectangular pixels on it with along continuous straight runs as example, the accessible image of present embodiment is not limited to these images.

Fig. 9 carries out the figure of the example of the image that 2-D shape reproduces for expression by filling the hexagon pixel.Even in image shown in Figure 9, make it possible to obtain as shown in figure 10 the pixel data that follows direction and arrange by getting as shown in Figure 9 location of pixels a to p, thereby make it possible to carry out various processing as described below along column direction.Pixel data among Fig. 9 follows direction to Figure 10 and the figure of the example of arranging along column direction for expression wherein is arranged on.

In addition, more generally, can adopt the pixel that is arranged on the curved surface, for example be arranged in the pixel on the insect compound eye, and each pixel needn't be of similar shape also.That is to say, can handle the pixel that is set to the 2-D compact manifold in discrete mode by widely distributed layout.

Return Fig. 3 and be described, next, carry out handling (step S2) as the expansion JPEG coding of describing in detail hereinafter with reference to Fig. 5 (JEX coding).Here, carry out the displacement of pixel data, make the high frequency coefficient that produces in the orthogonal transform of follow-up phase reduce.As mentioned above, this replacement Treatment is reversible, thereby when data are decoded, will never obliterated data.

Subsequently, utilize the orthogonal transform (step S3) of DCT.At first, as shown in the formula (k l) is expressed as f with pixel data f shown in 1 _KlHere, suppose pixel data f _KlBy being worth 0 to 2 ⁸8 bit data between-1 constitute.

[formula 1]

f _kl＝f(k，l) f _kl∈[0，…，2 ⁸-1]

By all arithmetical operation usage factor K as shown in Equation 2 with this pixel data f _KlTransform to the formula D in the frequency domain, carry out DCT.

[formula 2]

$D^{\mathrm{uv}}=\underset{k=0}{\overset{7}{\mathrm{\Σ}}}\underset{l=0}{\overset{7}{\mathrm{\Σ}}}{K}_k^u{f}_{\mathrm{kl}}{K}_l^v$

Here, in the appended suffix of each variable, subscript is represented frequency, and subscript is represented the space.

In addition, be used for the factor K of orthogonal transform by representing with following formula 3 and formula 4.

[formula 3]

$K_k^u=\frac{1}{2}C\left(u\right)\·\cos \frac{(2k+1)\mathrm{u\π}}{16}$

$C\left(u\right)=\left\{\begin{array}{cc}\frac{1}{\sqrt{2}}& (u=0)\\ 1& (u>0)\end{array}\right.$

[formula 4]

$K_l^v=\frac{1}{2}C\left(v\right)\·\cos \frac{(2l+1)\mathrm{v\π}}{16}$

$C\left(v\right)=\left\{\begin{array}{cc}\frac{1}{\sqrt{2}}& (v=0)\\ 1& (v>0)\end{array}\right.$

Note, can formula 2 be expressed as following matrix operation by adopting matrix expression.

[formula 5]

D＝KfK ^T

Here, the normal matrix transpose of appended subscript [T] expression K on the right of last K on right side.

More particularly, can utilize angle in the cosine function become 0 the fact in the following manner (formula 6) come design factor D ⁰⁰

[formula 6]

${\mathrm{<figure-callout\; id="00"\; label="D"\; filenames="C2005800221330102H1.png"\; state="\{\{state\}\}">D</figure-callout>}}^{00}=\underset{k=0}{\overset{7}{\mathrm{\&Sigma;}}}\underset{l=0}{\overset{7}{\mathrm{\&Sigma;}}}{K}_k^0{f}_{\mathrm{<figure-callout\; id="0"\; label="kl\; K\; l"\; filenames="C2005800221330089H1.png,C2005800221330093H1.png"\; state="\{\{state\}\}">kl</figure-callout>}}{K}_l^{}{}_0^{}=\frac{1}{8}\underset{k=0}{\overset{7}{\mathrm{\&Sigma;}}}\underset{l=0}{\overset{7}{\mathrm{\&Sigma;}}}{f}_{\mathrm{kl}}$

Therefore, can multiply by 8 by mean value and obtain coefficient D the view data in 8 * 8 block of pixels ⁰⁰

For D ⁰⁰Coefficient in addition, non-vanishing usually in the cosine function with the corresponding angle item of frequency, thus these coefficients are with the extraction in this 8 * 8 block of pixels changes corresponding at each frequency component.More particularly, for example, by following formula 7 design factor D ¹⁰

[formula 7]

${\mathrm{<figure-callout\; id="10"\; label="D"\; filenames="C2005800221330100H1.png,C2005800221330100H4.png"\; state="\{\{state\}\}">D</figure-callout>}}^{10}=\underset{k=0}{\overset{7}{\mathrm{\&Sigma;}}}\underset{l=0}{\overset{7}{\mathrm{\&Sigma;}}}{K}_k^1{f}_{\mathrm{<figure-callout\; id="0"\; label="kl\; K\; l"\; filenames="C2005800221330089H1.png,C2005800221330093H1.png"\; state="\{\{state\}\}">kl</figure-callout>}}{K}_l^{}{}_0^{}=\frac{1}{4\sqrt{2}}\&CenterDot;\underset{k=0}{\overset{7}{\mathrm{\&Sigma;}}}\cos \frac{(2k+1)\mathrm{\&pi;}}{16}\underset{l=0}{\overset{7}{\mathrm{\&Sigma;}}}{f}_{\mathrm{kl}}=\stackrel{\&RightArrow;}{V}\&CenterDot;\stackrel{\&RightArrow;}{W}$

Here, each vectorial V and W respectively in the expression 7 with the lower part.

[formula 8]

$\stackrel{\&RightArrow;}{V}=({\mathrm{<figure-callout\; id="0"\; label="V"\; filenames="C2005800221330089H1.png,C2005800221330093H1.png"\; state="\{\{state\}\}">V</figure-callout>}}_0,...,V_7),$ $V_k=\frac{1}{4\sqrt{2}}\cos \frac{(2k+1)\mathrm{\&pi;}}{16}$

[formula 9]

$\stackrel{\&RightArrow;}{W}=({\mathrm{<figure-callout\; id="0"\; label="W"\; filenames="C2005800221330089H1.png,C2005800221330093H1.png"\; state="\{\{state\}\}">W</figure-callout>}}_0,...,W_7),$ $W_k=\underset{l=0}{\overset{7}{\mathrm{\&Sigma;}}}{f}_{\mathrm{kl}}$

As intelligible with reference to these formulas 8 and formula 9, the k element of vectorial W is the summation of the pixel data of k row.In addition, the k element of vectorial V is the discrete function of strictly monotone decreasing.Like this, coefficient D ¹⁰Can be expressed as inner product of vectors, and by similarly deriving coefficient D ⁰¹(its expression formula clearly is not shown) also can be expressed as inner product of vectors.

Can calculate from D by carrying out this arithmetical operation successively ⁰⁰To D ⁷⁷64 DCT coefficients.

Get back to Fig. 3, after orthogonal transform, the DCT coefficient is quantized (step S4).By with described 64 DCT coefficients respectively divided by different coefficients (Q coefficient), and ignore its fractional part, carry out this quantification.Can adopt the value of selecting in advance as the coefficient that is used for this quantification, and need to prepare one group of desirable coefficient as quantization table.Usually, such as D ⁰⁰The DC coefficient and such as D ¹⁰, D ⁰¹Relative low frequency AC coefficient comprise summary with image-related information, thereby thereby need quantize as far as possible accurately to store this information by using relatively little Q coefficient.On the other hand, even ignore such as D ⁷⁷High-frequency AC information, these information also only produce limited influence to the visual effect of entire image.Therefore use big relatively Q coefficient that they are quantized.

By 77 scanning from component 00 to component, the DCT quantization parameter can be converted to 64 (step S5) in the 1-D array data in the mode of zigzag.

Subsequently, utilize so-called Huffman coding to carry out variable length code (step S6).The Huffman coding has the sign indicating number of short bit length by making the value with higher frequency of occurrences, and the value of the low frequency of occurrences has the sign indicating number of long bit length, comes packed data, but some complexity of its method, and not directly related, thereby omit its details with the first embodiment of the present invention.Except this describe, in general terms, also to point out, the DC coefficient is to encode by different modes with the AC coefficient, and continuously zero AC coefficient is got and made run length (run length), and the AC coefficient coding after zero be the class value of any class in 10 classes of classifying respectively continuously.

Carried out coded data by this processing and become JPEG extended code data, synthesis unit 13 forms file with these JPEG extended code data with mark and header information.Notice that above-mentioned quantization table, DC Huffman coding schedule, AC Huffman coding schedule etc. are recorded in the head and header information of encoding according to JEX.

Fig. 4 is the flow chart of expression decoding processing.Processing shown in Figure 4 can be to utilize the hardware decoders formula with all structures as shown in Figure 2 to handle, and perhaps can wait the execution decoding program to adopt the processing of software formula by using a computer.

When this handles beginning, under the help of mark, pending JPEG extended code is divided into and the corresponding sign indicating number of the pixel data of 8 * 8 block of pixels part, and after this, the Huffman coding schedule of reference record in head carries out Huffman decoding (step S11) to each yard part.

Subsequently, 64 1-D array coefficients are carried out contrary zigzag scanning, it is transformed to 8 * 8 arrays (step S12) of coefficient.

Subsequently, the quantization table of reference record in head carries out re-quantization (step S13).

Subsequently, by carrying out inverse DCT (IDCT) coefficient of frequency is transformed to pixel value (step S14) in the image space.At this moment, carry out contrary electrical level shift processing, in this contrary electrical level shift processing, being that the positive and negative view data at center is transformed to 0 to (2 as 0 of dynamic range center ⁸-1) the 8 positive bit data between.

Next, reference marker determines that whether pending image file is for having carried out JEX image encoded file (step S15).

Here, determine that pending file is to have carried out carrying out the JEX decoding processing (step S16) of following detailed description under the situation of JEX image encoded file.

Under the situation of the processing in finishing this step S16, determine in step S15 that perhaps pending file is not carry out under the situation of common jpeg image file of JEX coding, handle, 8 * 8 block of pixels view data are reproduced the precalculated position (step S 17) to the raw image files.After 8 * 8 all block of pixels having been carried out this piece reproduction, recovered one page image.

Fig. 5 is the flow chart of expression JEX encoding process.This Fig. 5 shows the details of the processing that will carry out in the step S2 of Fig. 3.

When this handles beginning, pixel data f (k, l) (the step S21) of 8 * 8 block of pixels that input is divided in step S1.As described at the step S1 of Fig. 3,, a sub-picture carries out this processing by being divided into a plurality of.Here, the n block of pixels in the block of pixels of being divided is carried out this processing, but should be appreciated that, change this n value by order all block of pixels are carried out this processing.

Then, in every row (changing on the direction of row number) to pixel data f (k, l) carry out addition, to produce the sum data sc (i) of each i row, as shown in Equation 10, similarly, in every row (changing on the direction of row number) to pixel data f (k, l) carry out addition, to produce the capable sum data sr of each j (j), as shown in Equation 11.

[formula 10]

$\mathrm{sc}\left(i\right)=\underset{l=0}{\overset{7}{\mathrm{\&Sigma;}}}f(i,l)$

[formula 11]

$\mathrm{sr}\left(j\right)=\underset{k=0}{\overset{7}{\mathrm{\&Sigma;}}}f(k,j)$

Subsequently, calculate the arrangement nc (i) be used for the i that 8 groups of sum data sc (i) sorted with descending, and the arrangement nr (j) (step S23) that is used for the j that 8 groups of sum data sr (j) sorted with descending.More particularly, for example, data array sc (0) is set in the zone on the memory of computer to sc (7) and the array that is worth 0 to 7.Data array sc (0) is applied in any one in the known sort algorithm in the programming to sc (7), can sorts to these data according to descending.When the array by identical mode logarithm 0 to 7 carries out data array sc (0) when this ordering of sc (7) is handled, the number that is write down will produce the arrangement nc (i) of required i.Also can obtain the arrangement nr (j) of j by carrying out identical processing.

Note,, can adopt ascending order to arrange here although with descending sum data sc (i) and sum data sr (j) are sorted.In addition, can adopt any ordering, as long as this ordering can reduce the frequency of image, and no matter be ascending order or descending.

Subsequently, determine whether to be used for columns to 8 * 8 block of pixels according to the processing of sorting.More particularly, utilize threshold value Thc to determine whether set up with following formula 12,

[formula 12]

sc(nc(0))-sc(nc(7))≥Thc×8

And utilize threshold value Nc (typical example value is 4) to determine whether set up (step S24) with following formula 13.

[formula 13]

Chc≥Nc

The arrangement nc (i) of i is used for the arrangement of sum data sc (i) being sorted with descending, thus sc (nc (0)) always more than or equal to sc (nc (7)), that is, the left side of formula 12 always is 0 or just.The left side of this formula 12 becomes an index, when this exponential representation becomes another row at row, and the maximum inhomogeneities of the variation of the summation of the picture element signal in the row.Therefore, under the less situation in the left side of this formula 12, there is inhomogeneities hardly in the signal between the different lines (summation in the different lines), thus the exchange between determining to need not to be listed as.

In addition, for example, as shown in the formula determining threshold value Thc shown in 14.

[formula 14]

Thc＝2 ^n-3

Thr＝Thc

Here, n represents the place value of the dynamic range of represent pixel data, thereby for example under the situation of 8 pixel datas, n=8.In addition, the number that will deduct from n is 3 here, but this is because block of pixels is made of 8 * 8 pixels, thereby is 8 pixels along the pixel line of column direction, promptly 2 ³Individual pixel.Therefore under the situation that for example adopts 16 * 16 block of pixels, become 4 from the number that n deducts.Yet this has described the example of the method that is used to obtain threshold value Thc, and this method is not limited thereto.

Notice that to the threshold value Thr that will adopt in step S28 described below, adopt the value identical with threshold value Thc usually, shown in this formula 14, but this also is an example, can adopt the value different with threshold value Thc.For example,, under the situation that has correlation on the specific direction, it is contemplated that to be provided with differently with threshold value Thr threshold value Thc at image.

In addition, all block of pixels are all adopted identical threshold value Thc or identical threshold value Thr here, but the threshold value that will adopt is not limited thereto here.On the contrary, can adopt different threshold values to each block of pixels.

Subsequently, use following formula 15 to calculate decision value Chc (and the decision value Chr that describes after a while).

[formula 15]

$\mathrm{Chc}=\sqrt[w]{\underset{i=0}{\overset{7}{\mathrm{\&Sigma;}}}{|\mathrm{nc}\left(i\right)-i|}^W}$

$\mathrm{Chr}=\sqrt[w]{\underset{i=0}{\overset{7}{\mathrm{\&Sigma;}}}{|\mathrm{nr}\left(i\right)-i|}^W}$

This decision value Chc (and decision value Chr) can be considered to replace the index of validity degree.For example, under the situation of decision value Chc=0, the added pixels value has made to change when being listed as from left to right and has diminished in row (changing on the direction of row number), thereby does not need the exchange that is listed as.Also deducibility, decision value Chc is big more, and then the quantity of the row that will exchange is big more, and perhaps the difference between switching purpose ground and the exchange source is big more.

Threshold value Nc represents the lower limit of validity degree.For example, supposed to obtain arrangement nc (i) as shown in figure 11.Figure 11 calculates for expression will be at the chart of the required element of the decision value Chc that arranges nc (i) acquisition.Shown in this accompanying drawing, under situation about moving, obtain bigger value to farther row.Be set to 1 weight w in use and calculate under the situation of decision value Chc, obtain following formula.

Chc＝4+1+1+0+4+1+1+2＝14

Notice that the value of decision value Chc changes according to the value of this weight w.More particularly, as shown in figure 12, when weight w is set to 1, two types nc (i) can obtain identical decision value Chc=8, and when weight w was set to 4, a decision value Chc became about 4.8, and that another decision value Chc becomes is about 1.7, thereby difference occurred.The chart how Figure 12 changes when weight w changes for the expression decision value.Therefore, under the situation of w=1, row between the sum of handling the row that will exchange on the identical basis and switching purpose ground and exchange source number poor.On the other hand, when weight w is set to bigger value, compare with the sum of the row that will exchange, row between processing and exchanging destination and the exchange source number is poor more emphatically.Therefore, when weight w was set to bigger value, the displacement of the row of apart from each other was compared and will be caused bigger decision value Chc with the displacement between nearer row only.The difference of row between switching purpose ground and the exchange source number is big more, then because the anamorphose that the exchange of row causes is big more, thereby increase weight w with more focus on anamorphose processing determine corresponding.This weight w can be provided with by the compression level that coding control unit 2 is provided with as required.If these weights for example are set to " 1 ", then provide following advantage: simplified arithmetical operation, handled load thereby reduced.

Note,, only determined the lower limit of two decision values, also need further to carry out determining the upper limit by step S24.That is to say that when above-mentioned two decision values became too big, the image in the block of pixels will greatly distortion after displacement.In this case, if use traditional jpeg decoder to recover the JEX file, then image will almost only reproduce on the piece rank.Therefore, under the situation that focuses on the general compatibility of decoding, preferably provide the upper limit with traditional JPEG.Provide this upper limit to make it possible to when recovering the JEX file, can reproduce to a certain extent image in the block of pixels by traditional jpeg decoder.

In step S24, satisfying under the situation of this condition, use 1 (displacement to row has been carried out in its expression) to come alternate data Pc (step S25).These data Pc will be recorded in the head of image file the data as header information after being.

Subsequently, actual f (k, column permutation l) (step S26) of carrying out.Here, with f (k, data l) are recalled (retreat) in another storage area etc., (nc (k), data l) are written to f, and (k is in Data Position l) with f then.Replace by carrying out this processing repeatedly, thereby make the data array of 8 * 8 block of pixels become

f(nc(0)，1)，f(nc(1)，1)，...，f(nc(7)，1)

This displacement is the concrete example of linear orthomorphism as described below.

At first, the pixel data f (0,0) of as shown in Figure 88 * 8 block of pixels is expressed as 8 * 8 matrix M shown in the formula 16 to f (7,7).

[formula 16]

Notice that be expressed as 8 * 8 square matrixes although this 8 * 8 block of pixels has in statu quo been carried out the nature statement, matrix notation is not limited thereto.For example, this 8 * 8 block of pixels can be expressed as the non-square matrix M ' of 4 * 16 shown in the formula 17.

[formula 17]

$M^{\&prime;}=\left(\begin{array}{cccccccc}f\left(\mathrm{0,0}\right)& f\left(\mathrm{1,0}\right)& \&CenterDot;\&CenterDot;\&CenterDot;& f\left(\mathrm{7,0}\right)& f\left(\mathrm{0,1}\right)& f\left(\mathrm{1,1}\right)& \&CenterDot;\&CenterDot;\&CenterDot;& f\left(\mathrm{7,1}\right)\\ f\left(\mathrm{0,2}\right)& f\left(\mathrm{1,2}\right)& \&CenterDot;\&CenterDot;\&CenterDot;& f\left(\mathrm{7,2}\right)& f\left(\mathrm{0,3}\right)& f\left(\mathrm{1,3}\right)& \&CenterDot;\&CenterDot;\&CenterDot;& f\left(\mathrm{7,3}\right)\\ f\left(\mathrm{0,4}\right)& f\left(\mathrm{1,4}\right)& \&CenterDot;\&CenterDot;\&CenterDot;& f\left(\mathrm{7,4}\right)& f\left(\mathrm{0,5}\right)& f\left(\mathrm{1,5}\right)& \&CenterDot;\&CenterDot;\&CenterDot;& f\left(\mathrm{7,5}\right)\\ f\left(\mathrm{0,6}\right)& f\left(\mathrm{1,6}\right)& \&CenterDot;\&CenterDot;\&CenterDot;& f\left(\mathrm{7,6}\right)& f\left(\mathrm{0,7}\right)& f\left(\mathrm{1,7}\right)& \&CenterDot;\&CenterDot;\&CenterDot;& f\left(\mathrm{7,7}\right)\end{array}\right)$

In the matrix notation shown in this formula 17 is the expression that is used for pixel data is divided into even number line and odd-numbered line, and when a processing one frame was divided into the horizontally interlaced image data of even field and odd number fields, this matrix notation was favourable.In other words, neighbor should preferably have high correlation, so that compressed image efficiently in the method.Therefore, can expect, can realize more effective digital coding by the pixel data that is assumed that each territory with high correlation is handled.The following matrix manipulation that carries out the non-square matrix of 4 * 16 shown in the formula 17:, and take advantage of 16 * 16 matrixes to carry out the row arithmetical operation by the right side by the capable arithmetical operation of premultiplication 4 * 4 matrixes.Carrying out this matrix notation makes it possible to carry out rank transformation between row that 4 pixels that are made of odd-numbered line (odd number row) are formed and the row be made up of 4 pixels of even number line (even rows) formation.

In addition, the example of another matrix notation comprises as shown in the formula the matrix M shown in 18 ".

[formula 18]

M″＝(f(0，0)f(1，0)…f(7，0)f(0，1)…f(7，1)f(0，2)…f(7，2)…f(7，7))

In the matrix notation shown in this formula 18 is the matrix with 1 row, 64 row, that is, and and the 64-D vector.Adopt such expression that this vector is carried out the huge matrix operation that the matrix arithmetical operation causes carrying out 64 row, 64 row, this has caused arithmetical operation to handle the shortcoming that becomes heavy and consuming time.On the other hand, the feasible arithmetical operation to pixel of this matrix operation has the highest degree of freedom.

Any invertible matrix computing of matrix M being carried out as shown in the formula the expression shown in 19.

[formula 19]

M→A·M·B

Here, A and B are respectively the matrix with inverse matrix.Carrying out under the situation of arithmetical operation along contrary direction, inverse matrix that can premultiplication A, and can right take advantage of the inverse matrix of B.Note,, can use this matrix operation no matter matrix M is square formation or non-square matrix.

For the block of pixels of view data is carried out this matrix operation shown in the formula 19, particularly, carry out this matrix operation in reliable mode, as the preliminary treatment of JPEG, the matrix component after the matrix operation must be in the dynamic range [0 to 2 of pixel value ⁸-1] in.This condition is summarised as adequate condition below with matrix A and the reversible condition of matrix B as necessary condition.

At first, for matrix A,

(A1) matrix A has inverse matrix (reversible).

(A2) each element a of matrix A satisfies 1 〉=a 〉=0.

(A3) each element a of matrix A follows the satisfied 1 〉=Suma of summation Suma 〉=0 of direction.

Next, for matrix B,

(B1) matrix B has inverse matrix (reversible).

(B2) each element b of matrix B satisfies 1 〉=b 〉=0.

(B3) each element b of matrix B satisfies 1 〉=Sumb 〉=0 along the summation Sumb of column direction.

For condition A3 and condition B3, described summation is more little, and promptly described summation approaches 0 more away from 1 more, and then the signal value of whole block of pixels is suppressed manyly more, thereby has reduced the difference between the signal,, has strengthened the effect of low-pass filtering (LPF) that is.Yet, when suppressing the signal value of whole block of pixels, also whole frequency component has been suppressed the DCT coefficient, thereby the error that causes owing to quantization step increases.

Under the situation of JPEG, for higher frequency coefficients, quantization step is bigger, and therefore, for higher frequency components, the influence that causes owing to this quantization error increases.(referring to the description of Japanese unexamined patent publication No. communique No.2000-308058 described below to quantization error.Note, for convenience of explanation, describe this quantization error as example, but can be used for understanding the increase of the quantization error that the minimizing owing to the common element value of matrix A and B causes about the conclusion of quantization error) with the simple division arithmetic of block of pixels.Therefore, the summation Sumb shown in summation Suma shown in the condition A3 and the condition B3 preferably is set to as far as possible the value near 1 respectively.

In addition, matrix element a among condition A2 and the B2 and the value of b are more little, and then the arithmetic error owing to the result that causes of Code And Decode process of series will be big more.Although this is reversible because of matrix A and matrix B, actual picture element signal is digitized.More particularly, because matrix element a and b be near 0, so the element of inverse matrix will have and 1/a or the suitable value of 1/b (these values are all greater than 1), that is, the element of inverse matrix increases.Therefore, matrix element a or b become more little, and be then big more owing to the matrix operation result being rounded to the arithmetic error that immediate integer causes after carrying out the inverse matrix computing.

Owing to this reason, can draw such conclusion, promptly, preferably be set to as far as possible value as the summation in the row of one in the matrix of operator and the summation in the delegation near 1, and when matrix element a and b itself are set to as far as possible near 1 value, improved the precision of digital arithmetical operation.

Some examples that satisfy such as the matrix of the condition of above-mentioned condition A1 to A3 or condition B1 to B3 can be shown.First is exemplified as permutation matrix, and second all elements that is exemplified as except diagonal entry lower-left triangular array part all is 0 matrix (following will provide more definite description).To carry out concise and to the point description to these two examples.

For first example, in permutation matrix, each element all is set to 0 or 1 value, and the summation perseverance of element of each row is 1, and the summation of the element of each row also perseverance be 1.The example of typical permutation matrix has been shown in following formula 20.

[formula 20]

$P=\left(\begin{array}{cccccccc}1& 0& 0& 0& 0& 0& 0& 0\\ 0& 0& 1& 0& 0& 0& 0& 0\\ 0& 1& 0& 0& 0& 0& 0& 0\\ 0& 0& 0& 0& 0& 0& 1& 0\\ 0& 0& 0& 1& 0& 0& 0& 0\\ 0& 0& 0& 0& 0& 0& 0& 1\\ 0& 0& 0& 0& 0& 1& 0& 0\\ 0& 0& 0& 0& 1& 0& 0& 0\end{array}\right)$

Therefore, the linear inverible transform for adopting this permutation matrix can draw such conclusion, that is, and and above-mentioned LPF effect and the quantization error minimum that occurs thus.Because having for each row and each, this permutation matrix is listed as the structure that all has single nonzero coefficient 1, so when the actual matrix that carries out the remarked pixel piece multiply by the arithmetical operation of this permutation matrix, should carry out this matrix operation, as the exchange of row in the matrix of remarked pixel piece and row.Notice that this permutation matrix is the pretreated matrix of DCT that is used as this first embodiment, below will describe in further detail.

Then, the matrix of second example is as follows.This matrix all is 0 matrix for all elements partly of lower-left triangular array except diagonal entry, the element of upper right gable (comprising diagonal entry) be selected as satisfying condition A2 and condition A3, and diagonal entry non-zero all.This matrix has been shown in the following formula 21.

[formula 21]

Here, a _i Satisfy 1 〉=a _i＞0 (i=0 ..., 7), b _IjSatisfy 1＞b _Ij〉=0 (i=1 ..., 7, j=..., 6, j＜i), in addition, respectively, under the situation of matrix B, element satisfies formula 22, and under the situation of matrix A, each element satisfies formula 23 respectively.

[formula 22]

$0<a_i+\underset{j<i}{\mathrm{\&Sigma;}}{b}_{\mathrm{ij}}\&le;1$

[formula 23]

$0<a_i+\underset{i>l}{\mathrm{\&Sigma;}}{b}_{\mathrm{il}}\&le;1$

The concrete example of this matrix is shown in the following formula 24.

[formula 24]

Be well known that, for diagonal entry all the element of non-zero and lower-left triangular array part all be zero matrix, its determinant is the product of all diagonal entries, that is, the determinant non-zero, thus this matrix is reversible.

Because this matrix is reversible and has the function that reduces high frequency coefficient usually, thereby can draw such conclusion, that is, it has reversible LPF function.When the spatial alternation that the matrix M of expression block of pixels carried out based on this matrix, must actually carry out the matrix operation represented on the mathematics, thereby preferably in integrated circuit (IC), carry out this matrix operation, so that can carry out the arithmetic processing of high speed.In addition, this matrix is carried out under the situation of computing combining with above-mentioned permutation matrix, carrying out this computing as matrix operation is effective and efficient manner.

Then, matrix A shown in the formula 19 or matrix B have inverse matrix, thereby matrix A or matrix B have feature shown below.

That is to say, on mathematics, prove, all by having made up as shown in the formula the basic matrix F shown in 25 _KlAnd all as shown in Equation 26 basic matrix F _kProduct (α) can be with any matrix notation with inverse matrix

A＝...F _kl...F _j(α)...F _mn...F _i(β)...

(for example referring to Max Koecher, " Linear Algebra und analytische Geometrie ", Springer Verlag (Berlin), 1985).

[formula 25]

[formula 26]

Note the F shown in the formula 25 _KlIn off-diagonal component 1 be the element (noting, in two suffix, capable number of left side postfix notation, right side postfix notation row number, therefore should expression opposite) of capable the 1st row of k with other matrix postfix notation partly, the F shown in the formula 26 _kDiagonal components α (α) is the element of the capable k row of k.In addition, F _k(α) must have inverse matrix, thereby α is a non-zero real.

To the basic matrix F shown in the right multiplier 25 of matrix M _Kl(that is MF, _Kl) produced such arithmetical operation, that is, wherein the element of the 1st of matrix M the row is added to respectively on the corresponding element of k row of matrix M, and with its result as the 1st new row.

In addition, by to matrix M premultiplication basic matrix F _Kl(that is F, _KlM) produced such arithmetical operation, that is, wherein the element that the k of matrix M is capable is added to respectively on the 1st corresponding element of going of matrix M, and the result who is obtained is capable as new k.

On the other hand, to the basic matrix F shown in the right multiplier 26 of matrix M _k(α) (that is MF, _k(α)) produced such arithmetical operation, that is, wherein each element of the k of matrix M row all multiply by α, and the result who is obtained is listed as new k.

In addition, to the basic matrix F shown in the matrix M premultiplication formula 26 _k(α) (that is F, _k(α) M) produced such arithmetical operation, that is, wherein each element that the k of matrix M is capable all multiply by α, and the result who is obtained is capable as new k.

In following formula 27, adopt basic matrix F _KlAnd F _k(α) basic matrix F is represented in combination Ben Shen _KlAnd F _kInverse matrix (α).

[formula 27]

F _kl ^-1＝f _l(-1)F _klF _l(-1)，F _klF _kl ^-1＝E

F _l(α) ^-1＝F _l(α ^-1)，F _l(α)F _l(α ^-1)＝E

Here, E is as shown in the formula the unit matrix shown in 28.

[formula 28]

And the matrix operation shown in the formula 19 is linear operation.That is to say, if matrix M is expressed as matrix M ₁And matrix M ₂And, then following formula 29 is set up,

[formula 29]

A·M·B＝A·(M ₁+M ₂)·B＝A·M ₁·B+A·M ₂·B

Linear relationship is satisfied in its expression.Therefore, but the matrix operation shown in the formula 19 is the linearity inverse operation, and is adopting this computing to change under the situation of image space distribution, but this computing can be called as the linearity inverse mapping.In other words, we can say, pixel is being expressed as under the situation of matrix, but the linearity inverse mapping is by this matrix being carried out the image space conversion that reversible linear matrix computing obtains.

Therefore, but the linearity inverse mapping is by continuously matrix M being carried out the mapping that following computing obtains:

1. second row or column is added to the arithmetical operation of first row or column.

2. delegation or is listed as the arithmetical operation of multiply by nonzero coefficient.

Notice that arrangement (comb) linear reversible graph image space conversion as shown in Equation 19 and DCT have as shown in Equation 5 produced as shown in the formula the matrix notation shown in 30.

[formula 30]

M→KAMBK ^T

On the other hand, be nonlinear operation with the corresponding matrix operation of level shift.Level shift is such arithmetical operation, and wherein with the matrix M and the matrix A addition of pixel data, the element of this matrix A is provided by the level shift amount.Shown in 31,

[formula 31]

M+A＝(M ₁+M ₂)+A＝(M ₁+A)+(M ₂+A)-A

Do not satisfy linear relationship.

In addition, before being used for carrying out DCT, the technology of describing in the background technology of Japanese Unexamined Patent Application communique No.2000-308058 carries out as shown in the formula the conversion shown in 32,

[formula 32]

M→(1/2)·M+128N

Here, N is that all elements all is 1 matrix.This conversion is used for pixel value is compressed to half of dynamic range, thereby pixel value is converted to intermediate value in the original dynamic range.Therefore, this conversion comprises the level shift suitable with above-mentioned level shift, and proof does not satisfy linear relationship easily.This conversion is made up with the DCT that will carry out subsequently, whole map table can be shown as shown in the formula shown in 33.

[formula 33]

$M\&RightArrow;\frac{1}{2}{\mathrm{KMK}}^T+1024{\stackrel{\&OverBar;}{D}}^{00}$

Here, as the matrix D on second on right side ⁰⁰(replacing representation) is to have only (0,0) coefficient of DCT to become the matrix of value 1, that is, in brief, wherein only important (0,0) is 1, and component in addition is 0 matrix.

Notice that the technology of describing is such conversion in this Japanese Unexamined Patent Application communique No.2000-308058, wherein, not only compare, and compare that quantization error all can increase usually with traditional JPEG with the JEX that describes among this first embodiment.

That is to say, if be w with the absolute value representation of the DCT coefficient among traditional JPEG, shown in 34,

[formula 34]

w＝|(KMK ^T) ^uv|

Then the quantization error Δ 1 after quantification and the re-quantization becomes following formula 35:

[formula 35]

${\mathrm{\&Delta;}}_1=\left[\frac{w}{q}\right]\&CenterDot;q$

Here, q is that (u, v), it represents quantization width to q=Q.

On the other hand, easy proof after utilizing technology described in the Japanese Unexamined Patent Application communique No.2000-308058 to handle, is carried out DCT, and when quantizing with re-quantization subsequently, quantization error Δ 2 becomes following formula 36.

[formula 36]

${\mathrm{\&Delta;}}_2=\left[\frac{w}{2q}\right]\&CenterDot;2q$

Can be with the absolute value w of DCT coefficient divided by 2q, and obtain discussing the ζ (ζ is the integer that satisfies ζ 〉=0) and first remainder, and this first remainder can also be further divided by q, obtain discussing the ξ (ξ=0 or 1) and the second remainder η (η is for satisfying the integer of q＞η 〉=0), that is, the absolute value w of DCT coefficient can decompose as shown in the formula shown in 37

[formula 37]

w＝ζ·2q+ξ·q+η

Therefore, if as shown in the formula calculating poor between Δ 1 and the Δ 2 shown in 38,

[formula 38]

Δ1-Δ2＝[w/q]·q-[w/(2q)]·2q

＝(2ζ+ξ)·q-ζ·2q＝ξ·q≥0

Be appreciated that then quantization error is 0 or q (quantization width), therefore, Δ 1 and Δ 2 are equal to each other, and perhaps the difference between Δ 1 and the Δ 2 increases.

Then, but as the concrete example of aforesaid linearity inverse mapping, consider linear orthomorphism.This linear orthomorphism be can be by bind lines in block of pixels exchange and the conversion that realizes of the exchange of row.Therefore, any linear permutation representation can the be served as reasons product of the basic displacement that the exchange of capable exchange or row constitutes.For conversion, can find pixel value in the block of pixels in the specific location in block of pixels after the conversion based on this linear orthomorphism.Therefore, even the mean value of pixel value in the block of pixels can not change after conversion yet, thereby linear orthomorphism is to the not influence of statistic such as the frequency of occurrences of the pixel with particular pixel values etc.

But this linear orthomorphism belongs to the type of above-mentioned linearity inverse mapping, thereby can be expressed as the product of above-mentioned basic matrix.Specifically, consider basic matrix F _k(α), can use F by substitution α=-1 _k(1).

Then, shown in Figure 13 as the example of representing matrix M, consider to k row with value a with have value b the 1st be listed as the basic displacement that exchanges.According to order shown in Figure 14 successively to this matrix application basic matrix.Figure 14 is the chart of the application order of the basic matrix of the basic displacement of expression between being used to realize being listed as.

At first, under first state, only choose above-mentioned predetermined value partly produced expression formula (a, b).

Then, in first step, if from the right side to matrix M as fired basis matrix F _k(1), then obtain (a ,-b).

In second step, if from the right side to matrix M as fired basis matrix F _Kl, then obtain (a, a-b).

In third step, if from the right side to matrix M as fired basis matrix F _l(1), then obtain (a, b-a).

In the 4th step, if from the right side to matrix M as fired basis matrix F _Lk, then obtain (b, b-a).

In the 5th step, if from the right side to matrix M as fired basis matrix F _l(1), then obtain (b, a-b).

In the 6th step, if from the right side to matrix M as fired basis matrix F _Kl, then obtain (b, a), and the basic displacement between therefore having carried out being listed as.

Therefore, can adopt the basic matrix product represent as shown in the formula such shown in 39 and be listed as between the corresponding basic displacement P of exchange _Kl

[formula 39]

$P_{\mathrm{kl}}^c=F_l(-1)F_{\mathrm{kl}}{F}_l(-1)F_{\mathrm{lk}}{F}_l(-1)F_{\mathrm{kl}}$

Note, be attached to this and replace P substantially _KlThe upper right corner on suffix " c " expression for the displacement of row.

If should replace P substantially _KlBe expressed as concrete matrix, then it is expressed as following formula 40 expressions.

[formula 40]

Here, in the diagonal component, the component with the value beyond 1 comprise component (k, k) and component (l, l), the value of these two components is respectively 0, yet in the off-diagonal component, the component with the value beyond 0 comprises component (k, l) and component (l, k), their value is respectively 1.

By as shown in Equation 41 basic displacement P being taken advantage of on the matrix M right side _Kl, cause the exchange between k row and the 1st row.

[formula 41]

$M\&RightArrow;M\&CenterDot;P_{\mathrm{kl}}^c$

Similarly, adopt the product of basic matrix as shown in the formula the corresponding basic displacement P of exchange between expression shown in 42 and the row _Kl

[formula 42]

$P_{\mathrm{kl}}^r=F_{\mathrm{lk}}{F}_l(-1)F_{\mathrm{kl}}{F}_l(-1)F_{\mathrm{lk}}{F}_l(-1)$

Note, be attached to this and replace P substantially _KlThe upper right corner on suffix " r " expression to the displacement of row.

By as shown in Equation 43 the matrix M premultiplication being replaced P substantially _Kl, cause k capable and the 1st the row between exchange.

[formula 43]

$M\&RightArrow;P_{\mathrm{kl}}^r\&CenterDot;M$

Because can by the arithmetical operation that matrix M is carried out taking advantage of on the right side represent and be listed as between the corresponding basic displacement of exchange, and can by the arithmetical operation of matrix M being carried out premultiplication represent and the row between the corresponding basic displacement of exchange, so these computings are tradable.Therefore, can at first carry out by making up a series of relevant exchange that basic displacement carries out or by making up the exchange of a series of and line correlation that basic displacement carries out with row.

Therefore, although the displacement that at first is listed as in the coding of the JEX in Fig. 5, the displacement of going subsequently, being somebody's turn to do order also can be opposite.

On the other hand, in above-mentioned steps S24, determine not satisfy under the situation of above-mentioned condition, the use expression is not carried out 0 of the displacement of row is come alternate data Pc (step S27).

After the processing in having finished above-mentioned steps S26 or step S27, next, determine whether to carry out the displacement of the line data in 8 * 8 block of pixels.Specifically, determine whether to satisfy the following formula 44 that has used threshold value Thr,

[formula 44]

sr(nr(0))-sr(nr(7))≥Thr×8

And whether satisfy the following formula 45 (this threshold value Nr is set to and the threshold value Nc identical value relevant with row usually) (step S28) that has used threshold value Nr.

[formula 45]

CHr≥Nr

As described in above-mentioned steps S24 and this step S28, four parameters to calculating accordingly with row and row, promptly, sc (nc (0))-sc (nc (7)), Chc, sr (nr (0))-sr (nr (7)) and Chr and corresponding threshold compare, thus can think with adaptive mode according to the opposed swap-in of the image of each block of pixels capable control.

Note, except determining whether to replace, can also determine according to following method according to these four parameters and each corresponding threshold: at first, by block of pixels is actually replaced, DCT, quantification and Huffman coding obtain to encode.In addition, encode by this block of pixels reality only being carried out DCT, quantification and Huffman, and do not replace, obtain coding.Subsequently, the amount of two kinds of codings (comprising header information) of being obtained is compared, and only have under the former situation, just adopt the result who obtains by displacement less than the latter.In this method, each block of pixels is all carried out twice encoding process, thereby the processing time is elongated, but can obtains more reliable data compression effect.Therefore, for the compression level that is provided with for coding control unit 2, can conceive a kind of being arranged so that and to select these measures.

In step S28, to determine to satisfy under the situation of these conditions, the use expression has been carried out 1 of the displacement of row is come alternate data Pr (step S29).These data Pr is the same with above-mentioned data Pc, also is to be recorded in the data of the head of image file as header information after a while.

Subsequently, actually carry out f (k, line replacement l) is so that the data array of 8 * 8 block of pixels becomes

f(k，nr(0))，f(k，nr(1))，...，f(k，nr(7))

(step S30).

On the other hand, in step S28, determine not satisfy under the situation of above-mentioned condition, the use expression is not carried out 0 of the displacement of row is come alternate data Pr (step S31).

Behind completing steps S30 or step S31, next, generation will be written to the header information Ph (n) (step S32) in the head of image file.

Figure 23 is the figure of the basic structure of the EXIF file of expression storage JPEG packed data.

This EXIF file consists essentially of: the mark SOI of expression packed data starting position; The application marker field APP1 to APPn of one or more information wherein can be described; The quantization table data, it is described from quantization table defined label DQT; Huffman shows data, and it is described from Huffman table definition DHT; Restart mark and insert definition at interval, it is described from restarting interval D RI; Frame header, it is described from frame section beginning label SOF; Scanning head, it is described from passing marker SOS; The packed data of after this scanning head, describing; And the mark EOI of expression packed data end position.

Stipulate that each size of using marker field APP1 to APPn is 64K byte or littler (K represents 1024), but the quantity of limit markers section not.For example, it is contemplated that the information of in using marker field APP1, describing Exif, in using marker field APP2, describe the information of Flashpix, and in using marker field APP3, describe the information of JEX.

The example of data structure of application marker field APPi of the information of JEX is represented wherein to have described in the right side of Figure 23.

As shown in drawings, the top in section has been described and has been represented that this is the APPi mark that i uses marker field, has described the byte length (this byte length is to be the variable-length of increment with the byte) of this application marker field APPi thereafter.Subsequently, describe the data head (for example binary value " 4A 45 58 ") that is used to expand JPEG (JEX), described above-mentioned header information Ph (n) thereafter.

Now, the quantity of header information Ph (n) will be the quantity of the piece that obtains by partitioned image, and still, as mentioned above, the size of section must be 64K byte or littler (comprising the description part such as mark etc.).Therefore, surpass under the situation of 64K byte, cross over continuous a plurality of application marker fields and describe this header information Ph (n) in size according to the JEX header information Ph (n) of all pieces.That is to say,, reach the position of the upper limit of using marker field APP3, in next uses marker field APP4, carry out the record of follow-up header information Ph in the record amount from beginning is recorded to successively and uses marker field APP3 according to the header information Ph (1) of first block of pixels.Note, each size of using marker field must be increment with the byte, and header information Ph (n) is incremental record by merging mode (packed manner) with the position, thereby in some cases, it is increment that the final data in the section may be not suitable for the byte.In this case, fill place value 0, make that this section is that increment finishes with the byte-sized as filler.

Note, in the above description, in the application marker field of EXIF file, described the JEX header information, but the part that can describe the JEX header information in the head is not limited thereto.For example, in the JPEG compression data file, can in its head, describe note, and data volume is the 64K byte.Therefore, can carry out such setting, that is, wherein use this note head partly to write down the JEX header information.

Figure 24 is for representing the figure of the basic structure of JPEG compression data file more in detail.

As reference Figure 23 partly as described in, the JPEG compression data file consists essentially of: the mark SOI (binary value with hexadecimal representation is " FFD8 ") of expression packed data starting position; The quantization table data, its description begins from quantization table defined label DQT (" FF DB " similarly); Huffman shows data, and it is described from Huffman table definition DHT (" FF C4 "); Restart mark and insert definition at interval, it is described from restarting interval D RI (" FF DD "); The quantity of vertical row, the quantity of horizontal pixel and quantization table detailed content, they expressions various parameters relevant with frame, it is described from frame section beginning label SOF (" FF C0 "); Scanning head, it comprises the various types of parameters relevant with scanning, it is described from passing marker SOS (" FF DA "); Such as the data of note etc., it is described from comment token COM (" FFFE "); Packed data, it is described from restarting mark RST (" FF D0 "); And the mark EOI (" FF D9 ") of expression packed data end position.

In the comment section of these data, at comment token COM (" FF FE ") note (that is, this expression is the section of the variable-length of increment with the byte) in the scope of size between 2 to 65535 bytes is described afterwards, note is described thereafter.In this comment section, describe for example binary value of " 4A 45 58 ",, describe above-mentioned header information Ph (n) thereafter as expansion JPEG (JEX) data head.

Figure 25 is the chart of structure and the concrete data instance of expression header information Ph (n).

Above-mentioned data Pc, the expression whether designator Ind, the expression that consists essentially of expression data starting position according to the JEX header information Ph (n) of a block of pixels has carried out the displacement relevant with row whether carried out with the above-mentioned data Pr of the displacement of line correlation, the arrangement nc (0) that is correlated with row to nc (6) and with the arrangement nr (0) of line correlation to nr (6).

Here, this designator is provided with 1 and is place value, is the data starting position and the above-mentioned filler of increment thereby distinguish with the piece.In other words, when designator Ind is not provided, be under the situation of " 00 " in two place values of the header information Ph of specific pixel piece and then, can not distinguish these positions is fillers, still represents the block of pixels of Pc=0 and Pr=0.Therefore, place value 1 always is set as designator Ind at the top of the header information Ph of each block of pixels (n).

As shown in Figure 5, under the situation of having carried out displacement, use 1 to come data of description Pc and Pr, and under the situation of not replacing, use 0 to come data of description Pc and Pr as place value as place value.At Pc is 0 o'clock, does not describe nc (0) to nc (6), on the other hand, is 0 o'clock at Pr, does not describe nr (0) to nr (6).

In addition, adopting under the situation of 8 * 8 block of pixels, the data volume of nc (i) and nr (j) is respectively 3, thereby nc (0) is 21 to the required data volume of nc (6), and nr (0) also is 21 to the required data volume of nr (6).The reason of not describing nc (7) and nr (7) in head is, arrangement nc (0) is digital 0 to 7 displacement to nr (7) to nc (7) and arrangement nr (0), thereby can know nc (7) to the information of nc (6) by obtaining nc (0), equally, can know nr (7) by obtaining nr (0) to the information of nr (6).Therefore, in the following decoding processing that will describe, carry out the calculating of nc (7) and nr (7).

In the example shown in Figure 25, Pc is 1 and Pr is 0, thereby has only described nc (0) to nc (6), and does not describe nr (0) to nr (6).Each columns certificate in the example shown in Figure 11 has been described in the example shown in Figure 25.

For above-mentioned reasons, the required data volume of the head of a block of pixels is any one in 3,24 and 45.Therefore, for the VGA image that constitutes by 640 * 480 pixels, the quantity of piece is (640/8) * (480/8)=80 * 60=4800, therefore maximum desired data amount be 27000 bytes (promptly, under situation about setting up) for all piece Pc=Pr=1, be about 26K byte (utilizing K=1024 to calculate here).In this case, because this data volume can be contained in the 64K byte as a section size,, perhaps this header information can be described in the note section so can in using marker field, describe this header information.Yet this is the situation at monochrome image.

Under the situation of coloured image, can adopt any conduct in the several different methods to be used to describe the method for header information.

At first, the method that can expect is to analyze block of pixels individually (particularly at every kind of color Y, Cb and Cr or every kind of color R, G, B, for example, the summation of the pixel value of wherein more every row or every row, obtain replacement sequence and determine whether the treatment step that is suitable for replacing), and determine to replace according to each analysis result.Then, generate the corresponding header information of displacement with each color respectively, and in head, be described.In this case, because every kind of color is all carried out optimum JEX, so can greatly reduce the encoding amount of image section.Yet for header information, required data volume is three times of the required data volume of monochrome image.In this case,, thereby can not be contained in the section, thereby must use a plurality of application marker fields even the VGA image also needs the data volume of 79K byte.Note, because do not forbid providing a plurality of note sections, so can in a plurality of note sections, describe the JEX header information.

In addition, another kind method according to the color component that has important function aspect picture quality (for example is, Y or G component) the analysis result of 8 * 8 block of pixels determine displacement, and this displacement is applied to all Y, Cb and Cr block of pixels or G, R and B block of pixels.In this case, the required data volume of header information is identical with desired data amount under the monochrome image situation, thereby can describe header information in a section that is used for the VGA image.Usually, human eye to the susceptibility of brightness signal Y greater than susceptibility to color difference signal Cb and Cr.Therefore, by carrying out JEX, can reproduce original image faithfully in the amount of information that reduces header information, more to focus on the reproduction luminance signal.This method especially has remarkable result to the lower image of colored degree.

For the image with more pixels, data volume will increase according to the quantity of block of pixels, therefore utilizes a plurality of sections descriptions of carrying out information as required.

Note, in the above description, for example, be to describe 21 bit data of nc (0) at 1 o'clock at data Pc, but also can further reduce data volume to nc (6).Because nc (0) is to any by in numeral 0 to 7 arrangement that constitutes of nc (7) representative, thus exist 8! Plant arrangement mode.Because

8！＝(2 ⁷)×315

＜(2 ⁷)×512＝(2 ⁷)×(2 ⁹)＝(2 ¹⁶)

Set up, thereby as long as the 16 bit data amounts of existence just can be described.In this case, for example, prepare in advance have 8! The table of=40320 elements, in this table, described series number 0 to 40319 and with the corresponding arrangement of each series number " 01234567 " to " 76543210 ".Subsequently, the series number of this table of record in head, and should when decoding, utilize identical table to obtain the original alignment data.

In this case, because for a block of pixels, the desired data amount of head is 3,19 and 35, so among Pc and the Pr one be 1 and another is 0 piece, the amount of header information can reduce 5, in addition, for the piece of Pc=Pr=1, the amount of header information can reduce 10.In this case, the maximum of header information becomes 21000 bytes for the VGA monochrome image, promptly about 21K byte.

In order further to reduce the amount of header information, consider to use suitable compression method to compress the header information Ph (n) that does not comprise the JEX extended head.Like this, can reduce the size of image file, and also can be contained in the section (for example note section) according to the header information of view data with high-resolution.Above-mentioned arrangement algorism (algorism) changes according to image, thereby the arrangement of being calculated has statistical property.In this case, adopt the coding (for example entropy coding etc.) of variable-length to make it possible to more effectively compress as compression method.

Now, output has been carried out the pixel data f of displacement (k, l) and the header information Ph (n) (step S33) that produces in step S32, this flow process has been back to processing shown in Figure 3 from this JEX coding then.

Note, all block of pixels are carried out processing shown in Figure 5, like this, carry out above-mentioned processing by the cyclic process that changes n in practice.

Then, Fig. 6 is the flow chart of expression JEX decoding.Fig. 6 represents the details of the processing that will carry out in the step S16 of Fig. 4.

When this handles beginning, import n inverse DCT (IDCT) the transform block f ' relevant with the n piece (k, l) and header information Ph (n) (step S41).Here, the blocks of data after the inverse dct transform is not arranged to f but the reason of being arranged to f ' is, when adopting inverse quantization unit 26 shown in Figure 2 to carry out re-quantization, produces quantization error, thereby causes the different value with original block data f.

Next, with second value that replaces with data Pc of header information Ph (n), and with the 3rd value (step S42) that replaces with data Pr.

Subsequently, whether the value of specified data Pc is 1 (step S43).

If Pc is 1, then information Ph (n) obtains nc (0) each data (step S44) to the nc (6) from the head, and according to these data computation nc (7) (step S45).

Subsequently, (k, row l) carry out inverse permutation (step S46), for example to f ' to nc (7) to utilize these nc (0)

f′(nc(k)，l)←f′(k，l)。

After in step S46, finishing this processing, perhaps in step S43, under the situation of Pc=0, next, judge whether the value of Pr is 1 (step S47).

Be that information Ph (n) obtains nr (0) each data (step S48) to nr (6) from the head under 1 the situation in the value of data Pr, and according to these data computation nr (7) (step S49).

Subsequently, (k, row l) carry out inverse permutation (step S50), for example to f ' to nr (7) to utilize these nr (0)

f′(k，nr(1))←f′(k，l)。

Like this, (k, l), this flow process is back to processing shown in Figure 4 from this JEX decoding processing subsequently to have exported the pixel data f ' that has carried out displacement.

Note,, all block of pixels are carried out processing shown in Figure 6, like this, carry out above-mentioned processing by the cyclic process that changes n in practice as processing shown in Figure 5.

And, even also can on the degree that can obtain the image general survey, reproduce the expansion jpeg file of the JEX coding generation of passing through Fig. 3 by traditional JPEG decoding processing.

That is to say normal JPEG decoding processing has been equal to omission among Fig. 4 the processing among the step S 15 and the decoding processing of the processing among the step S16.Therefore, do not carry out the displacement in the JEX decoding, thereby, can not always accurately reproduce the details in 8 * 8 data blocks, but the outward appearance conduct that can obtain image is the entire image of increment with the block of pixels for image.Particularly, the displacement of aforesaid pixel can not change the mean value of the view data in the block of pixels, thereby no matter whether carried out the JEX processing, can not influence the general survey of entire image.

Next, with reference to Figure 15 to Figure 22 the situation of concrete data being carried out conversion by above-mentioned processing is described.

Figure 15 is the chart of the data instance of original 8 * 8 block of pixels extracted of expression from view data.

This view data is the achromatic map of 8 gray scales, and pixel value is between 0 to 255.

Figure 16 has represented to calculate the summation that follows direction of 8 * 8 block of pixels data and along the situation of the summation of column direction.

In the step S22 of Fig. 5, calculate the sum data sr (j) of the capable pixel data of the sum data sc (i) of the pixel value in the i row and j.

Here, the maximum of sc (i) is 1480, and the minimum value of sc (i) is 807, and therefore, the left side of first predicated expressions among the step S24 is 673.On the other hand, by with 2 of Thc ⁵Multiply by 8, the right side of this predicated expressions is 256.Therefore, be appreciated that this first predicated expressions establishment.

On the other hand, the maximum of sr (j) is 1248, and the minimum value of sr (j) is 1107, and therefore, the left side of first predicated expressions among the step S28 is 141.On the other hand, in the right side of this predicated expressions and the step right side of first predicated expressions of S24 the same be 256.Therefore, first predicated expressions among the step S28 is false, and need not second predicated expressions is determined, and this flow process advances to step S31, and Pr=0 wherein is set.

On the other hand, in step S24, first predicated expressions is set up, thereby also must determine second predicated expressions.Here, weights be set to 1 and threshold value Nc be set under the situation of above-mentioned common value 4 formula 15 be judged.For Figure 15 and blocks of data example shown in Figure 16, as mentioned above, the nc (i) that calculate in step S23 becomes as shown in figure 11, and Chc becomes 14.Therefore, second predicated expressions is also set up.Also flow process advances to step S25, and Pc=1 wherein is set.

By the subsequent treatment among the execution in step S26, the block of pixels data become as shown in figure 17.Figure 17 has carried out the chart of the block of pixels data of displacement for expression.In the inverse permutation order that the downside of the pixel data that has carried out displacement shows the nc (i) of expression column permutation and is used for arrangement is returned to original alignment, the inverse permutation order that shows the nr (j) of expression line replacement on the right side of the pixel data that has carried out displacement and be used for arrangement is returned to original alignment.Yet, as mentioned above, because Pr=0, and do not carry out line replacement, so nr (j) is identical with its contrary (inverse).

Note, as intelligible according to the alternate form among the step S26, but the array of description below nc (i).The 0th row of the block of pixels among Figure 17 comprise that " 4 " are as nc (0).Therefore, the 4th of initial data shown in Figure 15 the enter the 0th row after being listed in displacement.Similarly, the 1st of the block of pixels among Figure 17 the row comprise that " 2 " are as nc (1).Therefore, the 2nd of initial data shown in Figure 15 the enter the 1st row after being listed in displacement.This also is applicable to other row.

Subsequently, the data of having carried out among the step S3 in Fig. 3 handling become as shown in figure 18.Figure 18 has carried out the chart of the data of level shift and DCT for expression.

Utilize predetermined quantization table the result that data shown in Figure 180 quantize to obtain to be become as shown in figure 19 at the step S4 of Fig. 3.Figure 19 is 8 * 8 a chart after expression quantizes.

Therefore, most high fdrequency component is 0, thereby can expect follow-up entropy coding packed data efficiently.

On the other hand, Figure 20 represents by in decoding processing shown in Figure 4 quantized data among Figure 19 being carried out re-quantization and inverse DCT (IDCT), these data are carried out level shift and are rounded to the data that obtain in 0 to 255 the scope.

Figure 21 carries out the chart of the block of pixels that inverse permutation obtains to data shown in Figure 20 for expression.

Therefore, carry out displacement according to contrary nc (i) shown in Figure 17 by processing, wherein, the 4th columns among Figure 20 is according to the 0th columns certificate that becomes among Figure 21, and the 2nd columns among Figure 20 is according to the 1st columns certificate that becomes among Figure 21, or the like.

Figure 22 is the result's that deducts original pixels blocks of data shown in Figure 15 in the decoded pixel blocks of data of expression from Figure 21 and obtain chart.

Figure 22 represents the error between the original pixels blocks of data of the decoded pixel blocks of data of Figure 21 and Figure 15.The summation of the absolute value of the error of each pixel is " 756 ", and the mean square of error value is " 125 ".

On the other hand, for the common jpeg file that does not carry out the JEX coding, can obtain as Figure 30 to result shown in Figure 33.Figure 30 carries out the result's of DCT chart to original 8 * 8 block of pixels data shown in Figure 15 for expression, the result's that Figure 31 quantizes the blocks of data among Figure 30 for expression chart, Figure 32 carries out re-quantization and inverse DCT, level shift and data is rounded to the result's who obtains in 0 to 255 scope chart the blocks of data among Figure 31 for expression, and Figure 33 is the result's that represents to deduct in the decoded pixel blocks of data from Figure 32 original pixels blocks of data shown in Figure 15 and obtain chart.

As mentioned above, in raw image data shown in Figure 15, only rearranged row, and do not rearranged row.In this case, the result behind displacement and the DCT compares with the traditional result who has only carried out DCT shown in Figure 30 as shown in figure 18, we can say that the DCT coefficient has following character.

(1) D ⁰⁰Constant.

(2) D ¹⁰(absolute value) increases.

(3) to nearly all v, D ^1vAbsolute value increase.

D ⁰⁰Be so-called DC component, its be with block of pixels in the proportional amount of mean value of pixel value.Much less, even replace (mean value that it does not change pixel value), D ⁰⁰Also remain unchanged.

Then, as shown in Equation 7, can be with D ¹⁰Be considered as the inner product between vectorial V (formula 8) and the vectorial W (formula 9), this vector V by k along with k increases and the function of monotone decreasing constitutes, the component of this vector W is the summation of the pixel data in the row.Therefore, obviously, the displacement between being listed as by the order of successively decreasing according to the summation of the pixel data in the row, the inner product of these vectorial V and W can increase.

And, for D ^1vAlthough can not resemble D ¹⁰Situation draw simple conclusion like that, but usually, along with the increase of the pixel value in every row, they also will increase, therefore, we can say that the displacement between being listed as by the order of successively decreasing according to the summation of the pixel data in the row has improved the possibility of increase.

DCT is orthogonal transform, thereby inner product keeps constant after conversion.Thereby the quadratic sum of DCT coefficient equals the quadratic sum of the pixel value of original pixels piece.Displacement can not change the quadratic sum of the pixel value of block of pixels, and therefore, the quadratic sum of the DCT coefficient of the block of pixels through replacing equals the quadratic sum of original DCT coefficient.Therefore, the DCT coefficient concentrate on low frequency range (0,0), (0,1), (0, v) in the time, high frequency coefficient must reduce.

In JPEG compression, after as mentioned above the DCT coefficient being quantized, carry out zigzag scanning, and carry out the Huffman coding according to the value of the DCT coefficient behind run length (it is the quantity for 0 continuous DCT coefficient) and this run length.Usually, under the situation that has a large amount of high frequency nonzero coefficients, the sign indicating number (the longest yard be 16 (run length)+10 (AC coefficient)=26) of much more long bit length will be produced.Therefore, the high fdrequency component of minimizing DCT is favourable to reducing the sign indicating number amount.

In Figure 33, the summation of the absolute value of the error in each pixel is " 900 ", and the mean square of error value is " 152 ".

Like this, as can be seen, the expansion JPEG among Figure 22 has less error, thereby has improved picture quality.

In addition, when common JPEG is encoded, original 8 * 8 block of pixels data shown in Figure 15 become 432 long, and to expansion JPEG when encoding, these data become 368.As mentioned above,, partly added 24 potential head portion information to head for the data instance among Figure 15, but still can reduce 40 data (that is, almost be in this case amount of coded data 10%).

In addition, even adopt common JPEG to decode, can not produce error to having carried out coded data by expansion JPEG yet.Although can not obtain with by expansion JPEG the same excellent image when decoding, but still can to obtain with the piece be the entire image of increment.For example, the image of the image capture device collection by 5 mega pixels has the resolution about 2500 * 2000, is that piece has produced about 313 * 250 pieces with this image division.Therefore, even when decoding, also can obtain to have the image of enough general surveys by traditional JPEG.

Figure 26 is the figure of the such system configuration of expression, and in this system configuration, the equipment that can handle expansion JPEG makes up with the equipment that can only handle traditional JPEG.

By this topology example shown in Figure 26, show the system that has made up digital camera 41 (it can produce by JPEG and carry out the jpeg image file 41a of coding and the JEX image file 41b that has carried out coding by JEX), personal computer (PC) 42 (it is merely able to decode by JPEG) and PC 43 (it can utilize JPEG to decode, but also can use JEX to decode by the software 44 that can utilize JEX to decode is installed).

Digital camera 41 is constructed to carry out JPEG compression method and JEX compression method, perhaps desired a kind of in these two kinds of methods.According to this digital camera 41, by acquisition target and utilize the JPEG method to compress the image that obtains to become jpeg image file 41a, and become JEX image file 41b by gathering same object and utilizing the JEX method to compress the image that obtains.As mentioned above, image according to JEX image file 41b is little according to the error of the image of jpeg image file 41a with respect to the error ratio of the original image of exporting from the image collecting device of digital camera 41, thereby the JEX image is the high quality graphic that has accurately reproduced more details.

Suppose jpeg image file 41a and JEX image file 41b to be transferred to PC 42 from digital camera 41 by WLAN, USB cable, storage card etc.PC 42 can carry out the JPEG decoding, thereby can obtain jpeg image file 41a has been carried out the display image 42a of decoding.On the other hand, for JEX image file 41b, still can under the situation that does not produce error, carry out decoding by JPEG decoding (rather than by JEX decoding), yet this will produce display image 42b, wherein the arrangement in the block of pixels does not revert to original order, thereby so this image become picture in the piece and deform and produce the state of so-called block noise.Yet the picture of entire image will never be out of shape, and therefore can obtain the general survey of image.Particularly, the high-definition image with a large amount of pixels has increased the quantity of the piece that will produce, and therefore, can more easily obtain the situation of entire image.

Next, suppose jpeg image file 41a and JEX image file 41b are transferred to PC 43 from digital camera 41.This PC 43 can utilize JEX to decode by the software 44 that for example provides by CD-ROM, DVD or communication line is installed.Therefore, jpeg image file 41a can be decoded into common display image 43a, and JEX image file 41b also is reproduced as the picture quality display image 43b higher than this jpeg image file 41a.

And, even,, can decode to high-quality display image 43b by being transferred to PC 43 with the corresponding JEX image file of display image 42b 41b in PC 42, producing the display image 42b that shows under the state of block noise.

Therefore, when improving picture quality, reduced data volume, in addition, even by traditional JPEG decoding processing, also can reproduce and utilize expansion JPEG to carry out the image encoded file, thereby excellent method for compressing image is provided with certain picture quality rank.

Note,, show digital camera as the example that is used to produce the device of expanding jpeg file, but be not limited thereto certainly by above description.As the example that is used to produce the device of expanding jpeg file, can mention image capture device.This image capture device comprises and is used to utilize collection optical object images such as electronic image acquisition means to obtain the device or the system of view data.The example of this image collecting device for example comprises Electrofax, video camera, have the mobile phone of camera, the PC of camera is installed, such as Medical Instruments, indoor/outdoor monitoring camera and the supervisory control system (being used for obtaining view data) of endoscope (for example gastroscope) etc. by the signal that obtains from these cameras being carried out signal processing in another place, be used for gathering electronically printing image etc. scanner, be used for gathering electronically the Film scanner of film image, or the like.In addition, for the device beyond the image capture device that is used to produce the expansion jpeg file, can be used as example such as the computer of PC etc.Promptly, in this case, treat to be not limited to by taking the natural image of acquisition by the image that expansion JPEG handles, but be included in the computer graphical that produces in the computer etc. widely, be rendered as the character data of bitmap etc., by various types of view data being synthesized or handle the view data that obtains, or the like.Therefore, the device that can produce this view data becomes the target of the device that is used to produce the expansion jpeg file widely.

In addition, computer such as PC etc. can become and can carry out the device that JEX handles by the mode of software by installing and carry out JEX coding/decoding software, but, on the other hand, the expansion board etc. that is used to carry out the JEX coding/decoding by installation can become and can carry out the device that JEX handles by hardware mode.

Notice that term JEX comprises the definition of broad sense and narrow sense.For the definition of broad sense, will comprise that the entire process that JPEG handled and be not included in processing among the JPEG (for example the displacement of view data etc.) is called JEX.That is to say that the JEX of broad sense comprises JPEG.On the other hand, narrow sense JEX represents the difference between generalized J EX and the traditional JPEG.In this case, have only with the corresponding part of processing and be called as JEX such as the displacement of view data etc.

The method that provides that is used to carry out software and hardware that the JEX coding/decoding handles is provided in these definition.

Specifically, by on PC etc., independently software being installed, can carrying out jpeg image with corresponding JEX encoding software of the definition of broad sense or JEX decoding software and handle and the JEX image processing.

On the other hand, JEX encoding software or JEX decoding software are installed in addition as peripheral software or addition type module etc. is installed by going up, except jpeg image is handled, can also be carried out the JEX image processing with corresponding JEX encoding software of the definition of narrow sense or JEX decoding software at the PC that traditional JPEG encoding software (basic coding handling procedure) or JPEG decoding software (handling procedure of decoding substantially) are installed etc.

As the method that provides of software, can optionally use these definition or these definition to coexist.For example, for the user who does not have JPEG coding/decoding software, by providing to these users and the corresponding JEX coding/decoding of the definition of broad sense software, the user can obtain following advantage: need not to carry out twice installation and can handle jpeg image and JEX image immediately.On the other hand, for user with JPEG coding/decoding software, by providing to these users and the corresponding JEX coding/decoding of the definition of narrow sense software, the user can obtain following advantage: the user can carry out the JEX image processing by expanding existing software.

The problem of this new standard is to be difficult to popularize.On the other hand, the JEX that the applicant proposes goes up compatibility, comprises present widely used JPEG, and this is favourable for universal JEX, because JEX has multifunctionality.

In addition, it is contemplated that by introducing the business method JEX encoding software for example is provided with compensation but freely issues the JEX decoding software and realize in a short time being extensive use of.

In addition, for software, it is contemplated that for example provides and the corresponding JEX coding/decoding of the definition of broad sense expansion board, perhaps provides and the corresponding JEX coding/decoding of the definition of narrow sense expansion board to the user who has JPEG coding/decoding expansion board (basic coding/decoding device).

Figure 27 can apply to JEX the block diagram of structure of the encoder of moving image for expression, and Figure 34 is the block diagram of the structure of the traditional moving image encoder of expression.

This decoder comprises subtracter 51, JEX processing unit 52, header information memory 53, DCT unit 54, quantifying unit 55, inverse quantization unit 56, IDCT unit 57, contrary JEX processing unit 58, adder 59, frame memory 60, motion correction unit 61, variable length code unit 62 and synthesis unit 63.

Therefore, the moving image encoder that can use JEX is by add the encoder that JEX processing unit 52, header information memory 53 and contrary JEX processing unit 58 obtain to conventional motion image encoder shown in Figure 34.In addition, synthesis unit 63 shown in Figure 34 is used to add the information such as mark etc., and synthesizer 63 shown in Figure 27 is used for further adding the JEX header information.

Subtracter 51 is used for deducting from input signal following with the prediction signal of describing, and it is exported as predictive error signal.

JEX processing unit 52 is used for carrying out column permutation and line replacement in the block of pixels by the mode identical with the situation of the processing of JPEG.

Header information memory 53 is used to store the JEX header information that is produced by JEX processing unit 53.

DCT 54 is used to utilize DCT to carrying out spatial frequency transforms from the block of pixels of JEX processing unit 52 outputs.

Quantifying unit 55 is used for the blocks of data from DCT 54 outputs is quantized.

Inverse quantization unit 56 subsequently to motion correction unit 61 is the processing units that are used for producing according to the blocks of data that has carried out quantizing prediction signal.

Inverse quantization unit 56 is used for the blocks of data from quantifying unit 55 outputs is carried out re-quantization.

IDCT 57 is used for carrying out contrary spatial frequency transforms by the blocks of data from inverse quantization unit 56 outputs is carried out inverse dct transform.

Contrary JEX processing unit 58 is used for reading the JEX header information that is stored in header information memory 53, and is carrying out the contrary column permutation and the displacement of driving in the wrong direction in the block of pixels of IDCT 57 outputs.

Adder 59 is used for the data from contrary JEX processing unit 58 outputs are carried out addition with the prediction signal of exporting from motion correction unit 61, and the result is exported as local decoded signal.

Frame memory 60 is used to store the local decoded signal from adder 59 outputs.

Motion correction unit 61 is used for the local decoded signal that is stored in frame memory 60 is carried out motion correction.Outputed to subtracter 51 and adder 59 respectively from the prediction signal of these motion correction unit 61 outputs.

Variable length code unit 62 is used for the blocks of data from quantifying unit 55 output is carried out processing such as zigzag scanning, Huffman coding etc.

Synthesis unit 63 is used for the coded datas of synthetic information such as mark etc., 62 outputs from the variable length code unit and information-storing device 53 JEX header information data of reading from the head, and with the result as the output of coding bit string.

Note, when moving image is decoded, should use the JEX header information to carry out JEX and handle, thereby omit detailed description in mode much at one.

Now, for example, mpeg file comprises as the I image of in-frame encoding picture (view data), as the P image of interframe forward predictive coded image (differential image data) and as the B image of bidirectionally predictive coded picture (differential image data).

By this moving image structure, there is several different methods for how which image above-mentioned displacement being applied to.

For example, it is contemplated that such method, wherein, only compress the I image by the coding that uses displacement, and the P image and the B image of encoding and compressing other by the tradition of not using displacement.In this case, need in the head part of I image, describe as described pretreatment parameter under the situation at JPEG.Adopt this method especially can reduce the sign indicating number amount of I image.

It is contemplated that another kind of method, wherein, by using the coding that the I image is analyzed the displacement that obtains to compress the I image, and use and be applied to P image and the B image of the displacement identical and carry out preliminary treatment other with the displacement of the corresponding block of pixels of I image.Adopt this method can reduce the sign indicating number amount significantly.

In addition, can carry out preliminary treatment to the difference signal block of pixels of P image and B image, rather than the I image is carried out preliminary treatment, this is unusual effective method.This is because in the difference signal block of pixels, and nearly all signal all becomes and approaches 0 little value, and only comprises that the part of motion becomes bigger signal value.Therefore, for example, comprise that by only rearranging the column or row that have than the clauses and subclauses of large-signal value just can easily concentrate on a position with the clauses and subclauses that have than the large-signal value, can reduce the high fdrequency component of DCT coefficient, and can reduce the sign indicating number amount.In addition, P image and B image comprise that all difference signal all approach 0 block of pixels, and these block of pixels are not carried out spatial frequency transforms such as DCT.Therefore, these pieces are not carried out pretreated displacement as spatial frequency transforms yet.In addition, for moving image compression such as MPEG 4 grades, do not describe header information for each image, but write this header information for each group that comprises a plurality of I images, P image and B image (" video object plane group ") such as note, user data etc.Therefore, for the parameter according to first embodiment, the pretreatment parameter that is used for a plurality of images will be collectively written into the head part of this group.

Notice that utilizing traditional MPEG playback to carry out under the situation of compression movement view data of replacement Treatment, anamorphose is too obvious, and can not ignore.Therefore, moving image compression is being carried out under the situation of replacement Treatment, prerequisite is to decode by the expansion mpeg decoder that utilizes replacement Treatment rather than by traditional mpeg decoder.For example, preferably can when the compression movement image, select compact model, and selecting when selecting compact model is to carry out replacement Treatment to realize high compression ratio (the spatial alternation pattern is set), has still carried out decoding and has not carried out replacement Treatment (the spatial alternation pattern is not set) because suppose by traditional MPEG.

According to this structure, handle by carrying out JEX when the decoding relevant with moving image, can obtain to have the more motion image data of high image quality and high compression ratio.

Notice that in the above description, the displacement of the pixel of 8 * 8 block of pixels data is to be undertaken by the displacement of combination row and column.In this case, exist after the conversion 8! * 8! Plant the pixel arrangement mode.On the other hand, with 64 pixels be set to exist in 64 location of pixels 64! The mode of kind.Therefore, therefore the pixel arrangement mode that exists the combination of the displacement of row and column not contain, should be appreciated that, be not total energy obtain to have maximal pressure contract than the pixel arrangement mode.In theory, 8 * 8 block of pixels can be expressed as the 64-D vector, with for example by the displacement by descending (or ascending order) to this vector components is sorted, the result that the same procedure that utilization is undertaken by zigzag scanning element 11 will sort like this is arranged in these 8 * 8 with the mode of the zigzag upper left corner since 8 * 8, thus the final pixel arrangement mode that obtains to have higher compression ratios.Yet, in this case, can expect that the size of the JEX header information that will describe becomes bigger in the head part, therefore, the whole size of image file becomes big.

According to the first such embodiment, before spatial frequency transforms, in image space, image space is carried out the image space conversion, like this, can suppress to carry out spatial frequency transforms high frequency coefficient afterwards, thereby the coding that can have high compression ratio, and can have the coding that high-quality and image deterioration reduce.

Therefore, but adopt the linearity inverse mapping, thereby can recover image as spatial alternation, and can be owing to decoding time space conversion obliterated data.

In addition, in the reversible mapping of linearity, but adopt the linearity inverse permutation particularly, thereby can carry out arithmetic processing at a high speed, but also do not change the mean value of pixel and statistical value etc.

Another advantage is, can also decode by traditional JPEG decoding processing and utilize expansion JPEG (JEX) image encoded file, and, be the noise of increment even produce in this case with the piece, also still can be on the rank that obtains the image general survey without difficulty reproduced image.

[second embodiment]

Figure 35 to Figure 59 represents the second embodiment of the present invention, and Figure 35 is the block diagram of the structure of expression expansion jpeg coder.

In this second embodiment, the part identical with above-mentioned first embodiment is additional identical Reference numeral, and omits description of them.To mainly only be described at difference.

Replace preliminary treatment according to the expansion jpeg coder of above-mentioned first embodiment, handle (this processing also is the conversion of reversible graph image space) preliminary treatment as common JPEG encoding process but carry out reversible low-pass filtering (LPF) according to the expansion jpeg coder of this second embodiment as common JPEG encoding process.

Specifically, expansion jpeg coder according to this second embodiment is the encoder that obtains by following processing: remove row operation determination unit 4 and line operate determining unit 5 in the structure shown in Figure 1 among first embodiment, and provide filter unit 14 to replace them as the spatial alternation device.To control required Control Parameter etc. to coding is input to this filter unit 14 from coding control unit 2.

In addition, be used to control the following scale factor SF etc. of the quantization table that produces by quantization table generation unit 9 from the input of coding control unit 2.This scale factor SF also is input to synthesis unit 13 by quantization table generation unit 9, and is added in the header information.

Figure 36 is the block diagram of the structure of expression expansion jpeg decoder.

Expansion jpeg decoder shown in Figure 36 is used for that expansion jpeg coder shown in Figure 35 has been carried out the image encoded data and decodes.

Specifically, expansion jpeg decoder according to this second embodiment is the decoder that obtains by following processing: removed line operate unit 31, row operating unit 32 and decoding control unit 30 in the structure shown in Figure 2 among first embodiment, and provide liftering unit 34 to replace them as the inverse spatial transform device.Will from head interpreter 29 be used for control the required Control Parameter of decoding etc. and be input to this liftering unit 34.

In this second embodiment, the flow process that whole JEX handles is identical with flow process shown in Figure 3 among first embodiment in a broad sense.

Figure 37 is the flow chart of the summary of the expression JEX encoding process relevant with the step S2 of Fig. 3.

This JEX encoding process among second embodiment divides two stages to carry out: JEX (LPF) preliminary treatment (step S61), and it is performed and is used for determining the required Control Parameter of low-pass filtering; And main handle (the step S62) of JEX (LPF), it is used for according to determined Control Parameter image being carried out low-pass filtering.

Figure 38 is the flow chart of the details of the JEX (LPF) among the step S61 of expression Figure 37.

When beginning this processing, carry out user's input (step S70) and data inputs (step S71).To the setting option that the item of importing comprises the setting option that is called as " Choice (selection) " and is called as " Method (method) " be set by the user.

Back one setting option Method is used for carrying out the setting option selected between the low-pass filtering treatment focusing on to improve compression ratio or focus under the situation that improves picture quality with narrow sense JEX coding.Here, Method=1 is set when selecting high compression ratio, and when selecting high image quality, Method=0 is set.

High compression ratio or high image quality that last setting option Choice is illustrated in the selection of Method item will reach how high rank.When selecting Choice=0, predetermined levels of the standard are set, the rank higher than levels of the standard is set when selecting Choice=1, and when selecting Choice=2, highest level is set.

On the other hand, by the data input, for example input is stored in the reference block N in the coding control unit 2 in advance _TestQuantity.As mentioned above, be that increment is handled image with 8 * 8 block of pixels, so under the situation of the VGA image that constitutes by 640 * 480 pixels, the total N of piece _TotalBe 80 * 60=4800.At this piece sum N _TotalScope in the quantity N of reference block _TestBe set to suitable value.

In following formula 46, illustrated according to N _TotalDetermine reference block quantity N _TestMethod.

[formula 46]

$N_{\mathrm{Test}}=\left\{\begin{array}{cc}{N}_{\mathrm{Total}}& (N_{\mathrm{Total}}\&le;8)\\ \max (8,[\frac{1}{4}\sqrt{N_{\mathrm{Total}}}\left]\right)& (N_{\mathrm{Total}}>8)\end{array}\right.$

Here, the square brackets representative in the formula 46 is no more than the maximum integer of the value in the square brackets, max (A, B) higher value among expression A and the B.

When this formula 46 of employing, for the example of above VGA image, reference block quantity N _TestBe 17.

Notice that this formula 46 is an example, and certainly determines reference block quantity N by other method _Test

Next, in the middle of all 8 * 8 block of pixels of composing images, be chosen in the reference block quantity N that imports among the step S71 at random _TestIndividual 8 * 8 block of pixels (step S72).Note, although from all block of pixels, select randomly, be not necessarily limited to this, but can select the block of pixels at correct position place.For example,, and wish processing to be optimized at this main object if having main object at the center of image, then it is contemplated that will be to be selected with higher ratio block of pixels be arranged near the picture centre.

Subsequently, according to as shown in figure 39 (η, SF) table comprises the generalized J EX encoding process (step S73) of low-pass filtering treatment to each 8 * 8 block of pixels view data of selecting in step S72.

(η, SF) the right chart of the Control Parameter that Figure 39 selects according to setting option Choice for expression.η in the Control Parameter shown in this Figure 39 is the parameter that is used to be provided with the low pass filtered intensity of wave.That is to say, in this second embodiment, in that (k, when l) being expressed as the matrix of 8 row 8 row, (k η) carries out low-pass filtering treatment as shown in the formula the LPF matrix L shown in 47 by multiply by with 8 * 8 block of pixels view data f.

[formula 47]

In this formula 47, along when left to bottom right direction is observed diagonal entry, the 1-η of first appearance be arranged on the capable k row of k (k=0 ..., 6) locate.Here, η can get (1/2)〉value in the scope of η 〉=0.Note, got rid of η=(1/2), this is because there be not (irreversible) in inverse matrix, as can from following with shown in the formula of describing 48 like that.In addition, got rid of η＞(1/2), this is because if not like this, and then the pixel data of another pixel position adds the ratio with the pixel data self that is higher than the specific pixel location place pixel data of this specific pixel location place to, thereby causes excessive low-pass filtering.

For this η, be 0 o'clock at η, (k η) becomes unit matrix to matrix L, and low-pass filter effect becomes 0, approaches 0 o'clock at η, and this low-pass filter effect becomes less, and when approaching (1/2) at η, this low-pass filter effect becomes bigger.

Note, to pixel data f (k, l) premultiplication LPF matrix L (k, in the time of η), carry out vertical filtering operation (being used to reduce the arithmetical operation of the difference between the adjacent lines), and to pixel data f (k, l) LPF matrix L (k is taken advantage of on the right side, in the time of η), carry out horizontal filtering operation (being used to reduce the arithmetical operation of the difference between the adjacent column).Now, suppose pixel data f (k, the element of k in certain delegation l) row is a, the element of (k+1) row is b, and to this pixel data right side take advantage of the LPF matrix L (k, η).So the element with the k of delegation row after the arithmetical operation becomes (1-η) a+ η b, and the element with (k+1) row of delegation becomes η a+ (1-η) b after the arithmetical operation.Therefore, with the difference of along continuous straight runs before the arithmetical operation being | a-b| compares, and the difference of along continuous straight runs becomes (1-2 η) after this arithmetical operation | a-b|.Therefore, consider above-mentioned scope 1 〉=(1-2 η)＞0 that η can value, can easily see, suppressed the difference of along continuous straight runs.Notice that (k under situation η), can use identical demonstration, thereby has suppressed difference vertically in premultiplication LPF matrix L.

This low-pass filtering remains unchanged the DC coefficient behind the dct transform.

In addition, the matrix that use formula 48 expressions are used to carry out contrary LPF arithmetical operation.

[formula 48]

By this formula 48 (it is the inverse matrix of formula 47), along when left to bottom right direction is observed diagonal entry, for the first time the factor (1-η)/(the 1-2 η) that occurs appear at the capable k row of k (k=0 ..., 6).

Note, be the example that is used to carry out the matrix of LPF and contrary LPF at the matrix shown in these formulas 47 and 48, but be not limited thereto.

In the example shown in Figure 39, the value of η has 5: promptly, and 0, (1/8), (1/6), (1/5) and (1/4).These are provided with according to practical application, and these are example certainly, and the value that can adopt other is as η.

On the other hand, the SF in the Control Parameter shown in Figure 39 controls the parameter of quantization step for being used for multiplying each other with quantization table (Q table).In the example shown in Figure 39, the value that be taken as SF has 6 kinds, and 1, (7/8), (6/8), (5/8), (4/8) and (3/8).Here, SF=1 represents to adopt original quantization table itself.For this scale factor SF, can carry out wherein other value being made as the setting of SF.

In the table shown in Figure 39, expression have zero (η is SF) to being the default value when the setting option Choice=0.In addition, expression have (η, SF) to being when the setting option Choice=1, will at expression have zero (η is SF) to the optional value of other selection.In addition, expression have △ (η, SF) to being when the setting option Choice=2, will at expression have and zero (η is SF) to the optional value of other selection.

Like this, the N in step S72, selecting _TestEach n in individual 8 * 8 block of pixels _i(variable n _iVariable for value on all benchmark 8 * 8 block of pixels) carry out generalized J EX encoding process, this generalized J EX encoding process comprises (η, SF) the combination low-pass filtering of carrying out that utilization is selected according to the value of setting option Choice.Like this, (k l) obtains to have carried out low-pass filtering treatment, DCT and Huffman image encoded data at each benchmark 8 * 8 block of pixels data f.

Subsequently, utilize generalized J EX decoding that coded image data is decoded, thus obtained to have carried out Huffman decoding, inverse DCT and contrary low-pass filtering treatment 8 * 8 block of pixels data f ' (k, l).Then, (k, (k, l), thereby acquisition is according to the summation of the absolute value of each pixel l) to deduct 8 * 8 block of pixels data f ' that carried out Code And Decode from 8 * 8 preceding block of pixels data f that encode.Like this, obtained the summation of the absolute value of difference, Abs (n _i, η, SF).

In addition, the code length of the Huffman coding that will obtain by generalized J EX encoding process and bit length addition according to the header information of 8 * 8 block of pixels, thus calculate Bit (n _i, η, SF) (step S74).

Utilize the Abs (n of such acquisition by the mode shown in following formula 49 and the formula 50 _i, η, SF) and Bit (n _i, η SF) calculates and all benchmark 8 * 8 block of pixels n _iCorresponding picture quality estimated value Abs (η, SF) and code length estimated value Bit (η, SF).

[formula 49]

$\mathrm{Abs}(\mathrm{\&eta;},\mathrm{SF}):=\underset{n_i}{\mathrm{\&Sigma;}}\mathrm{Abs}(n_i,\mathrm{\&eta;},\mathrm{SF})$

[formula 50]

$\mathrm{Bit}(\mathrm{\&eta;},\mathrm{SF}):=\underset{n_i}{\mathrm{\&Sigma;}}\mathrm{Bit}(n_i,\mathrm{\&eta;},\mathrm{SF})$

Here, symbol " :=" expression substitution (substitution).

(η SF) represents that its value is more little to picture quality estimated value Abs, and then picture quality is high more, and (η SF) represents that its value is more little to code length estimated value Bit, and then compression ratio is high more.

Subsequently, determine whether to according to the value of Choice from (η, all (η that SF) select in the table, SF) to having carried out handling (step S75), and if exist do not handle (η, SF) right, then flow process turns back to step S73, and wherein (η is SF) to handling at the next one.

Like this, in step S75, if determine that (η SF) to handling, determines further then whether the Method item is 1 (step S76) to all.

Here, under the situation of Method=1 (expression selected high compression ratio), from all satisfy Abs (0,1) 〉=Abs (η, SF) (η, SF) centering is selected to make Bit (η, SF) minimum (η 1, and is SF1) right.(η is to compare the necessary condition that picture quality can deterioration with the image of handling by common JPEG SF) to the condition Abs that is applied (0,1) 〉=Abs.Note, if exist two or more (η 1, SF1) to make Bit (η, SF) minimum, then select Abs (η, SF) have smaller value (η 1, and is SF1) right, thereby not only improved compression ratio, but also improved picture quality.On the other hand, (η, SF) (η SF), then gets (0,1) automatically, and low-pass filtering treatment (step S77) is not carried out in its expression to satisfying Abs (0,1) 〉=Abs if do not have.

In addition, in step S76, if determine not satisfy Method=1 (that is, this moment Method=0), high-quality has been selected in this expression, then satisfies Bit (0,1) 〉=Bit (η, (η SF) from all, SF) centering is selected to make Abs (η, SF) minimum (η 0, and is SF0) right.(η SF) is the necessary condition that size of data is no more than the size of the data of using common JPEG compression to the condition Bit that is applied (0,1) 〉=Bit.Notice that (η, SF) minimum (η 0, and SF0) right quantity surpasses 1, and (η, SF) less (η 0, and is SF0) right, thereby not only improved picture quality, and improved compression ratio then to select Bit if make Abs.On the other hand, (η, SF) (η SF), then gets (0,1) automatically, and low-pass filtering treatment (step S78) is not carried out in its expression to satisfying Bit (0,1) 〉=Bit if do not have.

Behind completing steps S77 or step S78, export that selected (η SF) to (step S79), finishes this JEX (LPF) then and handles.

Figure 40 is the flow chart of the main details of handling of JEX (LPF) of the step S62 among expression Figure 37.

This JEX (LPF) is main handle be used for according to JEX shown in Figure 38 (LPF) that preliminary treatment is provided with (η, SF) right, entire image is carried out low-pass filtering treatment.

When this handles beginning, import n (n=1 ..., N _Total) individual 8 * 8 block of pixels f (k, l) (step S81).

Subsequently, input data set DA (step S82).This data set DA is the data set that comprises aforementioned η, Thc2, Thr2, Mode vector sum Pos vector.Here, Thc2 is a threshold value, and it is used for determining the maximum V of following difference with the along continuous straight runs described _ScWhether enough need the application level filtering operation greatly.Similarly, Thr2 is a threshold value, and it is used for determining the maximum V of the following difference vertically that will describe _SrWhether enough need to use the vertical filtering operation greatly.Here the Mode vector is the 8-D vector, and is configured to represent whether to have carried out the value of vertical filtering or horizontal filtering.For example, respectively, " 1 " is the value of expression vertical filtering operation, and " 2 " are the value of expression horizontal filtering operation.Notice that these vectorial dimension 8 expressions are carried out maximum 8 times altogether by the low-pass filtering operation that vertical filtering operation or horizontal filtering operation constitute.If repeatedly repeat low-pass filtering treatment, image flattened gradually (planarize) then, thus the upper limit is defined as 8 times here to prevent undue complanation.Notice that this upper limit is not limited to 8 times usually certainly.The Pos vector is the vector that has write down the row number of 8 * 8 block of pixels that will carry out the operation of vertical filtering operation or horizontal filtering or row number.With reference to Figure 55 the concrete example of these Mode vector sums Pos vector is described below.

Next, with 1 substitution cycle count variables L oop (step S83).Here, as mentioned above " :=" the expression substitution.

Subsequently, whether determine Loop greater than 8 (step S84), and if Loop be not more than 8, then calculate the maximum V of the difference of along continuous straight runs _Sc, and this maximum V is provided _ScRow k in minimum value kc (step S85).

Subsequently, calculate the maximum V of difference vertically _Sr, and this maximum V is provided _Sr Row number 1 in minimum value lr (step S86).

Check the maximum V that calculates like this _ScWhether greater than maximum V _Sr(step S87) is if maximum V _ScGreater than maximum V _Sr, then further determine maximum V _ScWhether greater than threshold value Thc2 (step S88).If V _ScGreater than threshold value Thc2,, and Mode (Loop) is set :=1 and Pos (Loop) :=kc (step S90) then as shown in the formula the such executive level filtering operation shown in 51.

[formula 51]

f(k，l)←[f*L(kr，η)] _k，l

On the other hand, in step S87, if maximum V _ScBe equal to or less than maximum V _Sr, then further check maximum V _SrWhether greater than threshold value Thr2 (step S89).If maximum V _SrGreater than threshold value Thr2, then carry out the vertical filtering operation, and Mode (Loop) is set :=2 and Pos (Loop) :=lr (step S91) as shown in the formula such shown in 52.

[formula 52]

f(k，l)←[L(kr，η)*f] _k，l

Behind completing steps S90 or step S91, if in step S88 maximum V _ScBe equal to or less than threshold value Thc2, if perhaps maximum V in step S89 _SrBe equal to or less than threshold value Thr2, then make variables L oop increase progressively (step S92), and flow process is back to step S84, wherein repeat above-mentioned processing.

Like this, when confirming that in step S84 variables L oop greater than 8 the time, produces header information Phl (n) (step S93), out-feed head information Phl (n) and carried out the f of LPF (k, l) (step S94) finish the main processing of this JEX (LPF) then.

Notice that Figure 40 has described the processing to (n) 8 * 8 block of pixels.Much less, in actual treatment, at N _TotalIndividual 8 * 8 block of pixels are carried out this processing.

Figure 41 is the V among the step S85 among expression Figure 40 _ScFlow chart with the details of the computing of kc.

When this handles beginning, calculate by horizontal shift 8 * 8 block of pixels f (k, l) and the data (step S101) that obtain.

Subsequently, by (k deducts the data of horizontal shift (shifting left) and gets poor absolute value in l), calculates poor absolute value (step S102) from initial data f.Yet, although initial data by 8 row (k=0 ..., 7) constitute, the data of horizontal shift by 7 row (k=0 ..., 6) constitute, thereby at 7 row (k=0 ..., 6) calculate, the quantity of the absolute value of the difference of being calculated is 7 * 8=56.

In the absolute value of the difference of being calculated, search maximum V _Sc(step S103).Be used to search maximum V _ScProcessing can represent by the expression formula shown in the following formula 53.

[formula 53]

$\mathrm{Vsc}=\underset{k=0,\&CenterDot;\&CenterDot;\&CenterDot;,6}{\underset{l=0,\&CenterDot;\&CenterDot;\&CenterDot;,7}{\max }}|f(k,l)-f(k+1,l)|$

Subsequently, search maximum V is provided _ScRow k in minimum value kc (step S104), finish this processing then.Note, only when existing two or more that maximum V is provided _ScRow k the time, the just processing among the execution in step S104, when only having a row k under the situation of not searching selection row k.

Figure 42 is the V among the step S86 of expression Figure 40 _SrFlow chart with the details of lr computing.

When this handles beginning, calculate by vertical movement 8 * 8 block of pixels f (k, l) data of Huo Deing (step S105).

Subsequently, by (k deducts the data of vertical movement (upward displacement) and gets poor absolute value in l), calculates poor absolute value (step S106) from initial data f.Yet, although initial data by 8 row (k=0 ..., 7) constitute, the data of vertical movement by 7 row (k=0 ..., 6) constitute, thereby at 7 row (k=0 ..., 6) calculate, and the quantity of the absolute value of the difference of being calculated is 7 * 8=56.

In the absolute value of the difference of being calculated, search maximum V _Sr(step S107).Be used to search this maximum V _SrProcessing can represent by the expression formula shown in the following formula 54.

[formula 54]

$\mathrm{Vsr}=\underset{k=0,\&CenterDot;\&CenterDot;\&CenterDot;,7}{\underset{l=0,\&CenterDot;\&CenterDot;\&CenterDot;,6}{\max }}|f(k,l)-f(k+1,l)|$

Subsequently, search this maximum V is provided _Sr Row number 1 in minimum value lr (step S108), finish this processing then.Note, only when existing two or more that maximum V is provided _Sr Row number 1 o'clock, row number 1 is selected in the just processing among the execution in step S108 under the situation of not searching when only having a row number 1.

Figure 43 is the flow chart of expression JEX (LPF) decoding processing.

When this handles beginning, import n idct transform piece f ' (k, l), header information Phl (n) and the parameter η (step S111) that represents the intensity of LPF.

By the Mode that 0 substitution is all (1) ..., Mode (8) carries out initialization (step S112) to this 8-D vector M ode.Note, in Figure 43, be attached to the symbol of i front, promptly inverted " A " is the symbol of any i of expression.

Then, will be from the head second of information Phl (n) up to the position information of separator " 000 " with 2 be increment be updated in order Mode (1), Mode (2) ... in, and will from separator " 000 " to the last position information with 3 be increment be updated in order Pos (1), Pos (2) ... in (referring to Figure 44, Figure 45 etc.) (step S113).

Subsequently, the variables L oop (step S114) that 1 substitution is represented cycle count.

Next, whether definite the and corresponding Mode component of a vector of cycle count Mode (Loop) is 2 (step S115).If Mode (Loop) is 2, then utilize following formula 55 to carry out contrary vertical filtering operation (step S116).

[formula 55]

f′(k，l)←[L(Pos(Loop)，η) ^-1*f′(k，l)] _k，l

On the other hand, Mode in step S115 (Loop) is not under 2 the situation, determines then whether Mode (Loop) is 1 (step S117).

If Mode (Loop) is 1, then utilize following formula 56 to carry out flat against the current filtering arithmetic operation (step S118).

[formula 56]

f′(k，l)←[f′(k，l)*L(Pos(Loop)，η) ^-1] _k，l

Under the situation of completing steps S116 or step S118, make variables L oop increase progressively (step S119), determine that then whether variables L oop is greater than 8 (step S120).

Variables L oop be 8 or less than 8 situation under, flow process turns back to step S115, repeats above-mentioned processing at step S115.

On the other hand, if variables L oop greater than 8, perhaps Mode (Loop) is not 1 in step S117, (k, l) (step S121) finish this JEX (LPF) decoding then to export f ' through conversion.

Note, in Figure 43, show processing, but much less (n) idct transform piece, in actual treatment to N _TotalIndividual idct transform piece carries out this processing.

Figure 44 is the chart of the example of expression Mode vector sum Pos vector.

Shown in Figure 44 be exemplified as wherein carried out the example that 8 LPF handle to one 8 * 8 block of pixels.In this example, only i=4 or 8 o'clock, Mode (i)=2 just set up, and this shows and carried out the vertical filtering operation, and all other situation represents to have carried out the horizontal filtering operation.This Mode (i) is made of per two positions.Carry out the row number of filtering arithmetic operation or row for each i and number be represented as Pos (i), but also shown its bit representation.In k and the l scope between 0 to 7, thereby in them each all is represented as 3 value.

Notice that LPF handles and carries out at most 8 times, but is less than sometimes 8 times.For example, suppose LPF handle carried out j (7 〉=j) is inferior, then at Mode (j) back rs chacter, and behind this separator, write down Pos (i) ..., Pos (j).

Figure 45 is the figure of the example of expression header information Phl (n).

Shown in this accompanying drawing, the header information Phl (n) that is used for n 8 * 8 block of pixels comprises designator Ind, Mode (i), separator and Pos (i).

Designator Ind is 1 data, its expression data starting position, thus data and filler are distinguished, and 8 * 8 all block of pixels all are set to 1.

Mode (i) is for being recorded as the data among Figure 44 " 1 " or " 2 " part of 2 place values.

Separator is used to distinguish Mode (i) and Pos (i), and 0 (that is, using " 000 " of bit representation) for being represented by 3 place values.

Pos (i) is for wherein writing down the part of bit representation shown in Figure 44 in order.

Do not handle if carry out LPF at all, then Phl (n) becomes " 100 ".

Figure 46 is used for the chart of example of the global header of entire image for expression.

As above-mentioned first embodiment, this global header for example is recorded in the note COM part shown in Figure 24, perhaps be recorded in the application marker field of EXIF file, or the like.

The global parameter that this global header comprises " JEX " with binary system (ascii data) record, be registered as the classification of 4 place values, be registered as the value that comprises 38 place values (promptly, 24 altogether) and the head Phl (n) that is used for all pieces, in addition, also be added with filler if desired.

Figure 47 is the chart of the example of the expression place value in the classification of global header to be recorded.

In this example, classification is expressed as 4 place values.At first, narrow sense JEX processing is not carried out in classification " 0000 " expression, and in this case, generalized J EX handles and handles identical with common JPEG.

Then, classification " 0001 " expression has been carried out handling as narrow sense JEX as the displacement of describing in first embodiment.

The reversible LPF that classification " 0010 " expression has carried out describing in a second embodiment handles and handles as narrow sense JEX.

Classification " 0011 " expression has been adopted displacement and reversible LPF to handle together and has been handled as narrow sense JEX.Below will this situation be described as the 3rd embodiment.

Notice that other place value that can be appointed as classification is to be used for the reserve area that other JEX handles.

Figure 48 is the chart of the example of first 8 place value of expression global parameter shown in Figure 46.

Here, write down and the corresponding place value of parameter η as first 8 place value of global parameter.For example, each value for as shown in figure 39 η, an o'clock record " 00000000 " is a place value in η=0 respectively, record " 00000001 " is a place value when η=(1/8), record " 00000010 " is a place value when η=(1/6), record " 00000011 " is a place value when η=(1/5), and record " 00000100 " is a place value when η=(1/4).

Figure 49 is the chart of the example of second 8 place value of expression global parameter shown in Figure 46.

Here, write down and the corresponding place value of scale factor SF as second 8 place value of global parameter.For example, each value for as shown in figure 39 SF, record " 00000000 " is a place value when SF=1 respectively, record " 00000001 " is a place value when SF=(7/8), record " 00000010 " is a place value when SF=(6/8), record " 00000011 " is a place value when SF=(5/8), and record " 00000100 " is a place value when η=(4/8), and record " 00000101 " is a place value when SF=(3/8).

Notice that the 3rd 8 place values of global parameter shown in Figure 46 are reserve area.

Figure 50 is the figure of expression from the data instance of original 8 * 8 block of pixels of image data extraction.

Be divided into the zone of pixel value " 10 " and the zone of pixel value " 222 " in the view data shown in this Figure 50.It is the view data (for example character data etc.) with strong relatively contrast.

At first, with reference to Figure 51 to Figure 54 the situation of handling view data shown in Figure 50 by common JPEG is described.

Figure 51 carries out the result's of level shift and DCT chart to original 8 * 8 block of pixels data shown in Figure 50 for expression.

In addition, Figure 52 is the chart of the result of blocks of data after quantification shown in expression Figure 51.

In addition, Figure 53 carries out re-quantization, inverse DCT and level shift for expression successively to the blocks of data shown in Figure 52, and is rounded to the result's of the data in 0 to 255 scope chart.

Figure 54 is for deducting the result's of original pixels blocks of data shown in Figure 50 chart the block of pixels data of having represented from the carrying out shown in Figure 53 decoding.

The summation of the absolute value of the difference shown in Figure 54 becomes 685, and the code length behind the JPEG coding is 193.

On the other hand, Figure 55 to Figure 59 is illustrated in the example when utilizing reversible LPF to carry out the JEX processing.

At first, Figure 55 is a chart of representing the example of the pattern information extracted in the information from the head and filtering positional information.Example shown in this Figure 55 is corresponding with example shown in Figure 44.

Can be that vertical filtering arithmetical operation or horizontal filtering arithmetical operation are distinguished to arithmetical operation according to the pattern information of the Mode vector extraction of information from the head, but also can be according to the position that will determine to carry out filtering arithmetic operation from the information of vectorial filtering position kc that extracts of Pos and lr.

Then, Figure 56 carries out the result's of 8 low-pass filtering treatment shown in Figure 55 chart to original 8 * 8 block of pixels data shown in Figure 50 for expression.Can find that from the result shown in this Figure 56 the pixel with pixel value " 10 " is less, and reduce HFS on the whole.Notice that as understandable from Figure 56, the pixel value after the low-pass filtering treatment becomes the value with decimal point, this is owing to carry out low-pass filtering treatment by floating-point operation arithmetic, thereby with higher precision pixel value is mixed.

Subsequently, Figure 57 carries out the result's of level shift and DCT chart for 8 * 8 block of pixels data of having represented to the carrying out shown in Figure 56 low-pass filtering.As can be seen, this Figure 57 and Figure 51 are compared, the DC component remains unchanged under the contrary effect of low-pass filtering and is " 116 ".In addition, can see and reduced high frequency coefficient significantly.

Subsequently, the result's that the blocks of data shown in Figure 57 quantized for expression of Figure 58 chart.

In this example, use the scale factor SF (that is, using original quantization table itself) that is set to " 1 " to quantize.Can see, relatively shown in the result shown in this Figure 58 and Figure 52 as a result the time, the part with " 0 " (zero) high frequency coefficient increases.Therefore, can expect and obtain higher compression ratio.

In addition, Figure 59 is that its expression is carried out re-quantization, inverse DCT and level shift to the blocks of data shown in Figure 58 and carried out contrary LPF and be rounded to the result's of the data in 0 to 255 scope chart.

From the carrying out shown in this Figure 59 deduct original pixels blocks of data shown in Figure 50 the block of pixels data of decoding, and the summation of its result's absolute value is 522.This clearly illustrates that, compares with the JPEG decoded results shown in Figure 53 (summation of the absolute value of difference is 685), has improved picture quality.

In addition, the code length behind the JEX coding is 125, and the bit length of header information shown in Figure 45 is 44, thereby is total up to 169.Even comprise the header information part, 193 the code length that this also encodes less than traditional JPEG, thereby compression ratio is provided.

According to the second such embodiment,, also can improve identical advantage usually with above-mentioned first embodiment even by carrying out reversible low-pass filtering treatment as the conversion of reversible graph image space.

[the 3rd embodiment]

Figure 60 to Figure 63 represents the third embodiment of the present invention, the flow chart of the example that the JEX master that Figure 60 makes up low-pass filtering and displacement for expression handles.In the 3rd embodiment, the part identical with the foregoing description 1 and 2 is additional identical Reference numeral, thereby omits description of them.Describing will be main only at difference.

In the 3rd embodiment, carried out such setting, wherein can optionally carry out as in described displacement of first embodiment and as any favourable a kind of processing in the described low-pass filtering treatment in a second embodiment.Therefore, this shows and some 8 * 8 block of pixels is replaced, and other 8 * 8 block of pixels are carried out low-pass filtering treatment in a view data.

Note, carry out the main processing of the JEX shown in this Figure 60 (LPF+ displacement), rather than the main processing of the JEX (LPF) among the step S62 shown in Figure 37 of above-mentioned second embodiment.Therefore, when carrying out the main processing of this JEX (LPF+ displacement), precondition is to have finished JEX (LPF) preliminary treatment.

When this handles beginning, calculate header information Ph (n) and based on JEX (displacement) carried out the f that replaces (k, l) _P, and the summation Abs (n) and the Huffman code length Bit (n) (step S131) of the absolute value of calculating difference.

Next, calculate header information Phl (n) and based on JEX (LPF) carried out the f that LPF handles (k, l) _L, and calculate the absolute value of difference summation Abs (n, η, SF) and Huffman code length Bit (n, η, SF) (step S132).

Subsequently, determine whether Method is 1 (step S133).If Method is 1 (that is, having selected high compression ratio), whether the Huffman code length Bit (n) that then determines displacement is less than Huffman code length Bit (n, η, SF) (the step S134) of LPF.

And in step S133, if Method is not 1 (that is, having selected high image quality), whether the summation Abs (n) of absolute value of difference that then determines displacement is less than summation Abs (n, η, SF) (the step S135) of the absolute value of the difference of LPF.

If in step S134, the Huffman code length Bit (n) of displacement is equal to or greater than the Huffman code length Bit (n of LPF, η, SF), perhaps in step S135, the summation Abs of the absolute value of the difference of displacement (n) is equal to or greater than the summation Abs (n of absolute value of the difference of LPF, η, SF), then export the f that handles through LPF (k, l) and header information Phl (n) (step S136).

On the other hand, if in step S134, the Huffman code length Bit (n) of displacement less than the Huffman code length Bit of LPF (n, η, SF), perhaps in step S135, the summation Abs (n) of the absolute value of the difference of displacement less than the summation Abs of the absolute value of the difference of LPF (n, η, SF), then the f after the output displacement (k, l) and header information Ph (n) (step S137).

Like this, after carrying out step S136 or step S137, finish the main processing of this JEX (LPF+ displacement).

Note, as mentioned above, each 8 * 8 block of pixels is carried out processing shown in Figure 60.

Then, Figure 61 is the chart of the example of the expression header information that will produce in the step S137 of Figure 60, and Figure 62 represents the chart of the example of the header information that will produce in the step S136 of Figure 60.

The header information Ph (n) that produces when replacing is identical with the header information Ph (n) shown in Figure 25 of first embodiment, and the header information Phl (n) that produces when carrying out LPF is identical with the header information Phl (n) shown in Figure 45 of second embodiment, except and then writing down classification Cat in both cases after the designator Ind.In the example shown in Figure 61 and Figure 62, classification Cat is registered as 1 information, respectively, when carrying out replacement Treatment, " 0 " is recorded as place value, and when carrying out the LPF processing, " 1 " is recorded as place value.

Notice that in the above description, (k, η), (k, η) (k, in the time of l), (k, l) particular row or particular column have low-pass filter effect to this LPF matrix L to f at the f that is applied to be represented as matrix to have adopted all LPF matrix L as shown in Equation 47.Yet, be not limited to such matrix as the matrix of LPF, but for example can adopt as shown in the formula the LPF matrix L shown in 57 _T(χ).

[formula 57]

Here, χ is the parameter of expression filtering strength, and ξ=1-2 χ.In addition, to parameter χ applying condition (1/3)＞χ 〉=0.

This LPF matrix L _T(χ) whole 8 * 8 block of pixels had low-pass filter effect.(k has the LPF effect between being expert at when l) carrying out matrix operation, and (k has the LPF effect when l) carrying out matrix operation between row from the right side to f from the left side to f for it.And the example of the representative value of χ comprises (1/8), (1/6), (1/4) etc.By χ being defined as approximately or equaling 0 and, make L less than (1/3) _T(χ) reversible.Note, adopting this L _TUnder the situation (χ), the DC coefficient no longer is constant, but variation is to a certain degree arranged.

Figure 63 is the chart that is illustrated in the example of the header information when adopting above-mentioned low-pass filtering.

For adopting above-mentioned LPF matrix L _THeader information Phm (n) when (χ) multiply by total LPF, and then designator Ind (=1) and classification Cat (=1) are provided for the zone of data M ode afterwards, and for example " 11 " are recorded as its place value.

And parameter χ is registered as global parameter, thereby it is not included in the header information Phm (n) of each piece.Therefore, it all is constant for entire image that this example is constructed to filtering strength, and does not change according to piece.

According to described the 3rd embodiment, can provide the advantage identical with embodiment 1 and embodiment 2.In addition, because can select more suitably processing among JEX displacement and the JEX LPF, thereby can realize higher picture quality and higher compression ratio at each piece.

In addition, be used for entire image is carried out the matrix of low-pass filtering, make this processing need carry out less arithmetical operation by employing.This feasible processing can be faster, and reduced the required arithmetical operation load of this processing.

Note, the invention is not restricted to the foregoing description, and much less, under the situation that does not break away from the spirit and scope of the present invention, can carry out various modifications and application.

Claims (36)

1, a kind of method for encoding images, this method for encoding images comprises:

View data is carried out spatial frequency transforms, to obtain conversion coefficient;

With described transform coefficients encoding is coded data;

Produce the head part of described coded data, this head partly comprises the information relevant with described view data;

Wherein, before carrying out described spatial frequency transforms, but described view data is suitable for the inverse spatial transform of described view data; And

But the header information that the inverse transformation of described inverse spatial transform is required adds in the described head part,

Wherein, but described inverse spatial transform is selected from replacement operator and reversible low-pass filtering operation, but and described inverse spatial transform suppressed the high frequency conversion coefficient that causes owing to described spatial frequency transforms.

2, method for encoding images according to claim 1,

Wherein, the coding of described conversion coefficient makes the data volume of high frequency conversion coefficient reduce manyly than the data volume of low frequency transform coefficient.

3, method for encoding images according to claim 2,

Wherein, the coding of the described conversion coefficient application quantization step bigger than the quantization step that is used for described low frequency transform coefficient is quantized quantizes described high frequency conversion coefficient; And

But select described inverse spatial transform adaptively in response to described view data, to suppress the value of described high frequency coefficient.

4, method for encoding images according to claim 1,

Wherein, described view data is divided into a plurality of image blocks; And wherein

But at least one image block in the described image block is carried out described inverse spatial transform and described spatial frequency transforms.

5, method for encoding images according to claim 4,

Wherein, but but described inverse spatial transform comprises a plurality of inverse spatial transforms, but wherein each piece inverse spatial transform all is applied to a corresponding image block in the described image block;

But the described image block that has carried out described inverse spatial transform is carried out described spatial frequency transforms;

But the header information that is added comprises the information that the inverse transformation of described inverse spatial transform is required; And wherein

But described inverse spatial transform is suitable for corresponding image block.

6, method for encoding images according to claim 5,

Wherein, but before corresponding image block is carried out the piece inverse spatial transform, but determine whether and will carry out described inverse spatial transform corresponding image block;

If describedly determine it is sure, but then described image block is carried out described inverse spatial transform.

7, method for encoding images according to claim 4,

Wherein, in the described image block described at least one be expressed as matrix M, and wherein

Described replacement operator is expressed as the line replacement that comprises described matrix M and the computing of at least a displacement in the column permutation.

8, method for encoding images according to claim 4,

Wherein, in the described image block described at least one be expressed as matrix M, and wherein

Described replacement operator is expressed as the linear matrix computing

M→AMB，

Wherein, A and B represent invertible matrix.

9, method for encoding images according to claim 8,

Wherein, a matrix in described matrix A and the matrix B is to be used for by matrix operation matrix M being carried out the matrix of line replacement, and another matrix is the matrix that is used for by matrix operation matrix M being carried out column permutation.

10, method for encoding images according to claim 7,

Wherein, described line replacement is characterised in that, to the line replacement of advancing of matrix M, makes the summation of arranging the value of the matrix element in each row on column direction with descending or ascending order; And

Wherein, described column permutation is characterised in that, the row of matrix M are replaced, and makes on line direction the summation of arranging the value of the matrix element in each row with descending or ascending order.

11, method for encoding images according to claim 4,

Wherein, described conversion coefficient comprises and the proportional DC coefficient of the mean value of described at least one image block, and

But described inverse spatial transform keeps described DC coefficient constant thus.

12, method for encoding images according to claim 5,

Wherein, described image block is expressed as matrix;

But selection will be at least one the piece inverse spatial transform that carries out in the described image block; And wherein

The header information that is added comprises and is used for determining described at least one image block has been selected the required information of any operation in the described operation.

13, method for encoding images according to claim 1,

Wherein, but regulate parameter by at least one and regulate described inverse spatial transform; And

Wherein, described header information comprises described adjusting parameter.

14, method for encoding images according to claim 13,

Wherein, described view data is divided into a plurality of image blocks; And wherein

But at least one image block in the described image block is carried out described inverse spatial transform and described spatial frequency transforms.

15, method for encoding images according to claim 14,

Wherein, in the described image block described at least one be expressed as matrix M, and wherein

Described reversible low-pass filtering operation is expressed as one of following linear matrix computing:

M→AM

M → MB, and

M→AMB，

Wherein A and B represent invertible matrix.

16, method for encoding images according to claim 15,

Wherein, described matrix A and matrix B are the matrix that is used to realize to the low pass spatial filtering of the image block represented by described matrix M.

17, according to the described method for encoding images of claim 13,

Wherein, but when described inverse spatial transform was reversible low-pass filtering operation, described adjusting parameter comprised that the filtering of the filtering strength that is used to regulate described reversible low-pass filtering regulates parameter.

18, method for encoding images according to claim 17, this method for encoding images also comprises:

The quantization parameter that utilization is included in the described head part quantizes described conversion coefficient, wherein;

Regulate the described quantization parameter of parameter regulation in response to described filtering.

19, method for encoding images according to claim 13,

Wherein, described conversion coefficient comprises and the proportional DC coefficient of the mean value of described view data, and

But described inverse spatial transform keeps described DC coefficient constant thus.

20, method for encoding images according to claim 13, this method for encoding images also comprises:

Before being coded data, described transform coefficients encoding selects coding mode; And

According to described coding mode in a selected corresponding standard of coding mode select to regulate parameter.

21, method for encoding images according to claim 18, this method for encoding images also comprises:

Before being coded data, described transform coefficients encoding selects coding mode; And

According to described coding mode in a selected corresponding standard of coding mode select described adjusting parameter and described quantization parameter.

22, method for encoding images according to claim 21,

Wherein, described coding mode comprises high compression mode and high quality mode.

23, method for encoding images according to claim 17,

Wherein, but described inverse spatial transform has realized having the low pass spatial filtering that can select direction characteristic; And

Described header information also comprises and the relevant information of direction characteristic of actual selection.

24, a kind of method for encoding images that is applied to the JPEG encoding process, this method for encoding images comprises

A) with image division be a plurality of 8 * 8 image blocks;

B) each image block in described 8 * 8 image blocks is carried out discrete cosine transform, thereby obtain the DCT coefficient, described DCT coefficient comprises DC coefficient, low frequency AC coefficient and the high-frequency AC coefficient of the mean value of representing described 8 * 8 image blocks;

C) utilize quantization table that described DCT coefficient is quantized;

D) utilize run length coding, RLC and Huffman subsequently coding to encode through the DCT coefficient that quantizes to described; And

E) produce the head part,

Described method comprises:

Before carrying out described discrete cosine transform, but at least one 8 * 8 image block is carried out inverse spatial transform, but select described inverse spatial transform adaptively in response to described at least one 8 * 8 image block thus;

Extract the required spatial alternation parameter of inverse transformation of described spatial alternation; And

The header information of the described spatial alternation parameter of expression is added in the described head part,

But wherein said inverse spatial transform is selected from replacement operator and reversible low-pass filtering operation, but and described inverse spatial transform suppressed the high frequency conversion coefficient that causes owing to described spatial frequency transforms.

25, method for encoding images according to claim 24,

Wherein, but the described inverse spatial transform of selecting adaptively be characterised in that

No matter whether carry out described spatial alternation, described DC coefficient all remains unchanged; And

Increase the amplitude of described low frequency AC coefficient by carrying out described spatial alternation.

26, a kind of picture decoding method that is used for comprising that head part and coded data coded image data are partly decoded, this picture decoding method comprises:

Read the described head part of described coded image data, to extract the information relevant with view data;

Described coded data is decoded, to obtain the conversion coefficient in the spatial frequency domain;

Described conversion coefficient is carried out contrary spatial frequency transforms, thereby obtain second view data;

Described coding/decoding method also comprises:

But extract the required information of inverse transformation of inverse spatial transform;

But described second view data is carried out the inverse transformation of described inverse spatial transform, obtaining described view data,

But wherein said inverse spatial transform is selected from replacement operator and reversible low-pass filtering operation, but and described inverse spatial transform suppressed the high frequency conversion coefficient that causes owing to described spatial frequency transforms.

27, picture decoding method according to claim 26, this picture decoding method also comprises:

But before the required information of the inverse transformation of extracting inverse spatial transform, but be identified in the required information of inverse transformation that whether has described inverse spatial transform in the described head part; And

But if recognize the required information of the inverse transformation of described inverse spatial transform, but then described second view data is carried out the described inverse transformation of described inverse spatial transform, to obtain described view data; And

But, but then do not carry out the described inverse transformation of described inverse spatial transform and export described second view data as described view data if do not recognize the required information of the inverse transformation of described inverse spatial transform.

28, a kind of moving image compression method, this moving image compression method is used to utilize based on compressing in the frame of the view data in the picture frame and compressing based on the interframe of a plurality of picture frames, the motion image data that comprises a plurality of picture frames that are time series formula relation is encoded

Wherein, compression comprises in the described frame: picture frame is divided into a plurality of image blocks; Described image block is carried out spatial frequency transforms, to obtain conversion coefficient; And produce the head part that comprises the information relevant with described picture frame, compression also comprises in the described frame:

Before carrying out described spatial frequency transforms, but in the described image block at least one carried out inverse spatial transform; And

But the header information of the spatial alternation parameter that the inverse transformation that comprises described inverse spatial transform is required adds in the described head part,

But wherein said inverse spatial transform is selected from replacement operator and reversible low-pass filtering operation, but and described inverse spatial transform suppressed the high frequency conversion coefficient that causes owing to described spatial frequency transforms.

29, moving image compression method according to claim 28, this moving image compression method also comprises:

But before carrying out described inverse spatial transform, but determine whether to carry out described inverse spatial transform.

30, a kind of moving image compression method, this moving image compression method is used to utilize based on compressing in the frame of the view data in the picture frame and compressing based on the interframe of a plurality of picture frames, the motion image data that comprises a plurality of picture frames that are time series formula relation is encoded

Wherein, described inter-frame compression is drawn together: picture frame is divided into a plurality of block of pixels; By obtaining the difference between at least one and the respective pixel piece relevant in the described block of pixels, produce image block with the picture frame that will compare; Described image block is carried out spatial frequency transforms to obtain conversion coefficient; And produce the head part that comprises the information relevant with described picture frame, described interframe compression also comprises:

Before carrying out described spatial frequency transforms, but at least one image block in the described image block is carried out inverse spatial transform; And

But the header information of the spatial alternation parameter that the inverse transformation that comprises described inverse spatial transform is required adds in the described head part,

But wherein said inverse spatial transform is selected from replacement operator and reversible low-pass filtering operation, but and described inverse spatial transform suppressed the high frequency conversion coefficient that causes owing to described spatial frequency transforms.

31, moving image compression method according to claim 30, this moving image compression method also comprises:

But before carrying out described inverse spatial transform, but determine whether to carry out described inverse spatial transform.

32, moving image compression method according to claim 30,

Wherein, described head partly comprises the information relevant with the group that comprises a plurality of picture frames;

And wherein, described header information comprise with from the relevant spatial alternation parameter of a plurality of picture frames of described group.

33, a kind of being used for decodes to obtain the dynamic image decoding method of motion image data to the encoding moving pictures data, and these encoding moving pictures data comprise head part and coded data part, and described dynamic image decoding method comprises:

Read described head part, to extract the information relevant with view data, this view data is relevant with picture frame;

Described coded data is partly decoded, to obtain the conversion coefficient relevant with described picture frame;

Described conversion coefficient is carried out contrary spatial frequency transforms, thereby obtain second view data;

Described coding/decoding method also comprises:

But the required header information of inverse transformation from described head extracting section inverse spatial transform;

But described second view data is carried out the inverse transformation of described inverse spatial transform, with the acquisition view data relevant with described picture frame,

But wherein said inverse spatial transform is selected from replacement operator and reversible low-pass filtering operation, but and described inverse spatial transform suppressed the high frequency conversion coefficient that causes owing to described spatial frequency transforms.

34, a kind of picture coding device, this picture coding device comprises:

Spatial transform unit, but be used for view data is suitable for the inverse spatial transform of described view data, but wherein said inverse spatial transform is selected from replacement operator and reversible low-pass filtering operation;

The spatial frequency transforms unit, but be used for the described view data of carrying out described inverse spatial transform is carried out spatial frequency transforms, thus obtain conversion coefficient;

Head part generator is used to produce the head part that comprises the information relevant with described view data; And

The header information adding device, the information that is used for the spatial alternation parameter that the inverse transformation that comprises described spatial alternation is required is added described head part to,

Wherein, but described inverse spatial transform has suppressed the high frequency conversion coefficient that causes owing to described spatial frequency transforms.

35, a kind of picture decoding apparatus is used for the coded image data that comprises head part and coded data part is decoded, and described picture decoding apparatus comprises:

Contrary spatial frequency transforms unit is used for the conversion coefficient that produces from described coded image data is carried out contrary spatial frequency transforms, thereby obtains second view data,

The head interpreter is used for partly reading described spatial alternation parameter from described head; And

The inverse spatial transform unit is used for according to described spatial alternation parameter described second view data being carried out inverse spatial transform, thereby obtains view data,

Wherein, described inverse spatial transform is selected from the inverse transformation of replacement operator and the inverse transformation of reversible low-pass filtering operation, and described replacement operator can suppress the high frequency conversion coefficient that causes owing to spatial frequency transforms.

36, a kind of image encoding-decode system, this image encoding-decode system comprises:

Picture coding device according to claim 34; And

Picture decoding apparatus according to claim 35;

Wherein

Described picture decoding apparatus can be decoded to the coded image data that has carried out coding by described picture coding device.

Images