RU2093968C1 - Method for encoding and decoding images and device which implements said method - Google Patents

Abstract

FIELD: digital processing of video signals. SUBSTANCE: method involves adapting sampling of vectors together with pipe-line processing of input information and partial parallel computation. This results in possibility to achieve real-time operation which is close to TV frame sequence. EFFECT: decreased volume of video information to be transmitted through low bandwidth communication channels. 2 cl, 1 dwg

Description

 The invention relates to the field of image signal processing and can be used, in particular, for multiple compression (encoding) of an image in real time, followed by restoration (decoding) of the compressed image to its original state while preserving its small details, features and the other contained in the original image of relevant information.

 The image compression process can be used when transmitting aerospace video information from an aircraft to the Earth via narrow-band communication channels in real time; when transmitting information (images) in computer communication networks; on existing narrowband and telephone communication channels; economical archiving of aerospace, medical, forensic and other video information; high-speed transmission of large amounts of data between information storage devices and computer video terminals, as well as in a number of other applications of science and technology.

 There are a number of known methods of "compression-recovery" of images. For example, adaptive differential code-pulse modulation and predictive coding methods [1] provide a simple hardware implementation of the system for real-time operation. This method provides compression by 4-5 times, but the quality of image recovery substantially depends (decreases) on the noise level in communication channels.

 Compression methods based on orthogonal transformations such as the Fourier, Hadamard, Carrunen transforms [1] and others are resistant to pulsed and high-frequency noise communication channels. However, compression with good quality through these methods is achieved only for simple images and does not contain a large number of small details.

 To improve the quality of recoverable images compressed using orthogonal transformations, block coding is used [1]. In this case, not the entire image is compressed at once, but its parts into which it is divided (blocks) containing a certain (in particular, equal) number of pixels . This method is widely used in encoding television images in commercial video recording and television broadcasting systems, as well as in the creation of computer video films and is the basis for JPEG, MPEG image compression standards used in many countries.

The JPEG standard is the most common compression method and represents the following sequence of operations:
a / pii images in digital form and its buffering;
b / discrete cosine transform (DKP), in which the image is divided into blocks and each block undergoes the mentioned operation. This operation requires a lot of computing power, and information loss does not occur. D.K.P. It is a type of Fourier transform, has the inverse transform. The meaning of this operation is to switch from the spatial representation of images to its spectral representation and vice versa. Cutting off the most high-frequency elements of the converted image, it is possible in one direction or another, depending on the quantity, to choose between image quality (restored) and the compression ratio. Thus, D.K.P. converts a matrix representing the original image into a set of frequency coefficient matrices. The high-frequency elements of these matrices can be roughened or discarded without any special loss of information. The time required to calculate each element of the matrix D.K.P. strongly depends on its size, which JPEG recommends in the form of elementary matrices with a size (8x8) of elements;
in / quantization. After the operation D.K.P. each image block undergoes a quantization operation, i.e. divided with subsequent rounding by the corresponding element of a special quantization matrix. Here the main loss of information occurs due to rounding, in which the division result is rounded to the nearest integer with a fixed number of binary digits. The matrix obtained after quantization in the high-frequency region usually turns out to be filled with zeros;
g / further, RLE coding of the obtained matrices (repetition coding) is performed, at which the remaining frequency coefficients are encoded. The coefficients of each matrix are thus zigzagged, zero gaps are also encoded using the standard RLE program. Further, information from all blocks is formed into a single stream;
d / coding of "entropy". The stream received during RLE-coding is subjected to the standard compression method according to Huffman or arithmetic coding;
e / when decoding, everything happens in exactly the opposite order, i.e. after decoding by Huffman, decoding by RLE program follows, then restoration of the matrix of D.K.P. by means of the reverse D.K.P. finally, assembling the image from the restored blocks.

 Application of the considered method, depending on the complexity of the processed image, allows us to achieve a compression ratio of 10 to 60. However, with large compression ratios, the boundaries of pixel groups (blocks) begin to appear on the reconstructed image [1], while the reconstructed image itself will consist of a large number easily distinguishable monotonous spots, which makes, for example, viewing videos using this compression method tiring for vision. There is a loss of the fine structure of the processed image, therefore, it turns out to be difficult to use this approach to compress images containing a large number of details, for example, aerial photographs.

 An alternative to orthogonal transform block coding methods is image compression by adaptive vector quantization [4] In this case, the image is also divided into blocks and the set of pixels in the block is described by a vector. Adaptive quantization of vectors (A.K.V.) provides encoding of the mutual characteristics of pixels in the encoding block; therefore, the reconstructed image after such encoding preserves small details with a sufficiently large compression (up to 16 times or more).

 The analogues considered above are quite effective in their fields. However, none of them provides work in a fairly wide range of practical applications. Such areas of application include image compression applications where a combination of the advantages of all the above methods and, above all, in relation to reconstructed images in real time using existing narrow-band communication channels is necessary.

 a / Transmission of compressed images through communication channels. This area has become especially relevant in recent years, when the use of space-based visual observation means is expanding and there is a need to transmit large amounts of video information via space communication channels with limited bandwidth at the rate of operation of high-speed space-based video sensors.

 b / Local and global computer information networks. Communication between computers within such networks is often realized via significant narrow-band (including telephone) communication lines, and therefore the use of compression-recovery systems facilitates the transfer of large amounts of video data within such networks.

 c / Computing image processing systems have a bottleneck for transmitting video data from computer drives to graphic video terminals. Such transmissions are usually carried out on standard digital communication channels having a fairly narrow bandwidth. Therefore, the use of effective compression and restoration of video data increases the speed of image analysis in such computing systems.

 g / A separate problem is the archiving of large amounts of computational data presented in digital form. It can be a bank of space reconnaissance video data, a bank of medical images, forensic video data, etc. The use of effective "compression-recovery" of stored images allows reducing the required memory resources of such computing systems by 1.5-2 orders of magnitude.

 The closest to all these requirements are the last two methods considered, i.e. JPEG and AKV. The comparison of these methods showed that JPEG has a fairly good quality of the reconstructed images with a compression ratio of up to 15, but after 10 small details disappear. The AKV method gives results comparable to JPEG, but with compression ratios of the order of 10, it preserves fine details. At high compression ratios, the AKV method shows better results than JPEG, so it is taken as a prototype. Detuning from the main drawback of the AKV method, that is, eliminating the influence of noise in the communication channel, can be accomplished using principles for image compression borrowed from Kohonen neural networks [5] when used in image compression problems.

 It was decided to use the classical AKV method and modify it so that it retains the topology of the encoded data like Kohonen neural networks. To implement this, the following is required. Just as for the stable learning of Kohonen neural networks, the initial position of the clusters must be specially ordered so that, as the ordinal number of the cluster increases, the values of the spatial coordinates of its center change monotonously. For this, the initial cluster centers were arranged along one diagonal n-dimensional vector space, and the cluster numbers varied monotonously along the diagonal. Further, during the process of adaptation of the position of the clusters described above, the initial topology of their location is distorted, clusters with adjacent numbers can appear in different angles of the quantized vector space. To avoid this, after the elimination of each empty cluster and its opening near the largest of the remaining ones, the numbers of all clusters are reassigned as follows. The newly opened cluster is given a number adjacent to the number of the cluster to be split, while the previous or subsequent cluster numbers (depending on whether the number of the eliminated cluster was greater than or less than the distributed) is shifted by one, so that with the numbers of existing ones and the vacant number The liquidated cluster was attributed to one of the remaining nonempty ones. As a result, sustainable methods for training Kohonen neural networks for image compression-recovery were developed. With 16-fold compression, the reconstructed image retains the structure and small details of the original, and the root-mean-square error of image restoration is almost the same as the classical AKV method when processing images on which this method was trained. If the neural network is used to compress images that were not previously presented to it, the quality of its work exceeds the quality of work according to the method of A.K.V.

 Thus, the classical method of A.K.V. based on the preliminary construction of code tables (code books, code libraries, standard libraries) based on the learning process (adaptation) to compress specific classes of images. At the same time, the libraries used do not preserve the topology of the encoded data, which leads to large errors in decoding caused by small errors in the encoded data that occur, for example, during transmission over communication channels. The proposed learning process saves the indicated topology (vectors that are similar in content have similar numbers in the code library) and proceeds as follows. The original image is divided into blocks of pixels of a certain size K • L, each of which is interpreted as a vector in n K • L-dimensional space. It is necessary by the adaptive clustering method to divide this space into "m" clusters so that the cluster density corresponds to the probability density of vectors in the specified vector space. Before the beginning of adaptive clustering, the centers of the indicated “m” clusters are assigned and are placed in this vector space in some orderly manner. Then, based on an analysis of a sample of training data (n-dimensional vectors, i.e., all pixel blocks of the encoded image), the position of these centers is adapted to the statistical properties of the encoded data. Moreover, for each of the presented vectors, the deviation from the centers of the mentioned clusters is determined by the criterion of the minimum Euclidean distance. For each cluster, the new center position is calculated as the average of all vectors falling into the cluster, empty clusters are eliminated, and new clusters with this number are created with a new spatial position by halving the largest of the remaining clusters. In this case, one of the halves is assigned the number of the parent cluster, and the other is the number of the newly created (eliminated in another place vector space). The center of the mother cluster is taken as these two new clusters, to the initial coordinates of which are added two small random n-dimensional oppositely directed vectors (thus, the centers of the newly created clusters are spread in space). An important point in this case is the preservation of the previously adopted cluster arrangement order in the resulting code library, therefore, after the formation of a new cluster, they are reordered in order to maintain the established order. After this, the training sample of vectors is again presented for clustering and this process is repeated iteratively until the criterion for its completion is met, for example, until the moment when the change in the coordinates of the centers of the clusters does not become less than the specified threshold.

 Another important distinguishing feature of the proposed compression method is the use of the pseudo-floating point mode when storing and using reference libraries. The point is, studies have shown that when preparing code libraries by any of the methods used, the best results in the quality of images recovered after compression are achieved when the values of the library vectors are calculated and used in a floating-point format. However, this is very computationally inefficient when implementing the encoding procedure. In the present invention, it is proposed to use a library prepared in floating point mode for encoding. Then the obtained values of the vectors are multiplied by 64 and rounded to the nearest integer. In this case, firstly, double-byte numbers with 14 significant digits are used, which well emulate the operating mode with a floating point. Secondly, the two remaining bits (missing up to two bytes) are sufficient so that when calculating the minimum Euclidean distance during the encoding, the sum of the square of the differences of the vectors (reference and encoded) does not exceed two bytes, which provides a convenient software and hardware implementation of the algorithm. Thirdly, only integer arithmetic is used in the calculations. Naturally, when coding is performed, the values of the image vectors are preliminarily shifted by 6 bits.

 In the coding process, by determining the smallest Euclidean distance between the current vector and the reference, each coding block of image pixels is assigned the “closest to it” reference vector from the coding table, that is, the 16-component input vector is replaced by a single-component reference vector number. Thus, 16-fold image compression is performed, and in fact it occurs due to the reduction of statistical redundancy of video data due to the corresponding “training” of the table of reference code vectors.

 After transmission over the communication channel or "archiving", the compressed image can be decoded. Moreover, the value of each pixel of the decoded image is the address of the entrance to the decoding table, which stores the corresponding values of the reference vectors, the same as in the encoding table, i.e. the number of the reference vector is replaced when decoding by a 4 x 4 pixel block.

 It should be noted that this method has little sensitivity to the discrepancy between the encoded images and the images on which the coding tables were trained, therefore, the coding tables recorded once were suitable for compressing a wide class of images (for example, a class of space photographs, portraits, etc.). The formation of coding tables and tables of reference vectors is carried out in stationary conditions using a computer or an existing neural network. Then they are embedded in the ROM of the compression-recovery processes and are used for regular operation. If necessary, the class of encoded images can be replaced by replacing the ROM, or it is possible to organize the use of many encoding tables using narrower classes of encoded images (to increase the accuracy of recovery).

Thus, the proposed method eliminates the disadvantages and significantly expands the functionality of the classical AKV method. This is achieved by using Kohonen’s principle of neural networks for compression and creating a library of standards based on them that preserves the topology of the encoded data, which makes the proposed method stable against a wide class of images and allows to achieve the following technical results:
а / finding in the coding process by determining the minimum Euclidean distance for each image block closest to it from the reference library;
b / replacement of a multicomponent input image vector by a single-component number of the reference vector.

c / Restore the original vector after transmitting the encoded image by the number of the reference vector, the block content of which is called from the library of standards;
g / method has a higher decoding speed in relation to the described analogues, which makes it convenient when playing moving images.

 The drawing shows a functional diagram of an encoding-decoding device.

 The encoding-decoding device comprises: data selection register 1, first and second blocks of random access memory 2 and 3, first switch 4, random access memory unit of standards 5, subtraction unit 6, quadrator 7, control unit for selecting standard 8, first, second, third, the fourth drives 9-12, the first, second, third adders 13, 14, 15, the decision block 16, the output register 17, the third, fourth blocks of RAM 18, 19, the second switch 20, the block generating the addresses of the standards 21, the unit of formation and counting addresses of reference 22, RAM unit of reference 23, you a register 24.

 The encoding-decoding device operates as follows. The input image in the form of a digital sample of input data enters the first and second blocks of RAM 2 and 3 on 4 lines to provide information for subsequent blocks. Next, in the subtraction unit 6, the difference between the reference and current sample values of the image signal is formed. Next, in squared 7, a signal of the difference square is formed in parallel with 4 lines (from 1 to 4 samples). The results of the calculations for further processing arrive at 8 channels simultaneously. The results are accumulated in drives 9, 10, 11, 12. Since the size of the encoded block is 4x4, the square of the difference for the entire vector is obtained in 2 passes. The next 8 samples are read from the buffer. The received eight amounts are transferred to the summation block, where they are added up twice in pairs to obtain the total amount. The result is stored in decision block 16 and is used to select the minimum vector value in the library of coding standards. Further, the procedure is repeated for a given block of the encoded image until the required part of the library of standards is processed and the code of this block of the image is selected. Thus, the architecture is built according to the conveyor principle using inside parallel processing nodes. The cycle of the pipeline structure is determined by the longest data processing time in one of the nodes of the pipeline. After determining the closest vector from the library of standards, an encoded image string is formed. The coding block is implemented using the following elements: input data selection register, switches, control and sample block chips of the 1533, 1564 series, random access memory blocks 1 and 2 of the type MS6264-20NS chip from Texas Instruments, subtraction block 6 and quadrator 7, adders 13, 14, 15 on TMS2210 microcircuits, RAM units 5. Reference libraries are built on MS62256-20NC microcircuits, a control processor of the TMS320C10Nh or TMS3220C25FNh type. The input register of the image decoding unit provides communication with the bus, from where information in the form of an encoded image comes from. A line of encoded image enters the buffer memory of blocks 18 and 19, which provide continuous reception and output of data. Through the switch, data from one of the RAMs are sent to the unit for generating the physical address of the standard in memory, where the vector number from the library of standards is converted to the starting address of the location in the RAM space. The starting address of the reference goes to the formation unit and the counter of the memory address. After reading one line of the reference block, the starting address of the next vector is sent to the address generation device, and the line of the reference block is read at this address, etc. The read information is sent from the memory of the standards, goes to the input register and then to the formation of the line of the output image.

 The decoder can be implemented on the following elements: input register, switch, forming unit and reference memory address counter, output register (performed on the basis of 1553, 1554, 55PT17 series microcircuits), standard RAM based on MS62256-20NC (microcircuit based image forming unit series 174 and 1118).

Literature
1. Wintu R.A. Coding of images by means of transformations, TIIER, 1972 vol. 60, N7, p. 69-81.

 2. Radcheev R. Frazier R. Image Compression Tool for Working with the MIR PC Scanner, N 4, 1992 p. 53-45.

 3. Nasrebadi N. M. King R.A. Image Coding Using Vector Quantization A. Review JEEE Trans. on Cjmm 76 vol. 36 (8), 1988, pp. 81-93.

 4. Kun S. Matrix processors on SVIS. Per. from English M. Mir, 1991 p. 672.

 5. US patent N 5010401, HO4N 7/12, 1991. A coding-decoding device using vector quantization.

 6. PCT N WO-90-09079, HO4N 7/12. Method and apparatus for quantization.

Claims (2)

 1. A method of encoding-decoding an image, which consists in receiving a digital image signal, dividing it into blocks containing a set of pixels in each block, which are described by a vector, create a quantized vector space, and form a library from the signal corresponding to the first frame of the image standards by adapting the compression of specific image blocks, the library calculates analogs of standards for each of the image blocks, generates a difference signal between the current signal value and the corresponding the corresponding value from the library of standards, form the signal of the square of the difference between the current value of the signal and the corresponding value from the library of standards, sum the squares of the differences for eight blocks of the signal, the resulting sum of the squares of the differences are added in pairs to obtain the total, store the received code of the image block, when decoding the stored code of the image block is determined by the address of the block in the library of standards, this block is read into the buffer memory with the subsequent formation of the restored image, characterized in that the closest block from the reference library corresponding to each block of the image is found by determining the minimum Euclidean distance using the resulting total, when decoding, replace the multi-component input image vector with a single-component number of the reference vector.  2. A device for encoding-decoding an image, comprising an encoder, including an input data selection register, serially connected reference memory block, a subtraction block, a quadrator and a first drive, as well as a reference selection control unit, the output of which is connected to the input of the reference memory block, a decoder containing an input sample register, series-connected address generation unit, a unit for generating and counting a memory address of a reference, a memory block of reference and an output register, different the fact that the encoder introduced the first and second blocks of RAM on four lines, the first switch, the second, third and fourth drives, the first, second and third adders and the decision block, while the first and second outputs of the input data selection register are connected to the inputs the first and second blocks of RAM, the outputs of which are connected respectively to the first and second inputs of the first switch, the output of which is connected to the second input of the subtraction unit, the output of the quadrator is connected to the inputs of the second, third and fourth drives, the outputs of the first and second drives are connected to the first and second inputs of the first and second drives are connected to the first and second inputs of the first adder, and the third and fourth drives to the first and second inputs of the second adder, the outputs of which are connected to the first and second inputs of the third adder, the output of which is connected to the input of the decision block, the first output of which is the output of the encoder, the second output of the decision block is connected to the input of the control unit by selecting a reference, the second input of the memory block of the reference connected to the input of the device, the third and fourth blocks of RAM on four lines and the second switch are inserted into the decoder, the first and second outputs of the sample register are connected to the inputs of the third and fourth blocks of RAM, the outputs of which are connected respectively to the first and second inputs of the second switch, the output of the second switch is connected to the input of the reference address generation unit, the output of the output register is the output of the device.