RU2227322C2 - Photo and video information encoding method - Google Patents

Abstract

FIELD: photo and video information technology. SUBSTANCE: proposed method designed for use in developing digital coders for digital photographic cameras and video still cameras, video telephony, and digital television involves following procedures: entire current information residing in coder and related to one or more frames is divided during intermediate coding step into two or more groups of different volume; most data are processed inside coder proper and remaining data groups (one or more) are transferred to additional (one or more) remote coding devices (remote server or servers) wherein they are processed according to predetermined coding algorithm and then returned to coder, whereupon all data are finally encoded within this coder. EFFECT: multiple load reduction in data transmission line. 1 cl, 1 dwg, 3 ex

Description

The invention relates to photo and video information technology and may find application in the development of digital encoding devices in digital photo and video cameras, video telephony, video conferencing, standard and high definition television digital broadcasting, processing and storage of video information and photo images.

There are many different photo and video encoders that allow you to encode photo and video materials with a decrease in the amount of information (with digital compression). The best known encoders are JPEG (G.K. Wallace. The JPEG Still Picture Compression Standard Communication of the ACM. - 1991, Vol. 34, N4). JPEG-2000 (ISO / IEC FCD 15444-1: 2000 (V.1.0 16 March 2000) "JPEG 2000 Image Coding System") (for photos) and MPEG-2 (ISO / IEC 13818-2. Information Technology Generic Coding of Moving Pictures and Associated Audio Information. Part 2: Video./ Ed.l JTS I / SC 29, 1994), MPEG-4 (ISO / IEC 14496-2: 1999. "Information technology - Coding of audiovisual objects - Part2: Visual ") (for video).

These encoders are implemented in hardware devices that implement a specific compression algorithm, as well as in software products of various companies installed on a personal computer of the end user. As hardware devices, digital cameras and video cameras, hardware encoders installed in television studios for encoding digital video signals (MPEG-2 encoder COD / CO E-1000 of Tadiran Scopus Corporation. (Www.scoMS.co.il)) and (NEC NEC SmartVision HG / V encoder MPEG-2 encoder (www.nec.com)) optional hardware cards installed in the computer (PV 251 / C-Cube DVXplore - ProVideo corporation MPEG-2 encoder (www.nevton.ru] : Canopus DVStorm MPEG Module-Hardware MPEG-1, MPEG-2 encoder (www. Canoouscorp. Ru)).

Various software drivers for compressing photo or video information can be used as software products (MPEG-2 encoder program LSX ENCODER 3.5 (www.liqos.com): MPEG-2 encoder program MPEG Power Professional V2.0h (www.heuris.com): DivX Pro 5.0 - divX / MPEG-4 codec driver (www.divX.com)).

In the above hardware and software encoders, the encoding of photo or video information is carried out inside these devices without the use of computing and hardware resources remote from these hardware, which may not be owned by a user using a photo or video encoder.

Closest to the proposed invention is a video encoding system in which the user sends digital material to a remote server of the system all the material (video or photo) to be encoded, the server performs full encoding of the material using a specific photo or video encoder, and after some while the user receives ready-made encoded material (LoudEye Corporation’s Internet video encoding system (www.loudeye.com)).

The disadvantage of the above system is the huge amount of data to be transmitted to a remote server, which increases the cost of using the system and limits its performance.

The objective of the present invention is to develop a method of encoding photo and video information, which allows an economical way to transfer a large amount of information to a remote encoder with high performance.

The invention consists in a method for encoding photo and video information, including converting a photo image or a sequence of video frames into digital form, digitally processing in the encoder the values of the brightness and color pixels with decreasing the final amount of information (compression), and at the intermediate stage of encoding, all current information present in encoder and related to one or more frames, is divided into two or more groups having different sizes, most of the data is processed internally three encoders themselves, and a small (remaining) group of data (one or several) is sent to an additional one or more remote encoding devices (remote server or servers), in which it is processed using a specific (specified) encoding algorithm and returned to the encoder, after which the encoder is the final encoding of all data.

Unlike the known system in the proposed method, the data stream transmitted to the remote server is tens and hundreds of times smaller than the size of the source data, which significantly increases the performance and efficiency of the proposed encoding method compared to existing ones.

The encoding system in the present invention also differs from known encoding devices for photo and video information, in which remote computing resources are used only to transmit ready-made encoded information, and the encoding itself is carried out completely inside the device. Such systems include existing video telephones, video conferencing devices, television signal encoders. In these systems, remote computing resources are used only to create a medium for transferring data from the encoder to other devices (to provide communication), as well as to control access to encoded data and the time of use of the data transfer channel (billing).

The drawing shows one of the possible variants of the device for implementing the method of encoding photo and video information.

The device shown in the drawing contains a photo and video information encoder, consisting of a unit 1 converting the source data to the format used for encoding, a unit 2 for dividing data into data that will continue to be processed inside the encoder, and data that will be transmitted to a remote server connected to it block 3, in which the encoding of internal data is carried out, block 6, transmitting data for remote encoding, as well as control functions for connection with remote the server device, unit 4, which processes and multiplexes internal data with data received from the remote server device, and the unit 5 for converting data to the final format, and also contains a remote server device, consisting of unit 7, which controls the connection and providing the reception and transmission of data from the encoder, and transcoding module 8, providing encoding of data coming from the encoder.

The proposed method using the device depicted in the drawing, is implemented as follows.

The initial data arriving at the input of the device fall into block 1 (the block for converting the initial data to the internal format of the encoder), where they are converted into the digital format needed to implement the specified compression algorithm. The conversion may include standard components, such as digitizing an analog signal to digital in the case of an analog input, extracting the luminance component (Y) and two color-difference components (Cr, Cb) from the three color components of the signal (RGB), changing the spatial resolution of the color components Cr, Cb, noise filtering using digital filters. At the output of block 1, the digital data format should correspond to the format required for the compression encoder. For example, in the case of a JPEG encoder, this format is the data format in the form of the luminance component Y and two color-difference components Cr, Cb having a spatial resolution reduced by half.

In block 2 (block separation of encoded data into internal and processed remotely on an external device), a small group is selected from all digital data that will be processed on a remote device. The selection of a data group can be carried out in various ways. For example, a highlighted data group can relate to only one (example 1 below) or several image frames (examples 2 and 3). The selection of a data group may also be preceded by a series of digital transformations carried out in block 2 according to a given coding algorithm. In the case of JPEG and MPEG-2/4 encoders, such transformations are dividing the image into blocks of 16 × 16 and 8 × 8 pixels, motion compensation between frames, calculating the spectrum coefficients of the discrete cosine transform (DCT) in blocks of 8 × 8 pixels. In Example 2 below, a data group (one senior DCT coefficient) is allocated from the DCT spectrum of an entire block of 8 × 8 pixels. Data allocated by block 2 is sent through block 6 (interface module for exchanging data over the network) to a remote device consisting of blocks 7 (interface network module of a remote device) and 8 (transcoding module of a remote device). When sending data in block 6, they are converted to the desired network transfer protocol (for example, TCP / IP). After passing through the network, the selected data passes through the data reception interface 7 to the server unit 8, where it is processed according to the specified algorithm. This processing includes coding elements of a selected data group according to a coding algorithm. In Example 1, such processing is coding of reference frames. In example 2, such processing is quantization and coding with codes of variable length of the aging coefficients of DCT blocks of 8 × 8 pixels.

The data processed on the server is returned to the encoder through the interface unit of the server 7 and the receiving unit of the encoder 6. In this case, in block 7, the connection between the encoder and the server is simultaneously monitored and the connection time and the amount of transmitted data (billing) are taken into account.

Simultaneously with the remote processing of the data on the server in block 3 (the coding block of the internal data) of the encoder, the remaining data is processed after they are separated in block 2. This processing includes the steps of digitally encoding the remaining data according to the encoding algorithm used. In the example 1 described below, in block 3, the frames are fully encoded with motion compensation. In Example 2, in block 3, all coefficients of the DCT spectrum are completely encoded, with the exception of the highest coefficient. Coding includes quantization of coefficients and their coding with codes of variable length. In block 3, in the case of using a video encoder, motion vectors, macroblock types, and quantization coefficients are also encoded according to the MPEG standard.

In the case of video information, block 3 can use the encoding results of data processed remotely on the server. These encoding results come from block 4 (multiplexer of internal and encoded deleted data) to block 3 and are used in block 3 to restore the encoded frame, which is necessary to perform motion compensation when encoding a subsequent frame (group of frames). After the data allocated for remote encoding and the remaining data are encoded separately in blocks 3 and 8, in block 4 they are multiplexed. In this procedure, data encoded on a remote device is written (inserted) into the output stream at specific places that it should occupy according to a given coding standard.

The final processing of the encoded data is carried out in block 5 (converter to the final output format). The output digital stream is generated by adding the required headers, start and control codes.

Example 1. Encoding video information

As a video encoding algorithm, a well-known method is used, based on motion compensation between frames, with periodic encoding of frames without motion compensation (the so-called reference frames).

When coding with motion compensation, the reference frame is the initial frame for encoding on the encoder side and recovery of all frames on the decoder side, without which the processing of intermediate frames (using motion compensation) is impossible. In this example of an encoding method, all or a certain part of the data related to the reference frame is transmitted to the remote device. Since the reference frame repetition rate is usually much lower than the full frame rate, for example, in MPEG-2 television encoders, it is usually 10-15 times (once every 1/2 s at a frame rate of 25 frames / s), and in MPEG-4 video encoders, usually 200-250 times (once every 8-10 s), the amount of data transferred to the remote server device in this example will be 10-200 times less than the amount of the original video material. If only part of the data related to the reference frame is transmitted to the remote device, the data stream will be even smaller. In this example, block 8 in the drawing performs coding functions of the reference frame using the original algorithm, the components of which are absent on the encoder side and without operation of which it is impossible to encode all frames of the video sequence.

Example 2. Encoding photo or video information

In each frame, a small amount of data is sent to the server for encoding and necessary for the formation of the final bit sequence of the output data. This little information may be different and reflect certain features of the compression algorithm used. For example, in the case of using a JPEG codec for photo images or an MPEG-2/4 encoder for video images, such information can be zero (higher) discrete cosine transform coefficients of blocks of 8 × 8 pixels of the original image used in these encoders. In the absence of amplitudes of these coefficients, further coding of the image is impossible. Since the amount of these coefficients is 8 × 8 = 64 times less than the total number of pixels in the frame, the data stream between the encoder and the remote device is much smaller than the initial amount of all encoded information.

Example 3. A system for remote coding of video information based on the original Motion Wavelets video compression algorithm (Artamonov A.E., Sergeev A., Sokolov A.Yu. It is better to see once than hear a hundred times ...., Sum of Technologies. - 2002, No. 1 (9), p. 65; Remote coding system for IDM video companies using the Motion Wavelets algorithm (www.codec.ru).

In this example, we use both the remote coding of the reference frames described above and select only a small part of the data related to the reference frame, which are closely interconnected with the specifics of the encoding according to the Motion Wavelets algorithm. As a result, it was possible to obtain a significant reduction in the flow of data transmitted to a remote device. For example, when using a video resolution of 352 × 176 pixels and a frame rate of 25 frames / s (the source data stream in uncompressed form is approximately 35 Mbit / s), the data stream between the encoder and the remote server device does not exceed 3 kbit / s.

In the general case, using the original encoding algorithm for photo or video information, the data transmitted to the remote encoding server using the proposed method for encoding photo and video information should reflect the specifics of the digital transformations carried out to implement this encoding (compression) algorithm.

The result achieved using the proposed method is expressed in the possibility of encoding photo or video information, in which only a partial transfer of the photo or video encoder component to the user, and the process of encoding photo or video information is carried out by the user on his personal computer (mobile computer, mobile communication device, in user’s local computing environment, built on several computers, in a camera or video camera, in a separate hardware computing device ) without transferring to the user all the components of the encoder necessary for its full operation; a significant reduction in the amount of data transmitted by the end user to the remote information processing server, compared with the initial amount of available information to be encoded.

In addition, it provides the opportunity for the developer of the used photo or video encoder to protect their encoder from unauthorized copying and use, as well as to control the encoding process on remote devices of the end user.

Claims (1)

A method for encoding photo and video information, including converting a photo image or a sequence of video frames into digital form, digitally processing in the encoder the values of the brightness and color pixels with decreasing the final amount of information (compression), characterized in that at the intermediate stage of encoding, all current information present in the encoder and related to one or more frames, is divided into two or more groups having different sizes, most of the data is processed inside the encoder itself, and about experimental data group (one or more) are sent to one or more additional remote coding device (a remote server or servers) which is processed by a certain predetermined encoding algorithm and returns back to the encoder, the encoder then encoding all produced final data.