KR20020072478A - Streaming method by moving picture compression method using SPEG - Google Patents

Abstract

PURPOSE: A streaming method using a motion picture compression method employing SPEG(Stream Picture Expert Group) is provided to remove the buffering process and resolve the overload of a streaming server while minimizing the load of lines by minimizing the band width of image and voice signals. CONSTITUTION: A streaming method using a motion picture compression method employing SPEG includes the steps of setting threshold values of frames to compress and compressing the frames individually, assigning index values per image and aligning the images in sequence, removing overlaying parts after the individual images are compressed while remaining a single index value, residing internal operators of a streaming server in user groups of clients when the clients are connected via the above steps, for carrying out the reciprocal synchronization without any additional players, and directly sending the internal operators to applet or class on a common browser on the web by using java.

Description

Streaming method by moving picture compression method using SPEG}

The present invention relates to a method of streaming using a video compression method using SPEG, and more particularly, to a method for transmitting and receiving multimedia data including audio / video over the Internet by compressing without a separate device or a dedicated player.

Streaming is a technology that can receive voice and video in real time from the internet. It is an internet solution that implements multimedia contents such as audio and video on the internet web. Streaming transfers multimedia data to the PC via the Internet and minimizes the time required to send massive video data to the Internet.

The biggest drawback is that streaming services are affected by network bandwidth, and they are limited because they require more powerful PCs. However, with the recent expansion of the dedicated Internet line and the increase in PCs, these problems are gradually disappearing.

The market for the sale of Internet streaming is the rise of Internet media companies centered on existing broadcasters and newspapers, Internet broadcasting, in-house broadcasting through local area networks (LAN), and distance learning. The use of streaming technology enables not only real-time broadcasting through the Internet but also video on demand (VOD) service, which is becoming a core technology in the Internet broadcasting field. In addition, these technologies can be said to be divided into compression source technology and multimedia streaming technology, which are the core of streaming. It is also true that the current source of the majority of multimedia streaming technologies follows the mpeg scheme.

However, it is also true that there are many obstacles in the problems of chronic buffering and down and the price of streaming server, and there is also no solution to server overload.

First, the multimedia streaming technology that is the basis of the streaming technology will be described. Multimedia streaming refers to a technology for transmitting and receiving multimedia data including audio / video over the Internet in real time rather than for downloading. Streaming is a technology that does not receive and process all information at once, but continuously receives and processes enough information for processing.

In this part, it has a process called buffering, and multimedia streaming technology goes through the interworking process of audio signal and video signal in this process. In addition, since all audio signals and video signals cannot be connected at once, the time of waiting for the user while giving a regular time. The buffering process is repeated.

I will explain multimedia streaming in more detail. Multimedia streaming is generally used for on-demand <VOD> and live broadcast services of multimedia data. On personal computers, users can play multimedia streams directly without having to download the entire file. By simply clicking a button on the web page, the event starts in seconds (shortening the buffering time). Multimedia streaming events can be provided in real time or stored for viewing at any time as needed.

This part is dedicated to the streaming server, and it is the function of the web server that enables the connection to the server. That's why you need a large server. On-demand services have the advantage of seeing what you need right away and the continuity of the operation, but when you are connected to a server remotely from the outside, it takes a few seconds to several minutes depending on the line, and in severe cases The user will be asked to proceed with reconnection.

The multimedia streaming service is used for the following services.

(1) On-Demand Service

It is an on-demand service that provides multimedia data when a client wants it. The multimedia streaming system is used to transmit multimedia data stored in large quantities on a disk in real time.

(2) Live Broadcasting

The live broadcast service can be provided by providing the video of the video camcorder or the voice of the microphone to the client in real time. This live broadcasting service can be used to establish an Internet virtual broadcasting station.

Application field of multimedia streaming technology

There are various fields in which multimedia streaming technology can be utilized, including Internet broadcasters.

In particular, Internet broadcasters that are attracting attention these days can operate a virtual broadcast station through the Internet using a live broadcasting service. In addition, the VOD solution can be applied to virtual training of distance lectures. Currently, web banner ads in the form of image banners can be converted into multimedia ads and can be actively used for electronic commerce using video and voice.

In addition, it can be utilized in conjunction with the company's intranet such as in-house education and guidance system and broadcasting station. Currently, more advanced application solutions are being announced and applied directly to current web solutions such as advertisements, video mails, and bulletin boards. And IMT2000 technology, which is emerging as a fire these days, is also a major one.

It is also true that the buffering problems experienced by many users in video streaming, the high cost of streaming servers, and the heterogeneity of circuits remain the biggest problems without controlling the compression source. So why aren't they thinking about using other compression techniques and sticking to mpeg?

Of course, we'll compare the two compression methods in more detail below, but here's a quick look: That's because mpeg is the standard. In addition, it is the efficiency of proper inter-portability and compression of audio and video signals.

This step is called rendering, which is the process of converting digital information from the stream into sound and pictures. In the rendering of the content information, the client computer receives the data and the streaming server recognizes the player linked with the server (real audio, media player, etc.) and plays the content by itself, or the plug-in supports various consoles and OS. You can use it to insert streaming data into web pages, or to use SMIL, such as advanced presentations, to synchronize text, images, audio, and video with other items on a web page. These features are now available in mpeg, and we have continued to work in this area, enabling even the most advanced, integrated presentations available today. However, the concept of enduring buffering remains intact, and there is a disadvantage in that it is indispensable to use continuously, whether large or small.

The compression method will be described below. First, in order to understand this process, it is necessary to understand the concept of frames, which is the key to compression. This is the most important part of measuring streaming technology.

How many frames per second can you achieve? This is what determines the speed and quality of streaming technology.

end. Understanding Frames and Compression

1) Frame

A frame is a scene of a movie or TV that constitutes an image, and is not a moving screen. As you know, the screen of a movie or TV is moving, but in reality it seems to be moving to the illusion of the human eye as a series of still images. In order to look like this, it should be about 25 to 30 scenes (frames) per second. In other words, only 30 images per second must be shown in succession to appear as a natural operation. Displays frames per second in FPS (Frame Per Second).

2) the need for compression

In general, a scene that implements 256 colors at a resolution of 640 × 480 is calculated as 640 × 480 × 8 ÷ 8 and has a data amount of about 300KB.

In order to implement a video using such data amount, only one minute of data has a data amount of 300KB × 30frame × 60 seconds = 540MB. In other words, even a 1GB HDD can only store about two minutes. Therefore, the method of increasing the use time by compressing the parts having the same value is used. As an example, V-CD can store about 74 minutes of video using this compression despite its 680MB capacity. This is where compression source technology comes into play. Existing streaming technology has adopted the most international standard, mpeg, and it is also true that surprising research results have been published one after another. However, the capacity limitation may have been solved, but the time difference of buffering, which is a clear decryption part related to streaming, could not be reduced.

Compared to the high compression ratio, the buffering process, which is the decryption part, was inevitable.

I. Compression and Principle and Class

1) The principle of compression

Data compression should reduce the size of the data, but the content of the data must be original. In other words, the original data is compressed to produce new data. And restoring it back to its original state is decompression. For example, if the data is 33333333, it can be expressed as 38 simply. In this case, the effect of reducing 8 bytes to 2 bytes can be achieved. PKZIP, LHA, and ARJ each use more complex principles to compress and restore data back to their original data.

2) Types of Compression Techniques

Lossy Compression

The colors and resolutions of pictures and images that can be identified by the human eye are limited. Therefore, beyond a certain resolution, you do not notice a difference and look almost similar. That's why we can reduce the amount of data by eliminating any data that humans can detect. Audio or video can be compressed. This is how most streaming technologies borrow.

Lossless Compression

Lossless compression is a method of compressing data without damaging it. Data such as numbers, documents and programs should never be damaged. If damaged, the data is not available. Popular compression utilities use lossless compression. It is also a still image compression technique.

-DCT compression technology

1974 was a year of monumental invention that made the multimedia revolution possible. Three researchers, including Professor Lao of the University of Texas at U.S.A., published a paper in the IEEE journal of a new orthogonal transformation called the Discrete Cosine Transform (DC T). This DCT is particularly good at compressing video, making it the core of today's multimedia standards H.261, JPEG and MPEG. Lossless compression of texts, figures, and general data is possible to recover completely, but the compression ratio is about 2 to 1 on average. On the other hand, compressing data such as video, audio, and sound while allowing small loss that human eyes and ears hardly feel can easily achieve compression ratio of 10 to 1 or more. In the case of a video, there is a redundancy between screens and redundancy between pixels in a screen, and if you use the visual characteristics well, compression of more than 30 to 1 can be easily obtained as can be seen in MPEG video compression. Still images have only the redundancy of the pixels in the screen, and there is no redundancy between the screens, so the compression rate is somewhat lower than that of MPEG. Since the image is three-dimensional (video) or two-dimensional (still image) data with high redundancy, the compression is also large. In VSELP, a voice compression method for North American mobile communication, a compression ratio of about 8 to 1 is obtained, and in Dolby AC-3 or MPEG sound compression, 6 to 1 for a single channel and stereo or multichannel with high channel redundancy (e.g., In case of theater movie 5.1 channel), about 10 to 1 compression is obtained. The most widely used lossy encoding technique for the purpose of effectively compressing image data is transform encoding. The basic structure of this method is to quantize differently components by dividing the arranged data having spatially high correlation into various frequency components from low frequency components to high frequency components by orthogonal transformation. At this time, there is almost no correlation between the frequency components and the energy of the signal is concentrated on the low frequency side. The gain of the conversion encoding obtained at the same bit rate compared to the simple PCM is equal to the ratio of the arithmetic mean and geometric mean of the variance of each frequency component. In other words, the deeper the energy concentration toward the lower frequency, the higher the compression efficiency. A simple PCM for spatial data represents all samples in bits of the same length (eg mbits / sample), with a signal-to-quantization noise ratio of about 6m. On the other hand, the data transformed into the frequency domain by the orthogonal transformation allows the frequency component with a lot of energy (that is, a large variance value) to be allocated more bits to more faithfully express the frequency component. Whenever the variance is four times (ie twice the amplitude), one more bit is allocated, which has the same quantization error characteristic in all frequency components. Among the various orthogonal transformations, the most energy efficient characteristic of the video signal is theoretically the most effective in compression. However, this cannot be used realistically because the conversion function must be newly defined according to the image. Lao's team's goal was to find a feasible conversion with performance close to this KLT, and the result is the DCT mentioned earlier. DCT, which is now a core technology in many international standards, combines 8x8 pixels into a single block as a unit of transformation. The larger the block size, the higher the compression efficiency, but the harder it is to implement the transform. Experimentally, 8x8 was chosen as a compromise between performance and ease of implementation. Quantization of the DCT transform coefficients is possible by scalar quantization (SQ) and vector quantization (VQ). VQ is usually effective when the correlation between coefficients is high and is more complicated than SQ instead. Since DCT coefficients are almost uncorrelated, SQ is currently adopted by international standards. In addition, SQ is divided into linear and easy-to-implement nonlinear techniques, which are easy to implement. When the quantized coefficients are subjected to entropy coding (lossless) again, the performance difference between the two techniques becomes smaller. Currently entropy encoding is followed in international standards, so only H.261, JPEG, and MPEG-1 used linear techniques. However, MPEG-2 also employs nonlinear techniques to improve performance slightly. In addition, the current standard uses a combination of run length coding and Huffman coding for lossless compression using statistical characteristics of quantized DCT coefficients. Image compression is combined with many techniques such as DCT, quantization, run length coding, Huffman coding, and motion compensation DPCM (for video only).

Motion compensation compression

Human vision feels like a continuous image of nature when 16 or more images are displayed per second. In other words, in a moving picture, 16 pieces per second is the minimum sampling frequency (Nyquist frequency) for sampling a signal while preserving information. With that in mind, movies are shooting at 24 speeds per second, while TVs are shooting nature at 25 or 30 speeds per second. Movie is a format that saves one screen at once and lightens it on the screen at the time of screen unit, whereas TV basically has to transmit the image through radio waves. The CRT also displays images by scanning (the facsimile is similar in terms of scanning and transmitting). NTSC color TV systems adopted by the United States, Japan, and Korea use 30 sheets per second (exactly 29.97) per scan line at 525 lines per screen, and 25 sheets per second with 625 lines at PAL or SECAM methods adopted in Europe. Is sending. In addition, in TV, so-called interlaced scanning in which one screen (frame) is alternately transmitted into an even field consisting of even-numbered scan lines and an odd field consisting of odd-numbered scan lines in order to display a video more effectively using limited scan lines. I'm using the method. Therefore, NTSC per second is 60 fields, PAL or SECAM is 50 fields, so that it can be followed well even when there is a lot of movement such as sports scene. When a movie is broadcasted on TV, it is scanned and sent one by one through a converter called telecine (a combination of television and cinema). At this time, if the number of screens per second of film and TV is different, and the film is simply played at the TV screen speed, 25 screens per second of PAL or SECAM is not a big difference from 24 screens per second. Because of the screen, the movement is quick, the voice is high, and the movie is fast. Therefore, when transferring a motion picture film to an NTSC TV, it is necessary to adjust the screen speed. Since 60 fields are obtained from 24 screens per second, 5 fields are obtained from 2 screens. A simple and practical method is to scan three fields from the first screen of two screens and two fields from another screen. This is called "3: 2 pulldown". From a data compression point of view, if you take dozens of screens per second, like movies or TVs, you have very high redundancy between screens. For example, in the case of a fixed scene, the contents of each screen are the same, so if only the first screen is transmitted, the following screens can be completely transmitted with only simple information that "same as the previous screen". Also, even in a moving scene, the background part is often stationary, and the moving part can also greatly reduce the amount of data by sending information of "what part moved where". If the data is not compressed, there is a scene change and there is no correlation between the two screens. At this time, the rear screen is compressed only in the rear screen without using the information of the front screen. In the mid 70s, video calls using telephone lines were introduced under the name "picture phones." Although it was a breakthrough technology at the time, it was expensive because the market was not wide and semiconductor technology could not fully support it, but it failed. At this time, an attempt was made to reduce the redundancy between screens by dividing the moving part and the stationary part between neighboring screens by updating the area information of the moved area and its contents and not sending the stopped part. This method had to distinguish the moving parts, which made it difficult to implement in a real-time system such as a videophone. The method to overcome this in the early 80's is a block-by-block motion estimation and compensation method that is widely used today, such as MPEG or H.261. In other words, the screen is divided into units of a certain size (usually called 16x16 macroblock and 4 blocks of 8x8, which are DCT units). To compensate for the movement. The amount of data can be greatly reduced by encoding only the difference between the current macroblock and the macroblock of the previous picture obtained by motion compensation. A motion vector must also be transmitted so that the receiver can use it for video playback. Lossless compression using DPCM and Huffman codes is used. With this motion compensation compression technique, the efficiency of moving picture compression technology such as MPEG is significantly higher than that of still picture compression technology such as JPEG.

-H. 261

H.261, an international standard created by ITU-T (formerly CCITT), is a video compression method for video telephony and video conferencing using an Integrated Services Digital Network (ISDN). This, together with JPEG, the international standard for still picture compression, is the birthplace of MPEG-1 and 2, the international standards that are the center of the multimedia revolution today. The telephone, invented by Graham Bell in 1876, has since become the main means of human communication. ISDN emerged to integrate individual intertwined communication networks, including the telephone network, which is the backbone network that accommodates telephones and facsimile, the telex network that is used primarily for enterprise communications, and the digital and network switched circuits for data communications. to be. ISDN began to be practical in several countries in the 80s, when international standards were established as ITU-T's I-family. Channels defined by ISDN include B channel 64Kbps for basic information transmission such as voice and fax, H channel for high speed data transmission (384Kbps for H, 1,536Kbps for H ^{3 3} ), and various control signals. There are three types of D-channels (16kbps or 64Kbps). In practical use, these three channels can be combined as appropriate to form a basic or primary group connection. The basic connection to the home uses a two-wire spiral copper wire like the current telephone line, and the data transmission rate of total 1,44Kbps by time-division multiplexing two B-channels and one D-channel (16Kbps) (2B + D) Has The home wiring is a four-wire bus that can connect up to eight terminals, enabling simultaneous telephone-fax-low-speed computer communication and two household telephones. As part of the service using ISDN, G4 fax is a high speed and high resolution improvement of G3 fax using the current telephone line, and another is a video phone that transmits face and voice together. In this case, the voice is encoded by G.711 (64Kbps PCM) or G.728 (16Kbps LD-CELP), the face image is encoded by H.261, and then multiplexed by H.221. It becomes H.320 which is a telephone terminal. At this time, the bit rate provided by the ISDN basic connection is only 144Kbps, so there are not enough bits left for the video, i.e., the bits used for voice and control. Therefore, there are many restrictions on the number of screens and image quality that can be sent per second. This is based on the fact that video telephony is mainly used for information transmission through voice and video is an auxiliary function. The ISDN connection for office work is often a primary group connection with a much higher transmission rate than the home connection. The ISDN primary access is 1,536 Kbps in North America and Japan, and 2,48 Kbps in Europe, allowing a combination of three different channels, BHD. One way to take advantage of this high rate is video conferencing. Video conferencing is a powerful way to confer meetings while saving time and money between distant parties. Since many people meet each other on the big screen, they should have better image quality and sound quality than video calls where two people talk on small LCD screens. Therefore, the voice codec uses G.722 (64Kbps or less, SB-ADPCM method) which provides sound quality close to AM broadcasting, and the video codec uses H.261 as much as a videophone. H.261 is a video compression standard for video telephony and video conferencing of p × 64 Kbps, which takes into account ISDN basic connection and primary group access, so p is 1 ~ 30, and p is usually 1 ~ 2. , Video conferencing is above 6 (the H channel has a p value of 6). The person responsible for H.261 standardization at ITU-T was Okubo of Japan NTT (now GCT). The technique he used in the standardization process was to create a model called "Reference Model" and improve it at every meeting. . In other words, the participants were able to compare this model with the newly proposed technology, verify the objective excellence, add the technology to the model, and repeat this process to create a good algorithm by converging many element technologies in a short time. Since this technique was later adopted in MPEG, H.261 played a role in the birth of MPEG as an effective process of standardization as well as the technical content of image compression. H.261 is a standard for video compression / extension with built-in video telephony and conferencing terminal speeds of p × 64 Kbps, p = 1–30 connected to ISDN basic or primary group access. H.261 began its standardization work in 1984, was revised to "Reference Model (RM) 8", and the technical content was completed in 1988. It was finally approved by ITU-T in 1990 and finally confirmed as a recommendation. H.261 combines several lossy / lossless compression techniques for effective video compression. First, let's look at the input and output format of the video. 625/25/2: 1 (screen configuration is 576 × 720, mainly PAL and SECAM) with North America, Japan, Korea etc. which use 525/30 / 2: 1 (screen configuration is 480 × 720, mainly NTSC) in TV system There is a difference between the European countries, where the horizontal and vertical sync frequencies of the camera and monitor are different. Cameras and monitors for video phones are also compatible with existing TVs due to economic feasibility, so direct video communication between different countries is not possible. Therefore, in H.261, a common format called Common Intermediate Format (CIF) is used for conversion between two systems and used as the video input / output format of the codec. The CIF format takes a 1/4 size for broadcast and consists of 288 × 360 screens with 30 forward scans per second. Format). Its quarter size is QCIF (Quarter CIF). The camera output is converted to CIF after preprocessing and applied to the H.261 encoder, and the CIF screen reproduced by the H.261 decoder is converted to a normal TV signal after postprocessing and displayed on the monitor. For intra-screen compression, DCT in blocks (8x8 pixels) is performed to concentrate the energy of the block in low frequency components and then quantize it. Quantization uses a uniform quantizer to simplify the system. At this time, the human visual characteristics feel relatively less of the quantization noise of the high frequency component, so that the higher the DCT coefficient, the larger the quantization step. Therefore, the high frequency component is small in size, but the quantization step is large, and most of them are zero. In order to increase the compression rate by using the correlation between screens, the screen is divided into macroblocks (four luminance blocks and two color blocks) to obtain a motion vector indicating which position of the previous screen has been moved for each macroblock. As a motion estimation method, a block matching algorithm is widely used to find an error position of the minimum macroblock by finding an error with the current macroblock for each pixel position of the previous screen. In H.261, motion vectors are searched in pixel units, whereas in MPEG, which requires higher image quality, it is searched in half pixels. The motion vector must be sent to the receiver for picture reconstruction. DPCM is performed between the vectors to save bits and Huffman encodes the result. The quantized DCT coefficient has many zeros, but more so for inter-screen coded blocks than in intra-picture coding. Run-length coding is used to compress this efficiently. That is, it shows how many zeros are repeated and nonzero values are generated in the form of (run, level) while performing a zigzag scan starting from DC. This allows many consecutive zeros to be accommodated together in a "run" to achieve data compression. Since these (run, level) symbols have different occurrence frequencies, they are further compressed by using a Huffman code that encodes a symbol with a high occurrence frequency, that is, a small run or level value, into a short code. The amount of data generated through this process varies with time, and is large in the complex part or the part encoded in the picture. If the data transfer rate through the channel is constant, a buffer is required to write the generated data and read it at a constant speed. The receiving side needs a buffer of the same size as the transmitting side to write at a constant speed and read and decode at a variable speed. When the buffer overflows or fills completely over time, the continuity of the bit strings is broken and the decoding is suspended. To avoid this, it is necessary to control the bit generation amount by adjusting the quantization step by feedbacking the state of the buffer. H.261 opened the way for real-time video communications by combining high loss / lossless data compression techniques up to that time, which later led to MPEG-1 and 2, which prompted the multimedia revolution.

-H. 263

Since Bell invented the telephone in 1876, mankind has dreamed of making a video call that can talk to each other's faces. In the mid-'70s, a video phone called "Picture Phone" was introduced. However, it failed to enter the market because the technology level at that time could not meet the price and performance. In the 80's, the ISDN became standardized by ITU-T (formerly CCITT), and the application of the ISDN network was considered to be video telephony. This videophone (H.320 terminal) is targeted at the primary interface (two 64 Kbps channels and one 16 Kbps channel) for connecting to the ISDN network in the home, or the primary group interface (up to 30 64 Kbps channels) used in the office. It has a bit rate of (P = 1 to 30). The international standard for video compression at this time is H.261. H.320 videophones, like the G4 facsimile, are intended to be connected to the ISDN network, so they cannot be used on a regular telephone network, making it difficult to commercialize them at this time when ISDN is not widely available. In the early 90s, telecommunications companies such as AT & T.MCI developed and distributed video telephones using telephone networks. However, the compatibility of communication devices is very important in nature, and there was no compatibility between these video phones. MPEG4 was originally started to meet this demand, but gradually expanded its scope to eventually focus on multimedia database access or wireless multimedia communications, and standardization is expected to be completed in 1998. Based on this background, ITU-T SG15 started to make international standard of video phone operating within 64Kbps using telephone network and finally finalized its technical contents at the end of last year. The goal was to complete it in the short term, so instead of accommodating new algorithms like MPEG4, it was an improvement to H.320 videophones, the H.324 terminal. H.324 videophones are connected to the telephone network through a recently standardized V.34 modem (up to 28.8Kbps at the fastest speed for a telephone line modem), and video compression is a significant improvement over H.261. The voice compression is performed using CE723 G723. The improvement of H.263 over H.261 is as follows. First, in order to encode a motion vector of each macroblock, a more efficient method of taking the median value of three vectors is used in consideration of high correlation with a motion vector of a neighboring macroblock. In this way, a data reduction of about 10% is obtained. In addition, motion estimation can be performed for each block to subdivide motion within a macroblock. For efficient quantization of the DCT transform coefficients, the quantizer is improved to obtain about 3% data savings, and the variable length coding of the quantized transform coefficients improves the two-dimensional code of H.261, JPEG, MPEG1, and MPEG2 to improve intra-screen and inter-screen Use 3D code considering information. You can save about 5% of your data here. The syntax of bit strings is greatly simplified compared to H.261 to reduce the overhead of pure information bits. In addition, it is a technique that brings performance improvement but is complex and leaves it available as an option, such as syntax-based arithmetic encoding and PB frame. Syntax-based adaptive arithmetic coding is complex but saves 5-14%. PB frames can double the number of frames per second while allowing slightly more bits than other techniques for bit saving, resulting in a much more stable and smooth video visually. Several other factors have been proposed to improve the performance, but those that do not have a noticeable improvement compared to the complexity are excluded. H.263, which has many factors above, is used as a reference algorithm for evaluating the performance of the proposed schemes in the recent MPEG4 image quality evaluation. Surprisingly, H.263 performs better than most of the proposed schemes. It turned out to be better. This makes it possible to use a telephone line to make a decent videophone. Since the specification has already been completed, it is expected that from the second half of this year, a video telephone that can be directly connected to a telephone line will be introduced. The videophone can also be implemented on a PC, which is expected to take the concept of a traditional multimedia PC with PC communication and fax transmission and reception over the telephone line to the next level. In other words, it will not be far from home to make video calls via telephone lines.

-JPEG

To store the image on the disk or to transmit through the communication channel excessive data amount is a big burden. For example, for a natural color image of 440 x 640 pixels commonly used on computer screens, each having 8 bits of R (red), G (green), and B (blue) per pixel, one screen occupies approximately 0.9 MB. do. Considering that the current hard disk capacity of PCs is 560MB and 3.5-inch floppy disks are 1.44MB, the amount of video data is considerable. The Joint Photographic Experts Group (JPEG) is an international standard (ISO-IEC 10918) for efficient compression for the storage and transmission of still images, such as those used in computer electronic cameras, color fax color printers. In 1988, the technical content was completed after a dramatic competition between DCT in Europe, arithmetic coding in USA, and vector quantization in Japan. During the deliberation process, binary images (targeted by facsimile, etc.), in particular, needed a separate effective compression method, and created a separate working group to complete the Joint Bilevel Image Coding Experts Group (JBIG) standard. JPEG compression can be divided into lossless mode and lossy mode. Lossless mode is used for applications that should not damage the original at all, such as medical imaging, and lossy mode is used for many applications that increase the compression rate while allowing visually subtle loss. In the lossless mode, the pixels to be encoded are spatially predicted from adjacent pixels, and Huffman coding is performed based on statistical frequencies. Lossy mode combines lossy coding (DCT + quantization) and lossless coding (DPCM.runlength coding, Huffman coding, and arithmetic coding) to increase compression. The screen is divided into blocks (8x8 pixels) and DCT is performed for each block to concentrate energy on low frequency coefficients before quantizing. In this case, the quantization step is increased as the higher frequency coefficient is obtained by using the fact that the human eye feels less quantization noise of the high frequency component. Therefore, the high frequency DCT coefficients are small, and the quantization step is large, so that they are mostly zero. Lossy mode is divided into baseline and extended mode. The basic method uses run length coding and Huffman coding to further compress the quantized DCT coefficients. In other words, it shows how many zeros are repeated and nonzero values in zigzag scan starting from DC. Since these symbols have different probability of occurrence, they are further compressed using 2D Huffman code. At this time, since the DC corresponding to the average value of each block greatly affects the image quality, it is necessary to express more faithfully. Therefore, DC quantizes more finely than AC coefficients and encodes one-dimensional Huffman by taking the difference from the previous block for the quantized result. The extension uses arithmetic coding to compress the quantized DCT coefficients more efficiently. It is slightly more efficient than the Huffman code, but it is more complex and several companies, especially IBM, have patented this part, so the basic method is free of royalty and easy to implement. The expansion method also includes a function of gradually encoding a picture so that the receiving side selectively reproduces a picture that is gradually cleared. Various video data storage formats and compression methods are currently used in computers. Popular GIF and TIFF formats are based on lossless compression using the Rempel-Jib algorithm. JPEG, which has a much better compression rate and application range, has recently begun to be actively introduced into PCs and outputs files with a .jpg extension. There is also M-JPEG (Motion-JPEG), which compresses each screen into JPEG for moving pictures. M-JPEG has no motion compensation part, so the compression rate is slightly lower than MPEG. Instead, it is much easier to implement. Unlike MPEG compressed files, screen editing is possible. At the same time, JPEG, along with the H.261 standard for video telephony and conferencing, completed on the basis of DCT, will form the basis of MPEG-1 and 2, which will later become multimedia core technologies. In other words, MPEG obtains the code and method of the picture based on DCT from JPEG, and the inter-picture coding method using motion compensation DCT from H.261, respectively, and combines and develops them.

All. MPEG

1) What is Moving Pictures Expert Group (MPEG)?

This is a video expert group, which is a work group of the multimedia standard development committee under ISO.

The group was formed for the purpose of establishing a standard for digital video. MPEG also refers to the video and sound data compression technology established by the group. The most problematic part of multimedia is video. Unlike still images, video has a large amount of data, so a compression technique that can reduce the size is required. MPEG is a method of compressing a video including a human voice or various sounds. In other words, video and audio are compressed separately and synchronized. It is the most common streaming compression method and the foundation technology of media players and real networks.

In particular, this organization is interested in transmission technology. The MPEG1 standard has a 1.5Mbps transmission speed (the same as a compact disk; CD), and is preparing 4 and 9Mbps MPEG2, 20Mbps MPEG ++, and 40Mbps MPEG3. .

2) MPEG video compression method

MPEG compresses video using lossy compression. The main principle is as follows.

(1) Compress one image.

(2) The following image is compressed after removing redundancy.

This is where you determine the number of frames, which is usually handled by the streaming server.

(3) After the image of each object is compressed, the overlapping part is compressed again.

This means that if a person speaks for a few seconds in a scene in a movie, his or her dress or background will not change. The only thing that changes is the shape and expression of the mouth. Therefore, if you store the first video, you can reproduce almost the same video as the original when playing the second frame. In this case, the amount of data can be reduced to several tenths of the original.

However, during this process, there is a case of severe downsizing and intermediate down. And the resolution and color of the image are lower than the original (although the general public does not feel well).

In addition, the streaming server has to overload the data to be interpreted, and the server specification also uses expensive servers. You also need a dedicated player codec, which is a dedicated streaming player such as a real audio or media player.

3) MPEG standard

-MPEG-I: The first MPEG standard established in 1990 and used since 1992. It is used in CD, DAT (Digital Audio Tape), broadcasting VCR, etc., and provides VHS video and provides 1.5Mbps speed. . It has already been adopted and commercialized in video CD or CD-I / FMV (CD-Interactive / Full Motion Video). Video CD is a standard agreed by four companies such as Philips in the Netherlands and Sony Matsushita JVC in Japan in 1993. CD-I / FMV is the first multimedia device that Philips has already released. The complete dynamic recall was added. Video CD, which had only slightly monotonous playback function compared to CD-I / FMV, was released in version 2.0, which enhanced the interactive function in 1994. In addition, based on the Level 2 standard created in 1993, the multimedia PC, which recorded the explosive growth by providing animation-quality video and sound, can play movies on a CD on a PC by playing natural fairy tales with the latest MPEG1 card. It became. Furthermore, attempts are being made to implement MPEG1 decoders in software on Pentium-class or higher-performance PCs. Historically, the last 88 years have been an important time deciding the direction of future multimedia. First of all, CD, which appeared in 82 for high-fidelity audio, has been applied as a large-capacity (640MB, 74 minutes) data storage medium with 1.5Mbps playback speed under the name of CD-ROM from 1985. In addition, H.261, a moving picture compression standard for video telephony and conferencing, and JPEG, a still image compression standard, were nearing completion. In this context, the next goal of the experts who have been leading the standardization has been to establish a standard compression method to naturally capture video and sound on CD-ROM. MPEG is an alias for ISO-IEC JTC1 / SC29 / WG11, a committee formed for this purpose. There are several small groups in MPEG for efficient work. That is, the video small group responsible for the compression method of the video, the audio small group responsible for the compression method of the stereo sound, the system small group responsible for the packetization, multiplexing, and synchronization of the compressed video, audio bit stream, and the various requirements for standardization. Proposed requirements Small groups, video and audio small groups include implementation small groups that evaluate the difficulty of implementing algorithms, and test small groups responsible for performance testing. Many of these small groups meet in groups or jointly to complete the standard. MPEG standardization does not choose any of the proposed compression algorithms. Given that each proposal can have its own merits, the algorithm is evolving by absorbing good elements one by one. This method was chosen based on the fact that H.261 set up a reference model and revised it to the 8th stage and successfully led standardization. In MPEG1, this standard is called a simulation model. In the case of MPEG1 video, about 30 proposals were fused, resulting in a hybrid method (motion compensation DPCM + DCT + quantization + run length coding + Huffman coding) that synthesizes and improves JPEG's static compression method and H.261's moving picture compression method. The simulation model was revised up to the third stage and the committee's standard was completed. In the MPEG1 audio system, four consortiums compete with each other, and subband coding based on Philips MUSICAM forms the first and second layers, and AT & T's more complicated and efficient method forms the third layer. Considering the balance between sound quality and complexity, the second layer is the most widely used. The MUSICAM method is adopted not only in MPEG1 but also in the recently commercialized portable digital audio device, DCC (Digital Compact Cassette) .The MPEG1 system is a regulation on the data format for packetization, multiplexing, and synchronization of compressed video and audio. It can be used for various multiplexing, and has various advantages in transmission and storage processing, etc. MPEG1 is used not only for video CD, CD-I / FMV, next generation multimedia PC, but also for video dialtone (VDT), which is video on demand using a telephone line. In this case, KT is currently conducting a test service for 250 subscribers at the Seoul Banpo Telephone Bureau. After compressing video and audio with MPEG1 at 1.5Mbps, the data is transferred to an existing telephone line by ADSL (Asymmetric Digital Subscriber Line) technology. (Double helix copper wire), because the image quality is not satisfactory. Consideration is being taken to MPEG 2. MPEG-1 also serves as a stepping stone to the multimedia era by acting as a bridge to MPEG2, an efficient way to follow even more features. It is a combination of the encoding technique and the inter-screen encoding technique of H.261. The evolution from H.261 to MPEG1 is as follows.

H.261 is a video compression standard for video telephony and video conferencing. Therefore, the screen covered by H.261 is basically a so-called Head-and-Shoulder image that includes the human face and shoulders. In videophones, the camera is usually fixed and the speaker talks in front of the other person's face. The image at this time is characterized by a fixed background and some facial movements (especially large eye and mouth movements).

Therefore, the correlation between neighboring pictures without scene change is very high and the picture coding is very efficient. In H.261, the first picture encodes all macroblocks in picture (I picture), and subsequent pictures are forward predictive coded from the previous picture (P picture) to increase compression efficiency. In other words, the configuration of the screen is in the form of IPPP-. The P picture selects one of more compression among intra picture encoding (intra mode) and inter picture encoding inter mode after motion estimation and compensation are performed for each macroblock. On the other hand, in the motion compensation prediction encoding, once an error occurs, it is continuously propagated thereafter and the screen is not restored. To overcome this, H.261 specifies that each macroblock is forcibly coded in intra mode at least once for 132 pictures. The screen targeted by MPEG1 is a general picture seen in a movie or TV that is not limited to a human face as in H.261. Therefore, there is more movement and scene transitions occur frequently. Unlike H.261-based video telephony, whose main purpose is to communicate via voice, the application of MPEG is mainly focused on entertainment where image quality is important. In consideration of this, it is necessary to limit the propagation of errors and to speed up the screen recovery. Also, the random access function should be able to reproduce from any screen of the video sequence. In order to satisfy the above conditions, each picture must be encoded in the picture. An example is Motion JPEG (M-JPEG), which is partly used in broadcasting studios, where each screen is encoded in JPEG, allowing editing of screen units and no error propagating to the next screen. However, M-JPEG does not use the correlation between screens, which reduces compression efficiency. MPEG1 takes the intermediate form of H.261 and M-JPEG, so that the so-called I-picture encoded in the picture is placed at regular intervals, and the pictures between them are predictively encoded. The I screen is the basic unit for screen recovery and random access when an error occurs or when the power is turned on. The encoding of the I picture is naturally very similar to JPEG. More specifically, it is similar to the JPEG baseline system and consists of a combination of DCT + quantization + run length coding Huffman coding. The MPEG-1 predictive encoding screen is divided into two types. One is the P picture as seen in H.261, which performs motion compensation prediction encoding from the previous I or P picture. The other is the so-called B picture, which is a new concept introduced in MPEG1, and motion compensation prediction is performed in both directions from the nearest I or P picture before and after to encode an error. Therefore, the configuration of the screen is in the form of IBBPBBPBB IBB- when the B screen enters 2 screens and the I screen enters 9 screens once. P picture and B picture consist of a combination of motion compensation DPCM + DCT + quantization + run length coding + Huffman encoding. However, in motion compensation, the P screen is forward and the B screen is bidirectional. Here, the B screen has twice the amount of motion vectors (forward and backward), but the error is greatly reduced compared to the P screen, thereby increasing the compression efficiency. On the other hand, the I and P pictures before and after the B picture are coded in sequence, and then the B picture is coded, thus changing the transmission order and display order of the picture. Therefore, the delay of several screens is inevitable in encoding / decoding, and the amount of memory for screen storage is greatly increased. For example, in the digital broadcasting of current TV, if there is no B screen, 8 Mb of memory is required, but if there is a B screen, 16 Mb is required. As B screens contribute greatly to the improvement of picture quality, a tradeoff between picture quality and system complexity is necessary. Recently, B screens are used in many systems due to the development of semiconductor technology, which reduces the cost of implementation. MPEG1 is based on a low transmission rate of 1.5Mbps, so the picture quality of MPEG-1 based video CDs and CD-I / FMVs is somewhat lower than that of current broadcasts. Nevertheless, multimedia PCs, electronic entertainment, and the general movie for watching at home are said to be good quality and sound quality. In that regard, it is likely to be used more widely than expensive, high-quality and multi-functional MPEG2 for quite some time.

-MPEG1 audio

MPEG1 is an international standard for compressing and multiplexing video and sound at 1.5Mbps for CD-ROM applications. MPEG1 audio, which handles dual acoustic compression, compresses mono, hi-fi stereo, or bilingual sound to about 6 to 1 in and out. Considering the existing FM stereo broadcasting, hi-fi stereo audio of CD, and voice multiplexing of color TV, it can be seen that the above three modes of MPEG1 audio accommodate audio requirements in many applications. In the case of MPEG1 audio, four consortiums competed, and Philips' MUSICAM-based method formed the first and second layers, and AT & T and others proposed the third layer. The second layer is the most widely used due to its dual performance and complexity. Since video (3D) data is three-dimensional data that changes with time, the spatial and temporal correlation between pixels is high, so that the compression ratio can be increased to one tenth of that. However, the sound is basically one-dimensional signal, so the correlation between samples is low, so the compression ratio is usually less than one tenth. However, the case where the compression ratio can be slightly increased is the case of stereo that can utilize the correlation between left and right channels. There are three sampling frequencies for the input signal of MPEG1 audio: 44.1KHz for CD, 48KHz for use in DAT and production systems, and 32KHz for digital processing of FM audio (15KHz bandwidth). The number of bits per sample ranges from 16 to 24, with 16 bits (two bytes) being used on CDs and DATs and suitable for computer environments. The amount of data before compression is, for example, 44.1 KHz x 16 bits x 2 = 1.5 Mbps for CD. In MPEG1 audio, 64 to 384 Kbps is output during stereo compression in the second layer. Although it depends on the input sound, it is known that the difference from the original sound is hardly felt until the compression of about 6 to 1 (2560 Kbp s). MPEG-1 audio takes advantage of two human hearing characteristics for effective compression. The first is characteristic of single sound, and shows curve of ear sensitivity different for each frequency component in 20Hz ~ 20KHz band. In other words, the minimum size threshold to be heard is expressed as sound pressure, which is the lowest in the vicinity of 1KHz, which is best heard in the ear, and the high frequency of 20Hz or 20KHz is very high. The second auditory characteristic relates to the interaction between complex components. In other words, if a frequency component is present at a large value, a hill-shaped masking curve is created around it so that signals below this threshold are not audible and only a larger signal is heard. Combining the above two characteristics, the spectrum of the sound can be analyzed to obtain a masking value, an inaudible range for each frequency. If the quantization noise of each frequency component is kept within this threshold and is not heard, the bit can be saved. This is the fundamental principle of MPEG1 audio compression, which, despite its loss, shows little difference from the original. In MPEG1 audio, unlike video using DCT, so-called subband coding is used. First, the input signal is passed through a filter bank composed of 32 uniform width bandpass filters and divided by components from low frequency to high frequency. In addition, a more detailed Fourier spectral analysis is used to determine the masking threshold and to determine the quantization noise tolerance in each of the 32 bands. Once the bit rate is determined, the number of available bits is obtained by subtracting the overhead, so that it is allocated to 32 bands and quantized for each band. In this case, bits are allocated one by one in order from where a lot of bits are needed, that is, where quantization noise is to be reduced. This ensures that each band is allocated as many bits as needed within the limit of the total bits. Each time the bit is increased, the quantization noise of the band is halved, thereby improving the signal-to-noise ratio by about 6 kt. For each band, we get the scale factor, the quantized sample value, and the information about the allocated number of bits, the amplitude of the signal, and send them according to the specified format. MPEG-1 audio compression is also used in the Digital Compact Cassette (DCC), which has recently been released. Unlike most MPEG1 audio applications, DCC uses the coder and decoder together to make the coder easier to use.

-MPEG1 system

MPEG1 is an international standard for compressing and multiplexing video and sound within 1.5 Mbps to digital storage media (DSM) such as CD-ROM.

The standard related to double multiplexing is the MPEG1 system. The term system generally has a very broad meaning, but in the MPEG standard, a system can synchronize the bit streams and other additional data that occur as a result of compressing video and sound, and in some form (for example, in the form of a packet). This is a technical standard for multiplexing. Multiplexing of digital signals uses Time Division Multiplexing (TDM), in which each channel signal is separated in time and transmitted alternately. The simplest case is to divide the time constant from the beginning so that each channel occupies its own time zone. Since the receiving side also knows the time allocation of each channel, it is possible to select the desired channel signal. For example, a telephone exchange's electronic switch converts voice signals from multiple subscribers into PCM digital data and multiplexes in this way. The above method has the advantage of being relatively simple because the channel assignment is fixed, but it is less flexible. In other words, the amount of data generated by a channel changes every moment, adds a new channel, or removes a certain channel, and cannot cope with various types of processing and transmission demands of a complicated digital communication network. The solution to this problem is packet multiplexing. The MPEG1 system also employs this method. A packet is a bundle of bits. The length of packets constituting the MPEG1 system bit string may be fixed or variable. In the so-called header located at the front of each packet, various attributes of the packet are recorded so that the receiver can process the packet accordingly. have. For example, it contains a synchronization signal that indicates the start of a packet, whether the packet is video or audio, how long is the packet, and how error processing is performed. Headers are a kind of overhead, but they provide flexibility in processing. Since the length of the header is constant, the shorter the packet, the greater the ratio of overhead, resulting in lower data transmission efficiency. However, the size of the buffer that temporarily stores the packet for processing can be reduced, and the processing delay is also small. If the bitstreams of video, audio and additional data are independent of each other and are irrelevant, then all three types of packets need to be transmitted one after the other. However, since the MPEG1 system bit stream is basically a program, the video-audio-additional data therein is closely related to each other. In particular, the so-called lip-sync should be made so that the screen and sound do not deviate from each other in time. For temporal synchronization between individual bit streams, MPEG-1 systems use so-called time stamps. This indicates the time that should be displayed on the monitor or speaker after decoding by recording the time at the time of entering the encoder for each video screen and audio frame together with a tag. There are two types of time stamps: Decoding Time Stamp (DTS), which indicates the time for decoding, and PTS (Presentation Time Stam p), which indicates the time for display. In order to synchronize video and audio using the time stamp on the receiving side, there is a type of clock in the receiver that must process the bit string while matching the time stamp to the time of this clock. It's as if two people agreed to meet at what time they had to synchronize their clocks together. In the MPEG1 system, this reference time is called SCR (System Clock Reference), and the encoder sends this time to the decoder so that the clock of the decoder is matched to the encoder. To do this, multiple packets are bundled and sent in packs, and the SCR is sent whenever needed in the pack's header. In addition, the pack header carries necessary information so that the receiver can decode even from the beginning of the MPEG1 system bit stream. Since MPEG1 has been designed for CD-ROM applications from the beginning, the MPEG1 system does not consider much errors and relies on the inherent error correction scheme of the application field. The MPEG1 system is later refined into program streams in the MPEG2 system. In addition to the MPEG2 system, there are also transport streams used for multiplexing multiple programs instead of one in an error environment.

Application of -MPEG1

MPEG1 is an international standard for multiplexing (MPEG1 system) by compressing video and sound (MPEG1 video and MPEG1 audio, respectively) within 1.5Mbps on Digital Storage Media (DSM). To resolve the confusion that may arise from such formal names in the MPEG1 standard, let's clarify the meaning as follows: This will also make the utility of MPEG1 clearer in various applications. Digital storage media are not only CD-ROMs, but also digital audio tapes (DATs). Many systems use bit rates much higher than 1.5 Mbps, such as digital broadcasting, which uses about 8 Mbps per channel in bit rate. In terms of the MPEG1 specification, there are no restrictions on the application, and it can be used with only a few parameters changed in various multimedia applications including digital broadcasting. Indeed, DirecTv, a US digital satellite broadcasting system, initially compressed both video and audio using MPEG1. However, there are facts to keep in mind for this application. In other words, MPEG1 compression algorithm aims at less than 1.5Mbps (strictly 150KB ultra-1.2Mbps), which is the data playback speed of CD-ROM, because MPEG1's birth was to record and play back moving images and sound on CD-ROM. Optimization was achieved. In addition, since the data format of the MPEG1 system is considered to include a single program in a CD-ROM, there is less consideration in the case of multiple programs such as error correction and digital broadcasting. DirecTv, for example, did not even adopt the multiplexing scheme of the MPEG1 system even earlier. After all, the most appropriate application of MPEG1 is to CD-ROM. In the early days, consumer electronics companies were passive in commercialization because the image quality was slightly lower than the existing VHS tape and 74 minutes of play time could not accommodate a movie. However, in the fall of '92, Japanese JVC's karaoke system based on MPEG1 became very popular, and the commercialization of MPEG1 attracted the attention of companies again. Ultimately, Philips Sony Matsushita, a leader in the consumer electronics industry, joined forces to develop the karaoke standard into today's video CD standard. It was then upgraded to version 2.0, augmenting its capabilities as an interactive device in 1994. Philips developed and released a similar but slightly different CD-I FMV, incorporating full motion video based on MPEG1 into CD-I, an existing interactive device. The adoption of MPEG1 is also spreading in the computer and game industry. In the early days of the multimedia industry, the computer industry introduced level 1 multimedia PC (MPC) standards in level 1 and 1993. Level 2, based on a double-speed CD-ROM (300KB-second data transfer rate), is taking its place with Level 3 this year. MPC Level 3 requires a 75MHz Pentium or better CPU, a 4x CD-ROM and, above all, an MPEG1 decoder. In addition, as the performance of the CPU improves, the implementation of MPEG1 decoder by software is increasing, and a system in which MPEG1 decoding function is integrated into a VGA card is appearing. Game machine makers have recently become more prominent in integrating CD-ROMs with game machines. Moreover, game consoles are being transformed into multiplayer, a comprehensive entertainment device that can play all kinds of CDs, including game CDs and video CDs. Computers and games have a different demand for picture quality than regular TVs. In other words, a computer displays images on a smaller monitor than a TV (which is smaller when a window is opened and displayed), and a game may have a lower spatiotemporal resolution than naturalization. MPEG1 can realize enough quality and sound quality for this level of application much cheaper than MPEG2, so it will be used for a long time despite the advent of MPEG2. Of course, in the digitization of TV broadcasting, the requirements for image quality are further strengthened, so it is common to adopt MPEG2, which will be covered in this series.

-MPEG-II : A standard established in 1993 by improving MPEG-I. It provides the quality of broadcasting station handling, uses DVD, and provides a speed of 5 to 10Mbps. H261 is the international standard for video compression of P × 64Kbps (P-1 ~ 30) for video telephony and video conferencing using ISDN, and MPEG1 is the international standard for compressing and multiplexing video and sound within 1.5Mbps on digital storage media such as CD. to be. As such, digital image compression technology has been limited to communication media and storage media until 1990. MPEG-2 was discussed at the September 1990 US Santa Clara Conference, which ended the standardization of MPEG-1. The subject of MPEG-1 is mainly limited to video CDs of about 1.5 Mpps, because the quality remains at the VHS level and only 74 minutes are recorded per sheet, so a standard is required to realize high quality at higher bit rates. The initial goal of MPEG-2 was to achieve current TV quality at around 5-10Mbps. It was agreed to standardize MPEG-3 to complete MPEG-2 and then to succeed in HDTV (High Definition Television) quality. The standardization of MPEG-2 has been implemented in the same level of competition and cooperation as H261 and MPEG-1. The bifurcation of competition and cooperation in MPEG-2 was the MPEG conference held in Gurihama, near Tokyo, Japan in November 1990. JVC's laboratory conducted image quality evaluation on the images obtained from the algorithms proposed by each participating organization. Compared to the MPEG-1, the number of proposed institutions doubled from 16 to 32, and the application of the application to broadcasting has led to the participation of research institutes in the broadcasting field. After this evaluation, standardization was carried out in earnest based on a reference model called a test model. At this time, an event had a decisive impact on the standardization of MPEG-2. This was standardized by the Federal Communications Commission (FCC), which was pushing for the development of high-definition television (ATV). ATV was originally a national project that began in 1987 to determine America's next-generation broadcast system against Hi-Vision, Japan's high-definition television. Due to the nature of the project, it was forced to take on political colors, but on the other hand, it also brought a great technological leap. In other words, GI (General Instruments), one of the participating companies, announced a 15Mpbs degicipher method in 1990, showing that it is possible to compress information to obtain HDTV quality at this rate. This incident was particularly shocking to many who have been involved in the development of high vision in Japan. Later, ATV standardization in the United States entered a fierce competition by GI AT & T Zenith Thompson Phillips, which formed three consortiums and proposed four methods. Since there were no clear winners in the test of the four digital HDTV schemes, the FCC recommended unification to ATV proponents and eventually formed the Grand Alliance (GA) in the spring of 1993. The GA was eventually approved in October 1993 with a unification plan in a MPEG-2 compliant manner. Thus, MPEG-2 was formally adopted as the next generation TV broadcasting method in the United States. Similar movements took place in Europe. The EUREKA95 project, which is based on the analog HD-MAC method, was discontinued, and in October 1993, the DVB (Digital Video Broadcasting) project was newly launched, focusing on EU member states. The project sets the specification for the next generation TV system based on MPEG-2, laying the foundation for the digitalization of CATV, satellite broadcasting and terrestrial broadcasting. In keeping with the progress of the ATV project in the US, MPEG-2 decided to absorb MPEG-3 and target HDTV quality as it does not have time to make MPEG-3 for HDTV standard later. March 92 meeting). With these twists and turns, the standardization of MPEG-2 has progressed and convergence has finally taken place. Finally, TM0 (Test Model 0, January 1991 Singapore meeting), TM1 (March Haifa meeting) and TM2 (July Rio de Janeiro meeting) Improvements were made and finally the CD (Committee Draft) was completed in November 1993, including the HDTV, including the HDTV. The technical details of the standardization work were almost completed. The standardization work has been carried out with the aim of realizing the broadcast quality, and recently standardized, MPEG-2 has been equipped with digital satellite broadcasting (including DBS through the Korean satellite), high-definition TV, digital video disc (DVD), Employment has been decided in many fields, such as video on demand (VOD), which is driving the multimedia revolution. In particular, the DVD, which Sony Philips and Toshiba Matsushita, agreed to in recent years, can hold a 130-minute movie of MPEG-2 quality (average 6 Mbps) on a single sheet, which greatly affects the current video CD, VHS, and digital VCR. In addition, by realizing the large capacity of about 5GB, the ripple effect is expected to be very large in the field of computer auxiliary storage devices such as HDD, CD-ROM.

-MPEG2 video

MPEG2 video contains MPEG1 video, which maintains forward compatibility. While MPEG1 video stores video at low bit rates of 1.15 Mbps on digital storage media such as CDs, MPEG2 video is used to transmit and store high-quality video on higher bit-rate broadcast, communication, and storage media. As the range of applications varied, so too did the requirements to be met. The world's leading companies have proposed MPEG-2 video alloting to meet these requirements. Finally, in September 1991, 30 methods were evaluated by Japan's JVC Research Institute. 20 of them were DCT-based, like MPEG1, 5 were subband coding, and 5 were wavelet transform. As a result of the evaluation, DCT-based schemes still prevailed somewhat, and considering the compatibility with MPEG1 video, the standardization of MPEG2 video was decided as DCT-based scheme. MPEG2 video is a general-purpose compression algorithm that greatly expands and develops MPEG1 video, providing a number of tools to properly select and use them according to the application. In order to improve the compression efficiency, MPEG2 video has been considerably improved by reviewing each element of MPEG1 video and improving it little by little. In other words, many improvements have been made, including field-based processing, motion estimation and compensation, quantization, scanning of DCT coefficients, and variable length coding. The range of MPEG2 video is very wide, but each application uses only a specific function at a particular resolution. Therefore, MPEG2 video is classified into several types according to the resolution and function for the convenience of the encoder and decoder. First, the resolution of the screen is classified into four levels. Simple, the small screen like MPEG1 video targets, Main, the current TV screen size, High 1440, the screen size of HDTV HD, and High, the standard for US high-definition TV. will be. Depending on the function, it is divided into five profiles. SNR Scalable, Spatial Scalable, High, etc. are the main, hierarchical, and more advanced functions that are adopted for most applications, including simple and easy to implement, except B-frame using bi-prediction. It is. Specific examples of applications include digital DBS broadcasts using Mugunghwa satellite, which will be implemented from next year, and DVDs that Toshiba and Sony have recently agreed to. The main profile is the main profile. HDTV is Spatial Scalable Profile. High-1440 level, and American digital cable broadcasting is simple profile. Main level. The main purpose of MPEG2 video is to compress current TV or HDTV efficiently. The current TV picture quality is obtained from 3 to 9 Mpbs, and the HDTV picture quality is obtained from 17 to 30 Mpbs. The bit rate is selected in consideration of the capacity and required quality of a given channel. For example, about 7 Mbps for Mugunghwa satellite DBS, 17 Mbps for the US Grand Alliance HDTV system, and 5 to 6 Mbps for video or DVD on demand (ADSL-3 system) are allocated to video. In MPEG2 video, unlike MPEG1 video, which is intended only for progressive scan, which has been adopted by computers, much attention has been given to the interlaced video format used in TV. That is, one screen (frame) may be divided into two fields (a field consisting of even scan lines and an odd scan line) and encoded in a field structure, or encoded in a frame structure. Since a scene with a lot of motions has a big difference between two fields of one frame, it is effective to encode the field structure. The closer to the still image, the higher the correlation between two fields. In the frame structure encoding, each macroblock (16x16 pixel units) can be processed on a field-by-field basis so that partial motions on the screen can be easily processed. Many factors combine to make MPEG2 video more capable of compressing video than MPEG1 video.

MPEG2 video improves on several elements of MPEG1 video in order to achieve high compression rates. As mentioned in the last issue, both frame and field structures are accommodated for efficient compression of interlaced TV image signals. Even in the case of a frame structure, motion compensation prediction may be selectively performed in units of frames or fields for each macroblock 16 × 16. Newly adopted motion estimation and compensation method in MPEG2 is Dual Prime. This compensates for field-level motion, but effectively reduces the amount of motion vectors. This method still contributes to maintaining good image quality as a complementary measure to reduce the delay time and complexity of coding by omitting B frames. In the DCT, the frame structure can be selected that has a small amount of data generation in the frame mode and the field mode in units of macroblocks. Thus, both high and low motions can be effectively handled. However, only one field DCT is used in the field structure. In quantization of DCT coefficients, linear quantization in which MPEG1 has a constant quantization step is used regardless of coefficient size. On the other hand, in MPEG2, the smaller the coefficient value, the smaller the quantization step, so that non-linear quantization is used. The nonlinear method increases the complexity but reduces the average quantization noise, which leads to an improvement in the performance of the quantizer. In scanning for variable length coding of quantized DCT coefficients, only zigzag scanning is used in MPEG1. However, Alternate Scan method is additionally used in MPEG2, and one of the two is selected and used as the screen unit. Alternative scans have a relatively early scan of the high frequency components of the DCT, which is particularly effective for high interlaced scans. Only one two-dimensional VLC table was used in MPE G1 for variable length coding (VLC) of the scanned DCT coefficients. On the other hand, in MPEG2, an additional table for intra picture coding can be used. Intra picture encoding requires a separate table that reflects Roy because the DCT coefficients are much larger than the inter picture coding and have different statistical characteristics. By using this VLC, the amount of data generated during intra picture coding can be greatly reduced. The scalability function is a new concept introduced in MPEG2, and includes spatial scalability, temporal scalability, and SNR scalability. Spatial scalability first divides the picture into base layers with low spatial resolution (e.g. current TV) and high seniors (e.g. high definition TV). The base layer is encoded first, and then the interpolation component of the base layer and the difference component of the higher layer are encoded, and the two encoded bit strings are sent together. In this way, even the current TV receiver can decode the base layer bit string to view high definition TV with the current TV quality, and the high definition TV receiver decodes both bit strings to reproduce the high definition screen. In other words, as in the case of black and white TV and color, a digital TV receiver or a high definition TV receiver can receive both a digital TV broadcast and a high definition TV broadcast, thereby maintaining full compatibility. European digital TVs and high-definition TVs are based on this framework. Meanwhile, Korea's high-definition TV takes the form of simultaneous broadcasting (Simulcast), which exports the same program to digital TV broadcasting. Similar to spatial scalability, temporal scalability and SNR scalability are divided into a base layer and a high layer, and the encoded bit stream of the base layer and the encoded bit sequence of the difference component between the base layer extension and the high layer is sent. However, in the classification of the base layer and the high layer, the temporal scalability is divided by the time axis (the progress direction of the screen), and the SNR scalability is divided according to the resolution of the bit representation for each pixel. As such, MPEG2 video has various functions and superior animation compression capability than MPEG1 video, and it is used as a leading technology in multimedia era as it is used in broadcasting media including high-definition TV, communication media such as on-demand video, and DVD. Hold.

-MPEG2 audio

In response to the high quality of PEG2 video, MPEG2 audio has also been standardized for the purpose of multichannel high quality. Thus, MPEG2 audio was approved in November 1994 as International Standard 13818-3, and standardization was completed. MPEG2 audio is based on MPEG1 audio and several new techniques are introduced to improve compression efficiency. Compared to MPEG1 audio, MPEG2 audio includes the following features. First of all, it is multi-channelized. The stereo function of MPEG1 audio has been extended to 6 channels in MPEG2 audio, so that stereoscopic sound in a movie theater can be enjoyed as it is. Looking at the components of the six channels (i.e., the location of the speakers), five wideband signals (20KHz) and low frequencies of C (Center), L (Left), R (Right), LS (Left Surround), RS (Right Surround) It consists of a Low Frequency Enhancement (LFE) signal that provides only the component (120KHz) separately. This is commonly referred to as 5.1 channel, which is more three-dimensional than conventional Dolby Surround stereo sound consisting of four channels of L, R, LS, and RS. In addition, the Multi lingual function was enhanced. As can be seen from satellite broadcasting in various regions, such as Asia and Europe, the global village is becoming one with the full bloom of multimedia and information and communication technology. Correspondingly, MPEG2 audio has a function to send additional voices of up to seven languages in addition to 5.1 channels. MPEG2 audio also enables the use of sampling frequencies of 16KH z, 22.05KHz and 24KHz, which are half the sampling frequencies used in MPEG1 audio. This is because it is advantageous to reduce the sampling frequency when the bandwidth of the input signal is narrow in order to effectively compress a large amount of data of multichannel and multiring at a limited bit rate. Another consideration is backward compatibility with MPEG1 audio. This means that the MPEG2 audio bitstream can be played back in a limited way in the MPEG1 audio receiver. Specifically, 5.1 channel MPEG2 audio is reproduced in stereo in the MPEG1 audio receiver. For example, in the case of digital broadcasting using Mugunghwa-ho satellite, MPEG2 is a standard. A low-end model equipped with an MPEG1 audio decoder also receives MPEG2 bit strings and reproduces stereo sound. This is the same concept that color TV broadcasts are received in black and white, even in black and white TVs, and FM stereo broadcasts are received in mono. This backward compatibility with audio contrasts with the forward compatibility with which an MPEG2 video receiver in video can receive an MPEG1 video bitstream. More precisely, in audio, when designing an MPEG2 audio decoder, it is common to design an MPEG1 audio bit stream so that bidirectional compatibility is substantially maintained. For this compatibility, MPEG2 audio puts stereo components in the audio data portion of the bit stream of MPEG1 audio and additional components of MPEG2 audio in the subsequent additional data portion. When there is not enough bit space, the format (syntax) of the bit string is extended and the rest of the data is put there. As a result, the bit string format of MPEG2 audio has become very inefficient, which is one factor that degrades the performance of MPEG2 audio. To compensate for this, MPEG standardized the NBC (Non Backward Compatible) mode, which improved backward performance with MPEG1, instead of backward compatibility with MPEG1. NBC plans to complete international standards in 1997, including Dolby's AC3, which has been adopted as a standard in the US HDTV and film industry and is also likely to be adopted in recent digital video discs.

-MPEG2 system

The MPEG system is a standard for transmitting and storing MPEG video bit streams and MPEG audio bit streams. When multiplexing into one bit string like this, it is necessary to have a format suitable for a protocol or a storage format of a communication channel or a storage medium. In addition, providing a means of synchronizing video and audio (Iip sync) is also an important role of the MPEG system. The MPEG system includes the MPEG 1 system and the MPEG 2 system, which are already discussed. The MPEG 1 system multiplexes a single program in an error-free channel environment and is therefore used for a relatively narrow range of applications such as video CDs. More precisely, the error is corrected by the error correction capability of the channel, so the error is not considered in the MPEG1 system. On the other hand, MPEG 2 systems correspond to a wide range of applications such as broadcasting, communication, and storage media, and the format is much more complicated. There are two types of multiplexing in the MPEG 2 system. One is called Program Stream (PS), which multiplexes a single program in an error-free channel environment, a slight improvement over the MPEG-1 system. The other is a transport stream (TS), which multiplexes a plurality of programs in an error channel environment. Since multiple programs are multiplexed into a single bit stream, it is suitable for digital TV broadcasting in the multimedia era, and a scramble function for restrictive reception (encrypting the bit stream so that only paying subscribers can watch) can be added. In addition, directory information, individual bit string information, and the like can be loaded to facilitate random access. Since the PS is similar to the MPEG 1 system, we will mainly describe the TS. The MPEG 2 system adopts a packet multiplexing scheme used in time division multiplexing (TDM). At this time, each of the video and audio bit strings is first divided into a packet string of a proper length called a packet (PES). The PES packet has an upper limit of 64 KB in length to accommodate various applications, and can take either fixed length or variable length for each packet. Variable rates are also allowed and discontinuous transmission is possible. Each PES is multiplexed into one bit string to form a PS or a TS. Packet length is highly dependent on the transport channel or medium. For example, in the ATM (Asynchronous Transfer Mode), which is a protocol in a broadband integrated information network (BISDN), a 53-byte packet (cell) is used. Since the header containing the basic information about the double packet takes 5 bytes, the actual user information (Payload) is 48 bytes. As such, a short packet occupies a relatively large ratio of headers, which decreases transmission efficiency of user information, but has a low delay time and a small amount of buffer memory. The TS packet has a relatively short fixed length of 188 bytes in consideration of connectivity with ATM. If one byte of the 48-byte user information of the ATM cell is used for ATM Adaptation Layer (AAL), the actual user information is 47 bytes. Therefore, one TS packet can be carried in four ATM cells. Since the first 4 bytes of each TS packet are for headers, the remaining 184 bytes are part of user information carrying actual video or audio. In many applications, the code for error correction is added, and the length of the TS packet is determined in consideration of this. That is, to apply the Reed Solomon code having the best performance as the block error correcting code, the length of the TS packet is preferably smaller than 255, so it is determined to be 188 which satisfies the connectivity with ATM. For example, in digital broadcasting using Mugunghwa satellite, RS (204, 188) which adds an error correcting code of 16 bytes to each TS packet is used, and the receiving side can correct errors of up to 8 bytes out of 2,400 bytes. . In many cases, Reed Solomon codes that are resistant to clustering errors and TCM (Trellis Coded Modulation), which improves performance by combining convolutional codes or convolutional codes that are resistant to sporadic errors with modulators, are used together.

The MPEG2 system handles two types of multiplexed bitstreams. Among them, the program stream is a multiplexing method used when one broadcasting program (video + audio + subtitle) is used in error-free channel environment or CD, as seen in an error-free channel environment or CD. Transport Stream (TS) is a multiplexing method used to send several broadcast programs simultaneously in an error channel environment. For example, a program stream is used to store a program, such as a video CD, and a transport stream is used for digital broadcasting of multiple programs using a green light satellite. Let's take a look at the function of the transport stream in more detail with an example of Mugunghwa satellite broadcasting. Mugunghwa satellite is a broadcasting and telecommunications satellite with three broadcasting repeaters and 12 communication repeaters (although Unit 1 will be shortened due to the failure to launch, so that Unit 2 will be replaced). In the case of satellite broadcasting, the analog FM system such as Japan's satellite broadcasting or Star TV in Hong Kong can accommodate only one broadcast per repeater, but the digital method using MPEG2 can accommodate up to 4 to 8 broadcasts per repeater. have. In Korea, considering the lack of programs and image quality, 4 broadcasts per repeater are being considered. In this satellite broadcasting, multiplexing consists of the following steps. First, the program from each broadcasting station is compressed to about 30 to 1 and 6 to 1 video using MPEG2 video and audio using MPE G2 audio compression algorithm. This compressed bit string is packed into packets and transformed into video packets and audio packets, respectively. They are then loaded one by one into several transport stream packets with a fixed length of 188 bytes. One transport packet can carry 184 bytes of actual load, except for the 4 bytes of the header. It's like a taxi seating up to five people, but without the driver, you can ride four passengers. The header contains 13 bits of Program Identification (PID), which is used to indicate what station (ie video or audio) information is contained in the packet in this packet. In this way, a transport packet primarily multiplexed by each broadcasting station is secondly transported by multiple broadcasting stations, forming a bit string, which can be transmitted through one repeater. This final bit string is needed by the number of repeaters. Therefore, in digital satellite broadcasting, multiplexing is combined with time division multiplexing (TDM) and frequency division multiplexing (FDM), ie, repeaters operate in the form of FDM, each having a bandwidth of 27 MHz. However, four broadcasters share one repeater in TDM mode, with transport streams on each repeater receiving Reedsolomon codes and convolutional codes for error correction and transmission between terrestrial and satellite via QPSK modulation. When the transport stream is decoded, the above reverse process is performed, first select the repeater containing the broadcast to be received, QPSK demodulate the signal, and perform error correction. Select only the transport packet of the station you want to receive, and the double video packet In the coder, audio packets are decoded in the audio decoder to reproduce video and sound, respectively, and several program specific information (PSI) programs are required for this multi-step operation. It is a packet with PID = 0, which allocates a transport packet for each program, and when you go to the designated packet, it tells which packet contains the video and audio bit streams that make up the program. Map Table) is divided into PAT and PMT and described in tree form. If you describe all as one table, this table becomes too large and the memory to store the table becomes large. Because it takes a long time to access the program's information. The like NIT (Network Inform ation Table) that holds the link information between programs and (Conditional Access Table) that holds the CAT condition information received is used in the additional information table for the system operation.

MPEG-IV: A technology that improves MPEG-II and is developing for video conferencing system and video data transmission using telephone line. It can be understood as the direction of most streaming companies, and it can be compressed by individual object when compressing video.

In Dallas, USA, an international conference was held to evaluate the subjective picture quality of MPEG4, the next-generation multimedia international standard. This meeting, which was held in great interest with the attention of the world, had a significant meaning in which the future direction of MPEG4 was determined. In Korea, electronic companies such as Samsung, Hyundai, Daewoo, etc. actively participated in the proposal. While JPEG, H.261, MPEG1, and MPEG2 are all standards based on DCT, quantization, motion compensation DPCM, and Huffman coding, MPEG4 is a next-generation compression standard that is expected to be completed in 1998 for multimedia communication. (MPEG3, which was originally planned as a standard for HDTV, was absorbed and integrated into MPEG2.) Considering that the current standard accommodates many applications, early MPEG4 was designed for low bit rate encoding aiming at video telephony using public telephone networks. Focused on. Since then, MPEG4 has expanded its scope and its functions, and its main application areas are interactive access or wireless communication of AV data such as TVs and movies. The features of MPEG4 are largely divided into object-oriented interactive, high efficiency compression, and universal access. The object-oriented interactive function refers to a function in which multimedia elements (mainly AV) data accesses independently handle object elements of a screen or sound while combining them by linking with each other to freely configure the screen or sound. For example, the process of replacing only the main character with the background on the screen was previously possible only in the production stage, but in the MPEG4 user stage. High efficiency compression is a next-generation standard and must provide improved compression rates over existing methods. In addition, in general-purpose access, it is necessary to make the immunity strong even in the environment with many channel errors in consideration of the wireless communication environment. A single algorithm that satisfies all of these features is virtually impossible, so MPEG4 accepts many compression elements as a menu in the standard and selects them according to the application. In other words, you decide which tools you need for compression, combine these tools to create multiple compression algorithms, and then tie together one or more algorithms to create a profile that you choose according to your application. The hierarchical structure of these tools, algorithms, and profiles define and define a new language called MPEG4 Syntactic Description Language (MSDL). Therefore, data transmission and reception between MPEG-4 terminals first confirms which profile, algorithm, and tool decoders are available, communicates in a decodable mode, and, if necessary, first downloads a program necessary for decoding, and then transmits the contents. In the last MPEG4 subjective picture quality assessment, several selected bit rates in the range of 10Kbps to 1Mbps were evaluated by function for the specified video picture, from the simple to the complex. As a standard for comparing the degree of technological innovation, H.263 is an international standard for video conferencing using telephone lines, which is a relatively well-optimized standard. Many proposals are variants of H.263, and some have applied new techniques, such as fractal or wavelet transform, and the evaluation shows that there are few techniques better than H.263. did. Looking further into the room for technological innovation, 92 proposals were received from 27 institutions and 19 proposals were received from audio. The video compression algorithms proposed in Korea showed the mid-range quality of video, but the active activities in MPEG4 are a remarkable development, compared to the acquisition of data and trends without the proposal up to MPEG2. In MPEG4, new tools and algorithms will be proposed and evaluated, and the patents included in MPEG1 and MPEG2 have recently been returned to the patent holders with great compensation ($ 3 patent fee per MPEG receiver). There is a need to pay more attention to MPEG4.

All. M-JPEG

1) What is Joint Photographics Expert Group (M-JPEG)?

Streaming compression method based on the standardization standard created by the Still Image Expert Group, the original compression algorithm that enables faster streaming service by increasing the frame level with the high density and high computational capacity solved by the Still Image Expert Group. JPEG compresses data through the CPU without any extra equipment. M-JPEG took advantage of these jpegs to eliminate the buffering process.

However, in the existing jpeg method, since it was literally a still image professional, streaming implementation was virtually impossible. Literally, a still image means a processing method of a still pixel. Of course, 3-4 frame Mjpegs were also developed and used by several companies, but high-density streaming quality of more than 15 frames per second was impossible. It is also true that the problem of voice implementation has become a bigger problem. Because high density compression is possible, there is a problem in that a part that can insert streaming elements such as operations on duplicate values of data or the like is almost blocked.

2) Compression rate and video quality

Whatever compression technique is used, image quality and compression rate are inversely proportional to each other. The higher the compression efficiency, the lower the image quality. Conversely, the lower the efficiency, the better the image quality.

Modern image compressors can compress up to 10 to 50 times without visual loss of the image.

Currently, compression technology is remarkably developed, and techniques are being developed one after another to increase the compression efficiency but not to degrade the quality of the original image.

Compression ratio and image quality are as follows.

An object of the present invention for solving the above problems is to use a conventional MPEG-based compression method, the buffering process so that the video playback in real time on the Internet by skipping the buffering process without using a separate video player , Eliminate the overload of streaming servers, minimize the bandwidth of video and audio signals, minimize the burden on the line, eliminate unnecessary download processes, eliminate the use of separate players, and minimize the burden on server specifications It is possible to reduce the cost and to provide a way to work by porting it to a general-purpose browser using Java classes or applets.

The purpose of developing a new concept of compression technology and streaming technology is as follows.

1) The wired Internet MPEG-based streaming solution can be replaced with this technology.

2) By substituting streaming server of internet broadcasting station site, server cost is reduced and line cost is smoothly provided, and it is possible to watch video without slow client environment without burden.

3) It is a solution to solve the current Internet traffic problem caused by multimedia video, and it is a breakthrough technology that can reduce trillions of wons in line cost and infrastructure construction nationally.

4) It is the most suitable streaming solution for wireless internet terminal. Low capacity and high quality image providing technology will be the most important and core technology for next generation wireless mobile communication. It is the most suitable technology for CDMA2000, IMT2000 and PDA, HPC.

5) Develop streaming solution that can be applied to all video solutions of wired and wireless internet terminals.

6) Developing wireless internet video transmission equipment by developing encoding and decoding chips corresponding to this compression algorithm.

7) The next step in the development of this technology is to globalize new compression and streaming technologies and continue to develop key technologies in the wireless Internet market.

1 is a diagram of basic DCT transform coding.

2 is a diagram showing a configuration diagram of band division coding.

3 is a diagram showing a method of band division / DCT linkage in an SPEG frame;

4 is a diagram showing the configuration of a motion compensated interframe encoder;

5 shows a movement displacement measurement.

6 is a diagram showing coding of motion compensated prediction using DCT.

7 shows preprocessing and encoding for standardization.

8 is a diagram showing encoding through standardized hierarchical resolution.

9 is a diagram showing video encoding in a circuit switched network and a SPEG packet switched network.

Fig. 10 is a diagram showing clock synchronous reproduction at a variable bit rate using a DPLL.

Fig. 11 shows the definition of y [i, j].

Fig. 12 shows the compression of one block.

Fig. 13 shows the initialization of a combination arrangement.

14 shows an implementation of the Convolve function.

Fig. 15 shows a strategy for image manipulation.

Fig. 16 shows the C implementation of the Dissolve operation.

Figure 17 shows a C code implementation.

We have developed a new concept of compression algorithms and streaming techniques that are completely different from the MPEG method, which is the basis of video compression and streaming technology, to enable more efficient video streaming.

Features of the present invention

1) There is no buffering process that has always been a problem when watching existing Internet videos.

2) Overloading the video providing server (streaming server) has been solved.

3) The new compression method has dramatically lowered the data capacity and minimized the line load.

4) There is no need to download a separate player or program.

5) You can use Java classes or applets to increase the portability of the browser.

It is possible to replace the current MPEG-based streaming server by developing a new compression method and streaming technology with this advantage.

In the configuration of the present invention to achieve the object as described above and to solve the conventional drawbacks in configuring a method for compressing and streaming a video using a lossy compression method using SPEG on the Internet,

Determine the limit value of the frame to be compressed and then compress it for each object,

Step 2 to give the index values for each image and then sort them in order,

After the image of the object is compressed, the overlapping part is removed in three steps, leaving only one index value.

The fourth step of allowing the internal operators of the streaming server to reside in the user group of the client when the client connects to achieve mutual synchronization without a separate player;

The internal operators can be displayed in five steps so that they can be displayed as applets or classes in a general-purpose browser on the web using Java.

The first step of determining the limit value of the frame to be compressed and then compressing each individual object;

Rather than determining the number of frames, it sets the limit value, performs the calculation operation internally, and simultaneously decompresses several frame signals after delimiting different operators.

The second step of giving the index value for each of the respective images and then sorting in sequence;

Removing duplicate operation values and pushing them out of the limit operation value;

In this step, the number of frames is determined, and the frame operation is mainly performed by an internal operator, and the compression is performed by the CUP to remove buffering.

After the image of the object is compressed, the overlapping part may be removed while leaving only one index value;

After the compression step, each index value is recalculated one by one internal operator for each interval section of the compression format to unify the size of the video signal and the audio signal so that the mutual sinker part is made here to synchronize the bandwidth. It consists of densely limiting the limits.

In the method of unifying the magnitude of the video signal and the audio signal, the internal operator is combined with the video signal in the mono compression method of gsm to control the internal filter itself so as to prevent the overload of the streaming server.

In the control method, the steps of sharing and controlling the roles of operators are as follows.

That is, the basic operation of the video signal processing operator and the audio signal processing operator dedicated to the signal recognition operator processing that accepts only the signal;

Residing at an internal memory address of a memory filtering operator for filtering the two signals after the operation;

The specialized operator takes the cpu from the cpu to the moment when necessary and processes the operation.

Gsm is a voice compression method completed by a German professional speech recognition user group.

This is the most common voice compression technology used on phones.

Mono, however, has sound quality limitations that streaming professionals have refrained from using despite their strong compression ratio.

Hereinafter, the configuration and the operation of the embodiment of the present invention will be described in detail.

The present invention was conceived that the compression method of the existing media player and the real player can not exceed the limitation of the buffering that is being made in the base, and came up with a completely different compression method.

Therefore, the present invention has been studied in the point that it is not possible to view the streaming data comfortably and more quickly and stably from the user's point of view by eliminating the troublesome buffering that occurs when using the existing method.

Therefore, the existing mpeg method was boldly abandoned and a new approach was made from the original compression technology.

Therefore, M-Jpeg method is adopted, but this method also has many problems. As a result of several studies, it provides a more stable and efficient compression method and provides a streaming method with little buffering feeling.

More detailed description is as follows.

The present invention is achieved by providing a new concept of SPEG, ie, giving more complete continuity for each property of a frame. By dedicating continuity to the concept of frames through external operators, we were able to create more powerful streaming data that could be handled by small servers.

In the compression scheme, which sees the buffering of the streaming and the sinker process, the internal operators are dedicated to the internal operators, which are literally dedicated to complex streaming operations. In the present invention, buffering is eliminated through this internal operator.

Internal operators are easy to understand in terms of processors. The internal processor resides in the cpu and handles the complex load computations that the streaming server handles whenever the voice and video signals come in.

Of course, because of the cpu load problem, the basic processors of internal operators reside in the memory main address to help with the complicated operation of internal operators.

Therefore, the problem of expensive streaming server could be solved, and these could enable faster association process.

This internal operator programming process is implemented in the assembler closest to the machine language and follows the method of loading the components of internal operators into memory main memory for lighter operation.

In order to avoid collision with the existing browser, Netscape, Explorer, and Mozilla screen transfer methods were supported as much as possible. For faster and more powerful processing, signal processor and pixel operator voice processing operators were separately controlled to control the processors.

Therefore, it was possible to complete progressive motion jpeg (SPEG), a new concept of compression technology that goes far beyond Mjpeg's use of existing jpegs.

Video compression technology

Prior to the description of the image processing method of the present invention, basic image information is provided to the viewer as a series of images or "frames", the movement of which is represented by a small change between successive frames.

In this compression method, compact image compression is performed on each frame constituting the frame.

Since images are delivered very fast at a rate of about 30 frames / second, successive changes between frames appear to be naturally moving scenes in the human eye. The video screen is composed of space and time domain information. The space domain information is provided in each frame, and the time domain information is provided as a change existing between images, that is, frames, over time.

In other words, the compression rate of the image in streaming depends on it. The lighter the frame weight, the less burden there is on the system.

In order to kill the burden, each frame is divided into pixels.

In a digital imaging system, each frame of an image is sampled in "pixels". Sample values representing the brightness of pixels are quantized to 8 bits per pixel in black and white images. In a color image, each pixel has three brightness information representing color information, for example, RGB (red, green, blue) and is quantized to 8 bits. Since the image information thus constructed has a very large amount of information, an image compression (or encoding) technique is required for transmission or storage, which mainly uses a method of removing redundant information in the space and time domains.

In general, the redundancy in the spatial domain is due to the small degree of change between adjacent pixels in one frame, and the redundancy in the time domain is due to the slight difference in the movement of an object between frames. The method of removing redundancy in the spatial domain is called intraframe coding, which can be roughly divided into predictive coding (DPCM), transform coding, and subband coding. On the other hand, the redundancy of the time domain can be removed by using an inter-frame encoding method, which includes a motion estimation / compensation method that estimates and compensates for the motion of an object. In intra-frame coding and inter-frame coding, run-length coding, variable length coding, and the like, which are further compressed without loss by using statistical properties of data, are used.

In-frame coding

Predictive coding

Predictive coding is the oldest image compression technique. The concept of predictive coding is that the prediction error is very small when the current pixel is predicted from neighboring pixels. The predictive coding technique quantizes a difference (prediction error) between a current pixel value and a predicted value, and then encodes it. When prediction is performed using a large number of adjacent pixels, the prediction error may be reduced to improve performance. Since the advantages are not large compared to the complexity, the number of neighboring pixels used for prediction is usually 4 or less.

Types of image quality degradation in predictive coding include grain noise, gradient overload, and edge blur. The grain noise is due to the size of the quantization step is too large, and the gradient overload is due to the size of the step is too small, and have a mutually opposite relationship. Grain noise and gradient overload appear in the reconstructed image as a distortion of the boundary between the noise and the object within each frame of the image signal. When the video signal is displayed continuously in time, corner blurring occurs because pixels corresponding to object boundaries are quantized differently in neighboring frames.

Transform Coding

1 illustrates a basic DCT transform encoder. The basic concept of change coding is to obtain image data having a high compression rate by performing orthogonal transformation of image data to remove correlations present in the image.

Under the assumption that the image data remain statistically constant, the Karhunen-Loeve Transform (KLT) is known as the optimal transform in terms of mean squared error. However, because KLT's basic function is data dependent, it is very difficult to apply to applications that require real-time processing because of the disadvantage of sending basic function to decoder and the difficulty of practical high-speed calculation. Therefore, the orthogonal transform closest to KLT uses the discrete cosine transform, which maintains good performance even if statistical assumptions about the image's properties are not established.

Coding scheme of the present invention (band division coding)

2 is a configuration diagram for band division coding.

As shown in FIG. 2, band division coding is largely divided into two steps.

The first step is a band division step of dividing a video signal into each frequency band, and the second step is an encoding step of compressing each divided band according to its characteristics.

(1) fully reconstructed band split filter

The band division coding involves a decomposition filter bank in an encoder and a synthesis filter bank in a decoder. The decomposition filter bank decomposes an input into several bands having different sampling rates according to frequency bands. On the other hand, the synthesis filter bank increases the sampling rate of the decoder input and synthesizes the desired signal. Band split coding requires a short processing time for one band, but requires a plurality of processing devices (one processing device for each band).

(2) Coding of Band-Divided Image

3 is a diagram schematically illustrating a process of band division / DCT linkage in an SPEG frame.

The human eye does not find small changes in pixel values in the high frequency band. Therefore, a simple nonuniform quantizer with a dead zone that makes the surrounding small pixel values zero can be used without severe visual degradation, and higher compression ratios can be obtained by length-encoding non-zero values and consecutive lengths of zero. It may be. The quantizer zeroes out many of the pixel values in the high-frequency band in the quadrature region, resulting in longer zero continuous lengths.

In the band division coding method, since the sampling rate decreases after band division, it is frequently used for difficult ultra-fast encoding processing when bands are not divided as in high-definition TV.

Interframe Coding of the Invention

4 is a block diagram of a typical inter-frame encoder. In this basic configuration, there are two procedures: first, motion estimation and compensation, and second, compression after motion compensation. The motion of the object is estimated by calculating the relative displacement of the image data corresponding to the previous frame, and is similar to the predictive encoding that predicts data from the current data and the previous frame.

Motion displacement estimation

5 is a diagram illustrating a principle of measuring movement displacement.

Each frame of the image is first divided into N × N partitions. There are various criteria for determining motion estimation, such as a mean square error and an absolute error of a difference. As shown in FIG. 5, a given search area (N + 2L) Within x (N + 2L), where L is the maximum range of permissible displacements, the motion displacement at which the difference between the compartment in the current frame and the compartment in the previous frame is minimized is used for compensation.

Compression Coding After Motion Displacement Estimation

6 is a diagram illustrating a motion compensated predictive encoder using DCT.

The purpose of motion displacement estimation using a motion compensated prediction encoder as shown in FIG. 6 is to estimate current image data (or partitions) from previous frames or adjacent frames so that time redundancy can be easily reduced. The most widely used technique for reducing temporal redundancy is a motion compensated predictive coding technique, in which a prediction error defined by the difference between the current segment and the segment compensated using the motion displacement estimation is encoded. If we can accurately predict the current partition from the previous frame, we can increase the compression rate while reducing the prediction error.

In general, the compressibility of a motion-compensated coding scheme depends on several factors.

o The amount of motion displacement that can compensate for the motion of the object on the video screen (the size of the motion search area)

o Accuracy of motion compensation method for estimating motion

Standardization Trend of Image Compression Technology

We acknowledge that existing streaming methods are based on the standardization trend.

And the compression scheme of the present technology must also be based on the technical standard.

However, as shown above, it is possible to reduce a considerable burden in the compression of each internal frame and the video and sound processing methods, and this compression method can also be compared with various compression technologies called standards.

Image compression technology can be divided into three stages of compressing the signal input from various video signal sources into a standard common format (CIF) through a pre-processing process. The compressed image is transmitted through the digital transmission path, received by the receiver, reconstructed, made into a standard common standard, and then displayed through post-processing. FIG. 7 schematically shows a preprocessing and encoding process for standardization. to be.

Video sources come in many different forms, including video telephony / conferences, TV signals (NTSC, PAL, SECAM), high-definition TV signals (1050, 1125 scan lines, etc.), VTR tape signals (VHS, S-VHS), and video film. In the same application field, the specifications, resolution, and bandwidth required for transmission are different. The common standard is a digital video standard in which different signal sources are formed through preprocessing, and facilitates communication between different models, and each standard has a hierarchical structure as shown in FIG. 8.

8 is a diagram schematically illustrating encoding through standardized hierarchical resolution.

Standardization Trend of Image Coding

However, it is worth noting that this compression scheme also has a comparable compression rate and transmission rate compared to the above.

In countries where 525 scan lines are used in TVs (including our country), the minimum common standard, QCIF (Quarter CIF), is a 180x120 pixel size for video telephony.

CIF, four times the size of QCIF, is used for multimedia services using video telephony / conference and storage media, and international recommendations are being made or nearly finalized through CCITT H.261, ISO MPEG I as shown in Table 2. to be. The CCIR specification is a digital standard for analog CATV and TV signals, which is four times the size of CIF, and is expected to be applied to digital transmission of TV signals, digital VTRs, high-definition video telephony / conference and multimedia services. The still picture compression method based on the CCIR standard has been standardized in MPEG, and the compression method for the image has been proposed in MPEG II.

However, this SPEG is more powerful and has a dense image compression rate.

SPEG packet video

9 is a diagram illustrating an image encoder in a circuit switched network and an SPEG packet switched network.

When transmitting a compressed video signal using a circuit-switched network, a buffer memory is used to fit a constant bandwidth. As shown in FIG. 9, a device for notifying the state of the buffer memory is connected to an encoder, and when the buffer overflows, the compression rate is reached. Increase the amount of data in the compressed video signal to avoid overflowing the buffer, and when the buffer becomes empty, reduce the compression rate to increase the amount of data in the compressed video signal or fill in the meaningless bits, thereby increasing the bandwidth transmitted from the buffer to the circuit-switching network. Is set to the given identity bit rate. Therefore, there is a problem in that the image quality is changed according to the state of the buffer, and also it is difficult to efficiently use a given bandwidth.

On the other hand, a packet communication network capable of transmitting a variable bit rate does not require a buffer to maintain a constant bandwidth, so that the design of the encoder is simpler and has the advantage of maintaining a constant image quality. Moreover, under the same quality requirements, packet multiplexing may bring about much better bandwidth utilization efficiency than circuit switched networks through statistical multiplexing of multiple video signals. Based on these results, packet transmission seems to be a good way to satisfy both "constant quality" and "efficient bandwidth usage."

Problems of Video Transmission in Packet Networks

The compressed video signal is formed into a packet by adding a header from a packetizer, where a delay occurs. This depends on the degree of change in the bandwidth of the compressed video signal, but depends on the degree of change in the bandwidth of the compressed video signal, but if the average bandwidth of the compressed video signal is 1 Mbps to 100 Mbps, the time delay is 5 ms to 400 ms on average. It is about. Encoding in circuit-switched networks In order to change the variable bit rate to the constant bit rate, a time delay of about 10 ms to 100 ms occurs in the buffer.

After packet formation, these video packets arrive at the receiving terminal through multiple packet multiplexing and packet switching networks. At this time, packet jitter and packet loss occur according to the traffic condition in the network. This packet jitter and loss should be compensated at the receiving terminal.

In addition, there are two main causes of packet loss in the network. One is the loss of packets due to the limited buffer memory of each packet switch and multiplexer as the traffic increases, and the other reason is that a bit error in the transmission system arrives at the intended receiving terminal if a bit error occurs in the header of the packet. This is a loss of packets due to failure.

4.2 Clock Synchronous Playback Under Packet Jitter

10 is a diagram illustrating a clock synchronous reproduction principle at a variable bit rate using a DPLL.

Synchronization between the transmitting terminal and the receiving terminal is required to restore the received video packet. Therefore, the receiving terminal must reproduce the clock of the transmitting terminal.

Because of this, the incoming clock must be regenerated through the incoming packets. However, the problem of clock synchronization is complicated because arriving packets are not constant due to jitter, and sometimes packet loss can occur in the network.

If the arriving packet has an equal bit rate, even if there is packet jitter, by observing the state of the buffer of the receiving terminal, the transmission clock can be extracted using a digital phase-locked loop (PLL), while the video packets are transmitted at a variable bit rate. In this case, since the buffer memory state of the receiving terminal does not represent the actual transmission clock, clock information cannot be extracted directly from the buffer memory state. Therefore, a commonly used method is to include clock information in a transmission packet and find it on the receiving side. A method of calculating a number of clock patterns detected for a predetermined time from a packet arriving first, and then comparing the number of clock patterns formed in a receiver clock (VCO) during the same time, and adjusting the receiver clock through a filter as a difference. .

The method shown in FIG. 10 is a time averaging method. Since jitter has a mean value of 0 on a long time axis and a difference in the transceiver clock continues to accumulate, the jitter is a method of adjusting the receiver clock with the difference to converge on the transmitter clock. The memory shown in Figure 4.2 continues to add the difference in clock patterns, so that over the first few cycles (in this case, about one minute), the jitter's effect remains on the receiver clock, but as time passes, it converges to the transmitter clock. It will play a role.

The use of clock synchronous reproduction allows for more compact and finer image processing of the compression technique.

In addition, let's look at the compression method of the following compression techniques through the above image media.

The internal compression structure of this technique is as follows. To mention more details by default;

Algorithms for Manipulating Compressed Images

The new technique implements operations directly on compressed JPEG data. This results in a performance improvement of 50-100 times over algorithms that must decompress and compress the results before application. Multimedia applications that operate on audio and video data will enable many new uses for computers.

For example, collaborative work systems include video conferencing windows and a hypermedia education system includes educational elements of audio and video.

Most research in multimedia applications has focused on compression standards, synchronization issues, storage media, and software architectures, and few techniques for manipulating digital video data in real time, such as the implementation of special effects and image composition, have been reported. .

For example, suppose you implement a brute-force algorithm to control the brightness of a compressed image by releasing the image, correcting each pixel value, and compressing the resulting image. The problems faced in implementing this approach on current workstations will arise from two sources. The amount of data to be manipulated (26.3 Mbytes per second for uncompressed 640x480 24-bit video at 30 frames per second) and the computational complexity of image compression and decompression.

The above describes algorithms that implement computations on compressed digital pictures.

These algorithms allow to perform 50-100 times faster than the brute-force algorithm of many traditional image manipulation operations. Since compression typically significantly reduces the amount of data with 25 or more compressions, this speedup is the result of performing operations directly on the compressed data. As a result of the speedup resulting from the small amount of data, most of the computations related to compression and decompression are eliminated, and traffic to and from memory is reduced.

It describes how to apply this technique and how to evaluate the performance of representative algorithms.

First, a compression model will be described.

It shows how algebraic operations and scalar addition and multiplication in the pixel direction, which are the basis of many image transformations, can be implemented in compressed images.

Use these operations to implement two common video variants. Overlap and fade one video sequence with another.

Implement operations and compare their performance brute-force.

Finally, we discuss the limitations of the technique, extend it to other compression standards, and relate this work to other work in this field.

Compression model

This chapter describes the compression model used in transform-based coding.

Starting with a general overview of transform coding, we briefly describe the CCITT Joint Photographic Expert Group (JPEG) standard for transform-based image coding.

A detailed description of the JPEG algorithm can be found elsewhere. Foley and Van Dam presented a detailed description of the picture format, and Lim discussed other variant coding techniques. All the results in this description appear in Lim's book and are mentioned here without proof.

Transform-based coding

A common technique for picture compression is deformation based coding. A typical variant coder handles the pixels of an image as many matrices. Linear transforms, such as discrete cosine transforms, apply this matrix to create a new matrix of coefficients.

To recover the original image, inverse linear transformation is applied.

The transformation has two effects.

First, it gathers the energy of the picture so that many of the modified coefficients are almost zero.

Second, it resolves the picture like a spectrum into high and low frequencies. Since the human visual transition system can hardly receive certain frequencies than others, some coefficients can be approximated more roughly than others without serious degradation of the picture.

A common way to characterize the latter is to quantize the coefficients. A simple way to quantize a coefficient is to truncate the lower bits from an integer (for example, right arithmetic shift operation). A way of controlling data loss rather than an arithmetic shift is to divide values by constants, quantize values, and integers. Round the value to approximate.

By multiplying the result by the quantization value, an approximation of the original value can be recovered.

The larger the quantization value, the coarser the approximation, but the greater the loss of large bits.

When the transform coefficients are quantized in this way, most of the coefficients are typically zero ('0').

For example, a compression experiment of 24: 1 shows that about 90% of the coefficients in the deformed image will be zero ('0').

JPEG algorithm

One of the standards for transform coding of still images is the JPEG standard. The remainder of this chapter describes the relevant properties of JPEG and introduces related terms.

Assume that the original image is a 24-bit image composed of three elements of one luminance (Y) and two chroma (I and Q) in a 640x480 pixel. That is, for each pixel in the original image, it is associated with three pairs of 8-bit values (Y, I, Q).

Because each element is treated similarly, only the Y element is described in the algorithm.

The Y element is divided into a series of four squares of eight pixels wide and eight pixels high. Each block is an 8x8 matrix of integers in the range 0-255. The first step in the algorithm is called the normalization step, which takes all the values from -128 to 127 by subtracting 128 from each pixel in the matrix. (This step is skipped for I and Q elements, since they are already between -128 and 127). The resulting matrix is y [i, j] and i, j 0 ... 7. 11 shows the relationship of y [i, j] to the whole image.

The second step in the algorithm (DCT step) applies a discrete cosine transform to this 8x8 matrix, producing a new 8x8 matrix. DCT is similar to a fast fourier transform whose value in the resulting 8x8 matrix is frequency related. That is, the lowest frequency component is top left and the highest frequency component bottom right. The new matrix is called Y [u, v], and when u, v 0 ... 7, by the definition of DCT,

In the third step of the algorithm, q [u, v] is obtained by quantizing the elements of Y [u, v] with frequency-dependent values. This quantization step is defined as follows.

The matrix of integer quantization values is called a quantization table, or QT. Differential QTs are typically used for luminance and chroma elements. The choice of QT determines the amount of compression and the quality of the decompressed image. The JPEG standard includes recommended brightness and chromatic QT due to human factor studies. A common experiment is to step through these basic QT values to get different picture quality. In particular, given two pictures with q1 [u, v] and q2 [u, v] of QT, for all u, v some constant gamma is often found as

You will use this value later.

The fourth step of the algorithm converts an 8x8 matrix from equation (2) into a vector of 64 elements using a zigzag shown in Figure 2.

This ordering is an empirical approach to bring together the low frequency components near the beginning of the vector and the high frequency components near the beginning.

We call it a zig-zag vector and this step is called the zig-zag scan step.

In most pictures, the vector Yzz will contain many sequential zeros. Therefore, the next step in the algorithm is run-length encoding, which encodes a vector as a pair of (skip, value). skip indicates how many indices in the Yzz vector must be skipped to reach the next nonzero value. It is stored in value. By convention, the pair (0,0) indicates that the value remaining in Yzz is all zeros.

At this point the block is called a run-length-encoded block (RLE), and each (skip, value) pair is called an RLE value. RLE blocks are denoted by YRLE [x], and YRLE [x] .skip and YRLE [x] .value represent the skip and value of the xth element of the array. Our algorithm works on RLE blocks.

In the last step, conventional entropy coding methods, such as arithmetic compression or Huffman coding, compress the RLE blocks. 12 graphically shows all the steps in the compression process of one block.

Tracking Note 3 backwards (that is, from right to left) explains how to release the data. The first step of release recovers the RLE block from the entropy coded bit stream. Restore the zigzag vector Yzz [x] to make it a single process via the RLE block YRLE [x]. From Yzz [x], recover YQ [u, v] by reversing the zigzag scan. Then we multiply each element of YQ [u, v] by q [u, v] from the appropriate QT to recover the approximation of Y [u, v]. In the last step, we obtain the picture block y [i, j] from Y [u, v] using inverse DCT (IDCT).

Equation (4) is very similar to equation (1), but the summation is done for u, v rather than i and j.

Using this technique, we can adjust the compression rate by changing the QT value.

Experience has shown that a compression ratio of about 24: 1 can be achieved without significant loss of image quality (i.e., 1 bit per pixel). At a compression ratio of 10: 1, the released image is usually indistinguishable from the original. Entropy coding reduces data size by about 2.5: 1. Therefore, the data size of the RLE block is typically 10 times smaller than that of the original picture if all of the compression is 25: 1.

Algebraic operations

This section shows how scalar addition, scalar multiplication, pixel direction addition, pixel direction multiplication, and four algebraic operations of two pictures are performed on RLE blocks. In the following calculation,

HRLE = (FRLE, GRLE)

FRLE and GRLE are RLE representations of input pictures. HRLE is an RLE expression of the output image. And GRLE is a function of real values. In an implementation, the values stored in the HRLE data structure will be integers. Therefore, the value returned by the function must be rounded to the nearest integer value. In order to simplify the notation in the following calculations, this rounding is implicit.

To make the notation even simpler, we perform all the calculations on quantized arrays FQ [u, v], GQ [u, v], and HQ [u, v]. Since the RLE blocks are data structures representing these arrays, the derived equations will be valid on the provided RLE blocks. We perform the appropriate index conversion. Another notational convention is

1. Capital letters F, G, H indexed by index u, v, w represent compressed pictures.

2. Lowercase letters such as f, g, h indexed by i, j, k represent decompressed images.

Greek characters such as * represent scalars.

4. QTs are represented in an array with subscripts representing images.

For example, the QT of the compressed picture H is qH [u, v].

5. The characters x, y, z represent zigzag ordered indices.

Often you will index QT with a single zigzag index (like x). In such cases the conversion to indices such as [u, v] would be implicit and clear from the context.

Scalar multiplication

Consider a scalar multiplication operation of pixel values. In this operation, if the pixel value of the original image is f [i, j], the value of the corresponding pixel in the output image h [i, j] is given as follows.

h [i, j] = f [i, j] (5)

Using the JPEG compression algorithm and the linearity of equation 5, it is easy to show the quantized coefficients of the output image. HQ [u, v] is just a different sized copy of FQ [u, v].

Specifically, using equations (1) to (5),

qF (u, v) and qH (u, v) are the QTs of the input and output images, respectively. Integer rounding is implicit at the end of the right. In other words, in order to perform a scalar multiplication operation on the compressed picture, one can perform directly on the quantized coefficients as long as the QT of the picture is taken. If the QT between pictures is proportional (as in equation 3), the equation is simplified to:

HQ [u, v] = FQ [u, v] (6b)

If the quality of the input and output images is the same, there is a special case where = 1. If a value in the input is FQ [u, v] is zero ('0'), the value corresponding to the output is also zero. You can implement this operation on an RLE block by simply scaling those values in the data structure.

We do not have to recreate quantized arrays or zigzag vectors. When implemented this way, the operation is made very fast, avoiding unnecessary multiplications where FQ [u, v] is zero.

Scalar addition

Now consider the scalar addition operation. If the value of the pixel in the original image is f [i, j], the value of the pixel corresponding to the output image h [i, j] is given as follows.

h [i, j] = f [i, j] + (7)

Adding a constant to each pixel changes the average (ie, DC component) value. DCT is stored in section [0.0]. Only this coefficient will be affected. Equations (1) through (4), equations (7), and the properties of the DCT can easily prove this. The result of such a calculation is

If the QTs of two pictures are proportional (as in equation 3), this equation will have a particularly simple form and is represented by the following equation.

Again, we know that scalar addition operations can be performed directly on quantized coefficients. More importantly, in the common case where = 1 (i.e., the picture quality of the output picture is the same as the picture quality of the input picture), this operation involves a calculation much more than the corresponding operation on the released pictures. Only the (0,0) coefficient of the quantized matrix is affected.

Pixel addition

13 is a diagram illustrating initialization of a combination arrangement.

The pixel addition operation is described by the following equation.

h [i, j] = f [i, j] + g [i, j] (9)

As with scalar multiplication, the operation you want to perform is linear. Since the JPEG compression algorithm is also linear, the quantized coefficients of the output image HQ [u, v] are summed and scaled with copies of FQ [u, v] and GQ [u, v]. In particular, it can be shown using equations (1) to (4) and equation (8).

In other words,

Once again, referring to the QT of the pictures, the operation can be performed directly on the quantized coefficients. Also, if the QT for all pictures is proportional (with proportional constants F and G), the equation is simplified as follows.

Pixel multiplication

Finally, the multiplication operation of the pixel is represented by the following equation.

Is a scalar value.

Although mathematically redundant, scalars are convenient for scaling pixel values when they are multiplied. For example, image g uses this formula when it contains pixel values in the range [0..255]. And wants to translate them into the [0..1] range. as g is a mask. This operation is realized by setting to 1/256.

F (v1, v2), G (w1, w2) and H (u1, u2) are the quantized values of the compressed picture for f, g, h, respectively. Using equations (1), (2) and (11), the value of H (u1, u2) can be calculated as follows.

By paying attention to two facts we can calculate this effectively.

1. For typical compressed images,

G (w1, w2) and F (v1, v2) are zero ('0') for most values of (v1, v2) and (w1, w2).

In the function WQ (v1, v2, w1, w2, u1, u2), only about 4% of the terms of 256K elements are not zero (in other words, the matrix WQ is very sparse). .

When implementing this technique, we must consider to calculate only the terms that will contribute to the sum. We will take advantage of the first fact when implementing this technique on RLE blocks. Zeros are easily skipped. To take advantage of the second fact, we use the data structures described in the next section.

Because the algorithm works on RLE blocks, zigzag ordered indexes are used to refer to data elements. By convention, x, y, z represent zigzag indices of pairs (v1, v2), (w1, w2) and (u1, u2), respectively. With this substitution, equation (12a) can be written as (12c) as follows.

The sum of x and y phases is from 0 to 63.

In order to efficiently calculate equation (13), the following data structure is introduced.

The combination element is a pair of numbers z and W. z is an integer and W is a floating point value. The combination-on list is a list of combination terms. The combination array is a 64 * 64 array of combination lists. The C code in Note 4 initializes the collation array comb [x, y]. The array contains an empty list when the code is entered. The function ZigZag (u1, u2) returns the zigzag index associated with the (u1, u2) element.

The function AddCombElt (z, W, comb [x, y]) inserts a combination element (z, W) from the combination list stored in the global combination array comb [x, y] and returns the modified combination list. The storage location is not important.) Assume that the array W [8] [8] [8] is initialized with the values of the W function in equation 12c.

Using the initialized combinatorial array, the C code shown in Figure 5 effectively implements Equation 13 on two RLE blocks f and g. This algorithm is called a convolution algorithm. Note that comb [] is a constant in the code. It is calculated only once for a given QT. It can be applied to an unlimited number of images. The code works as follows. Assume that the array hzz representing the zigzag vector is all zero. The zigzag indices x and y are calculated for each pair of RLE values in the two input images f and g. Then calculate the product of their data values stored in tmp. Use the z value of each combination term in the list of combinations stored in comb [x, y] to determine the term in the output array hzz that will be affected and to compute the product of W * tmp for each term.

In this way, only multiplication of nonzero products calculates in hzz. When this algorithm is used in a program, the last step is to run-length code zeros, perform integer rounding, and produce an RLE block of the result (of course, the use of integer arithmetic can increase performance). For the sake of clarity, we have chosen to describe a floating point implementation.)

Summary of operations

We have shown how pixel addition, pixel multiplication, scalar addition and scalar multiplication can be implemented in quantized arrays. As mentioned above, these variations can be computed directly on the RLE blocks. Table 1 summarizes the mapping to the operations on the RLE blocks of the image operation. In the table, the symbols F, H (x) are defined as follows.

The function Convolve (F, G,, qF, qG, qH) is defined in equation 13 and implemented in references 4 and 5.

Applications

Typically video data is transmitted in the order of compressed pictures.

Entropy coded data cannot be manipulated directly, but it shows how some operations are performed on RLE blocks. With respect to Fig. 12, if the picture is encoded in entropy, the operation is performed on the RLE blocks. And entropy encodes the result. It can be a shortcut for most of the image compression and decompression. This is due to a faster algorithm.

The basic operations in the table can be combined to form more powerful operations such as Dissolve (simultaneous fade out and fade in two orders of images) and subtitles. Implementation of these operations typically involves calculating an output picture that is an algebraic combination of one or more input pictures. Porter and Duff discussed many examples of such operations. One way to perform the combination on a pair of RLE blocks is to use a zigzag vector as an intermediate representation to calculate the expression. For example, to multiply two RLE blocks, and add the third one, we will call the Convolve function of FIG. 14 on the first two RLE blocks. The third RLE block is added to the zigzag vector, and execution length coding and entropy coding are performed. 15 diagrammatically illustrates our strategy.

Dissolve operation

In order to simplify the presentation, the entropy encoding and decoding steps are omitted.

At time t (typically 0.25 seconds), we want to dissolve the order of pictures S1 [t] in order of S2 [t]. In other words, it should indicate S1 [0] at t = 0. At t = t we have to denote S2 [t]. And want to display a linear combination of pictures in between.

D [t] = (t) S1 [t] + {1- (t)} S2 [t] (14)

(t) is a linear function of 1 at t = 0 and t = t.

In Table 1, this operation can be mapped to the corresponding operation on the RLE blocks as follows. From the table, scalar multiplications can be performed directly on RLE blocks if the coefficients in the first half of the equation are changed to S1, D (x) and similar substitutions are performed for the multiplication by {1-}. From the table also know that we can add the coefficients together directly to get the desired result. Since the QTs of these two new RLE blocks are the same, qD (x). Thus, the equation in equation 13 can be implemented as follows.

We can effectively implement this equation by noting that the RLE format of the data will skip zero terms. The C code for implementing this operation on the RLE block is shown in FIG. The function Zero makes the array passed to it zero. The function RunLengthEncode performs run-length encoding of h. The arrays gamma1 and gamma2 have precalculated values defined as follows.

These values can be calculated once for each pixel or for the order of pictures having the same QT. Dissolve, on the other hand, is called for each RLE block in a picture. To test the performance of this implementation, we created a program to run and compare both brute-force and RLE algorithms of images in main memory.

The algorithms were run on 25 separate pairs of pictures on Sparcstation 1+ with 28 Mbyte memory. The test images were compressed at approximately 1 bit per pixel. Table 2 summarizes the results. As can be seen from the table, the speedup was over 100 to 1 on brute-force algorithms.

Subtitle operation

The second example operation overlays a subtitle with a compressed picture f as a subtitle. Although workstations can support this operation in many ways (such as displaying subtitle text on a separate window), you choose this operation for two reasons. First, it's a common operation that most people know. And secondly, it acts as a specific example of a common operation of image masking that is used when one wants to combine part of an image with another.

Subtitles are marked with an S in a compressed image of white letters on a black background, with one pixel values of 127 and -128 for white and black, respectively. The output image can be made by adding together the image obtained by multiplying the mask to be black by f and the area on f where the text is to be displayed and f. The subtitling output picture, h, is given by

Using Table 1, the corresponding operation on the RLE block is

In Figure 17, the C code implements this operation. The code is divided into two stages. The SubtitleInit function is called once when QT is defined for a picture or sequential pictures. Subtitle is then called for each RLE block in the picture.

Like the Dissolve operation, the Subtitle function uses a zigzag hzz vector to store intermediate results. Like the Dissolve operation, the performance of programs implementing brute-force and RLE algorithms on images stored in the main memory is compared. The test parameters are the same as the Dissolve operation. Table 3 summarizes the results and shows a nearly 50-to-1 speed improvement over the brute-force algorithm.

The field of application of the technology can be used for both on-demand services and multimedia streaming fields such as live broadcasting.

The biggest feature of the present invention is distinguished from the existing MPEG scheme in the following seven ways.

1. The disappearance of the buffering process

2. Eliminate overload of streaming server

3. Minimize bandwidth of video and audio signals <Minimize line burden>

4. Eliminate unnecessary download process

5. Eliminate the use of separate players.

6. Minimization of server specification burden <Price reduction>

7. Portability of browser <Java class, applet available>

The video compression method of the present invention SPEG is as follows.

SPEG also uses lossy compression to compress images, the main principle of which is as follows.

① Set the limit value of frame to compress and compress by each object. Instead of setting the number of frames, the limit value is set and the operation is performed internally.

At the same time, different operators delimit the area and compress multiple frame signals, increasing the number of frames that can occur per second.

Therefore, more detailed results in terms of video can be enjoyed according to the present invention, and streaming servers also do not need as high specifications as other compression technologies.

② Give index values for each image and sort them in order.

In this alignment part, duplicate operation values are removed and pushed out of the limit operation value.

Here, the number of frames is determined. In general, frame operations are performed by internal operators and compression is taken up by CUP.

More compact compression is possible, and resources from buffering are eliminated because no resources are drawn from outside.

When the client is connected to the streaming server of the present invention at the time of connection, the internal operators of the present invention reside in the user group of the client, that is, mutual synchronization is achieved. So the user's frequency of use seems to be no buffering.

It is also true that this part does not require a separate player.

It also enhanced portability on the web to make it easier for users, and made these operators available directly to applets or classes using Java.

So if you just synchronize to your favorite Internet browser in Java, it will be displayed immediately without any buffering or external operation.

③ After the image of the object is compressed, the overlapping part is removed leaving only one index value. Therefore, lighter compression is possible and each index value is recalculated one by one by the internal operator.

The mutual sinker portion of the video and audio signals is made here.

Compared to the existing mpeg infrastructure, the synchronization of the signal is formed right at this point where the signal system is formed.

Therefore, the task of unifying the magnitude of the video signal and the audio signal is a priority.

I was very much concerned about this part, and the research result succeeded in combining the internal operator with the video signal in the gsm mono compression method, and it was possible to control the internal filter itself to realize basic live broadcasting.

In this part, the streaming server's overload is largely eliminated, so the server specification does not need to be too high.

Instead, the stronger the CPU environment, the single or dual, the less the burden on internal operators.

This type of compression allows for more native compression without device dependence.

The present invention avoids the dependence of the operation by limiting CPU resources because the internal operator handles various filtering.

Instead of having a single operator, different operators are used for each section of the compression format, so multiple operators share the limitations of one operator.

So we could take advantage of the more stable M-JPEG.

There have been many trials and errors, but in the end, various combinations of audio and video signals can be used to overcome the limits of computational errors.

Therefore, the present invention can be further developed into the IMT-2000 infrastructure by eliminating more compact compression and unnecessary filtering.

Considering the integration of IMT-2000 technology with this compression technology, which eliminates the need for external equipment, the IMT-based compression technology can only install an internal CPU that is more frequent or efficient than the existing MPEG 4 object-specific compression capability. Can be combined with -2000.

The future of this compression technology could be much more, including more control of internal filtering and the creation of denser speech signals.

This compression technology can be applied to various multimedia streaming solutions and can be used as a more personal streaming technology. For example, by integrating this streaming technology into personal broadcasting stations, which have been limited to existing radio stations such as private broadcasting stations, it also enables streaming broadcasting.

In addition, various applications such as wireless communication, education, and telemedicine are possible, and users can contribute to the development of streaming technology by widening the users' choice of streaming technology that is limited to the existing MPEG.

The present invention is not limited to the above-described specific preferred embodiments, and various modifications can be made by any person having ordinary skill in the art without departing from the gist of the present invention claimed in the claims. Of course, such changes will fall within the scope of the claims.

The present invention made as described above has the advantage of eliminating the buffering process, eliminating the overhead of the streaming server, the advantage of minimizing the burden of the line by minimizing the bandwidth of the video signal and audio signal, unnecessary download process The advantages of eliminating the problem, eliminating the use of a separate player, minimizing the burden on the server specification, and reducing the price, and the advantage of porting it to a general-purpose browser using Java classes or applets. The present invention is expected to have a great effect when sending and receiving multimedia on the Internet.

Claims (9)

In configuring a method of compressing and streaming a video using a lossy compression method using SPEG on the Internet, Determine the limit value of the frame to be compressed and then compress it for each object, Step 2 to give the index values for each image and then sort them in order, After the image of the object is compressed, the overlapping part is removed in three steps, leaving only one index value. The fourth step of allowing the internal operators of the streaming server to reside in the user group of the client when the client connects to achieve mutual synchronization without a separate player; Streaming method using a video compression method using SPEG characterized in that the internal operators to go through a five-step to be displayed as an applet or a class in a universal browser on the web using Java. The method of claim 1, The first step of determining the limit value of the frame to be compressed and then compressing each individual object; Instead of determining the number of frames, it uses a video compression method using M-JPEG, which is characterized by the step of setting a limit value, performing internal calculations, delimiting different operators, and then compressing multiple frame signals. By streaming method. The method of claim 1, The second step of giving the index value for each of the respective images and then sorting in sequence; Removing duplicate operation values and pushing them out of the limit operation value; In the above step, the number of frames is determined, the frame operation is mainly performed by an internal operator, and the compression is performed in the CUP to remove the buffering. The streaming method using the video compression method using M-JPEG. The method of claim 1, After the image of the object is compressed, the overlapping part may be removed while leaving only one index value; After the compression step, each index value is recalculated one by one internal operator for each interval section of the compression format to unify the size of the video signal and the audio signal so that the mutual sinker part is made here to synchronize the bandwidth. Streaming method using the video compression method using M-JPEG, characterized in that the step of densifying the limits. The method of claim 4, wherein The method of unifying the magnitudes of the video signal and the audio signal includes controlling the internal filter itself by tying an internal operator with the video signal in a mono compression scheme of gsm; A basic operation performed by the video signal processing operator and the audio signal processing operator dedicated to signal recognition operator processing that accepts only signals; Residing at an internal memory address of a memory filtering operator for filtering the two signals after the operation; Streaming method using a video compression method using M-JPEG characterized in that to prevent the overload of the streaming server consisting of the step of processing the operation by bringing the expert operator from the cpu to the moment whenever necessary. 5. A streaming method according to any one of the preceding claims, characterized in that it is a compression technique Stream Picture Expert Group (SPEG). Streaming method that enables complete frame continuity through external operators. Streaming method that eliminates buffering by handling streaming in such a way that the internal operator resides on the CPU and dramatically reduces the load on the CPU while the internal operator's processors reside in memory. Signal processing operator and pixel processing operator Streaming method in which processors are controlled separately by using voice processing operators.