KR20040011903A - A motion picture encoding method based on human recognition capability and scenario or image similarity - Google Patents

Abstract

PURPOSE: A method for encoding moving picture files considering the similarity of a script or video, and the recognition ability of a human being is provided to compress frame by frame, and use index vectors as recognition ability and similarity information, thereby preventing pictures from being broken. CONSTITUTION: Pictures of 0th to 10th frames are moving pictures processed by 30 frames per second and it is assumed that 6th to 10th frames are the same as 0th to 5th frames. If only 15 frames are encoded per second, the pictures of 0th, 2nd, 4th, 6th, 8th, and 10th frames are extracted. The 0th frame has the similarity with the 6th frame, the 2nd with the 8th frame, and the 4th frame with the 10th frame, thereby finally necessary picture information are the 0th, 2nd, and 10th frames.

Description

A motion picture encoding method based on human recognition capability and scenario or image similarity}

In general, a video is composed of several sequential still images, each of which is called a frame. That is, the video is composed of sequential frames. Many researches have been conducted on the video encoding method, and various researches are still underway. Such research may be conducted by individuals or individual companies at home and abroad, and research and development may be carried out by forming a national research organization in various forms. The most representative of them is the Moving Picture Experts Group (MPEG). Specialists and companies in the video field gathered around MPEG to set the standard for global video files. The standards announced by the organization include Mpeg, Mpeg-2, Mpeg-4, and Mpeg-7. Currently, Mpeg-4 or video files produced by modifying some of them are mainly used on the Internet, and Mpeg-2 is used for broadcasting or general media (DVD). In addition, various types of encoding techniques have been announced and used by various developers or individual companies. Although various forms exist, technically, most video file encoding methods are 1) intermediate still image compression, 2) image compression through interpolation between still images, and 3) background and objects or objects within the background. Are separated and compressed using motion information (motion estimation) of each object itself. In other words, most video file encoding methods are applied in various forms to achieve the goal of "preserving the information as much as possible and reducing the size of the video file to the minimum" of the three techniques listed. In the current popularization of the Internet, additional files or external elements (mostly streaming servers do this) add to the efficiency of transmission over the network. If the video file itself is provided with information for transmission using network information such as transmission speed, it is advantageous to use the video file in a network environment. Some encoding techniques actually provide such information.

The problem to be achieved in the present invention is largely 1) smaller than the existing video file while maintaining the information, 2) considering the human cognitive ability, 3) to propose a new video encoding method that can be flexibly applied to the network environment will be. In addition, it is to avoid using change information between frames or movement of objects between frames, which are mainly used in the Mpeg series, and to create a video file faithful to each frame itself. This is not so easy to extract information between frames, and when using information on changes between frames, the amount of information is increased in the case of a severely dynamic (high video change) video, so a single frame Rather than compressing the file, the file grows in size and may require more computation to encode and decode. Therefore, the large frame used in the present invention does not use motion estimation or interpolation information between frames, and encodes each frame as it is, so that even if there is a severe change in the video, there is no loss of video information at all. We will present a method of encoding. This is a method to be taken in Mjpeg 2000 in which Jpeg images are sequentially merged or Mjpeg2000 in which Jpeg2000 images are sequentially merged. In addition to this, the total similarity information between the frames (index vector) and the frame rate (frame rate per second) in consideration of the human cognitive ability are used.

(Figure 1) Process of extracting index vector

(Figure 2) Block diagram showing the encoding method of the proposed video

(Figure 3) Decoding sequence of the video produced by the method of (Figure 2)

(Figure 4) Real-time video transmission and reception

The present invention relates to a video encoding method. A file encoded by the method of the present invention can be largely divided into two parts, actual image information and an index vector. The actual image information is literally compressed with a Jpeg, Jpeg2000, or other method as it is. The image index information is an index of an image that is actually used in consideration of overlapping images, similar images, or human cognitive ability. The frame rate (frame rate) per second of the video proposed by the present invention does not exceed 18 frames at most, and in the general case, the frame rate is about 15 frames. If human cognitive ability actually exceeds 18 frames per second, it is almost unrecognizable and it is hard to notice a real difference, so it does not increase the number of frames per second anymore. For example, if you want to make a video of frames 1-30, and the original frame rate is 30, this video is 1 second video. Normal encoding requires 1 to 30 images. However, if only 15 frames are used in consideration of human cognition, 1, 3, 5,... This means that you can make a one second video with only 15 frames of frame 29. In fact, this difference is not easy to perceive by human perception. The image index vector holds this information. Depending on the image, there may be similar images that are adjacent in time or far apart in time. Scenes that recall the past in the film appear as duplicates of these images. In the present invention, such information is also used for encoding, and such information is described in an index vector. For example, if 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, and 10 images are processed at 30 frames per second, and frames 6 to 10 are the same as frames 0 to 5, Suppose In the proposed encoding method, if only 15 frames per second are encoded, images of 0, 2, 4, 6, 8, and 10 are extracted in consideration of cognitive ability, and the similarities of the extracted images are 0, 6, 2 and 8 Frames 4, 10 and 10 have the same information. Then, the last necessary image information is frames 0, 2, and 4, and only the remaining indexes need to be maintained. If each frame is 10 kbytes, the 110 kbyes video is compressed to 30 kbytes. Of course, there is additional index information, but the size is negligible compared to the size of the image itself. In the example, the adjacent video is taken as an example. In fact, even if the time is far apart, it does not matter at all. This is a different meaning from the interpolated video described in Mpeg. If similar information about a temporal neighbor is used in Mpeg, the present invention is virtually irrelevant to time, and compares similarity between frames in the entire video. The process of extracting the index vector is shown in FIG. 1. In the second process, the text in the front square is an indication that the image is similar. Although there are various methods for expressing the similarity between frames, the present invention uses a signal-to-noise ratio (SNR) between frames. In the case of a video, roughness similarity information can be extracted from a script, so using scenario information may be efficient. When comparing the similarity of images directly, it is easy to implement in hardware because the calculation is simple.

In Figure 2, the overall encoding method is simply shown in a block diagram. First, in order to measure the similarity between frames, an index vector is extracted by script information or a direct comparison between frames, and a frame required by the index vector is compressed as it is. Then, the video is finally expressed by the image information and the index information. In FIG. 1, image information is generated by compressing only frames 0, 2, and 4, and a video is expressed like index vector information. Figure 3 shows the process of decoding the encoded file. As shown in the figure, decoding proceeds in the reverse order of encoding.

The encoding method of the present invention can also be used for real-time image transmission using a camera. If not only the image but also the index information is exchanged between the receiver and the sender, the methods of FIG. 1 and FIG. 2 may be applied as it is. That is, each time a new video is transmitted, if an index is assigned and index information is sent together, the receiving side shows the received video and keeps it with the index. After that, if the image sent from the transmitting side is the image with the index, the transmitting side does not send the image, but only the index, the receiving side appears as if the normal information comes. In addition, for real-time transmission through the network, it is necessary to readjust the frame rate according to the communication environment, which can be implemented by adjusting the communication frame rate or changing the compression rate of each frame. Between the sender and the receiver, you can choose between high quality, high quality video or low quality video quality. 4 shows information transmission contents using the proposed encoding method in a real-time image transmission situation. In addition, the concept of the index can be applied to the video compression method of the Mpeg series that uses only the information of the adjacent video.

The video encoding scheme proposed by the present invention is to compress each frame one by one without using any motion estimation or interpolation between frames used in most high compression video encoding techniques. Motion estimation or interpolation between frames is hard to expect the effects of compression in the case of images with a large change between adjacent frames. Most video encoding techniques are indifferent to compression of the same frame, which makes a big difference in time. In the present invention, only similarity information between frames is used regardless of time difference. This makes it possible to create smaller size video files with relatively simple hardware and can be applied to real time applications such as video communication. Basically, each frame is compressed one by one, so it is possible to process some parts in high quality and some parts in low quality in time, and it is generally shown in video files using algorithms such as motion estimation such as mpeg-4. The phenomenon of a broken image on a severely dynamic screen does not appear fundamentally. Of course, the often reminiscent part of a movie is just an index, so the effect of compression is natural.

Claims (1)

Video file encoding method considering script or video similarity and human cognition ability