KR20140102286A - Encoding and decoding using perceptual representations - Google Patents

Abstract

Encoding a video signal that includes pictures includes generating recognition expressions based on the pictures. The reference pictures are selected and motion pictures are generated based on the recognition expressions and the reference pictures. Motion vectors and pointers to the reference pictures are provided in the encoded video signal. The decoding includes receiving motion vectors based on the perceptual representations of the reference pictures and pointers to the reference pictures. The decoding of pictures in the encoded video signal may include selecting reference pictures using pointers, and determining predicted pictures based on the motion vectors and the selected reference pictures. The decoding may include generating reconstructed pictures from the predicted pictures and the residual pictures.

Description

[0001] ENCODING AND DECODING USING PERCEPTUAL REPRESENTATIONS [0002]

Motion processing involves the use of motion vectors in both motion estimation and motion compensation. Motion estimation is a process of determining motion vectors. A motion vector generally describes the transformation of objects from one two-dimensional image to another two-dimensional image, from adjacent frames or pictures in a video sequence. Motion compensation is the process of applying determined motion vectors to objects in one picture to synthesize the transformation of the described objects into subsequent pictures in the video sequence. The combination of motion estimation and motion compensation is a key part of video compression and is often highly demanded in terms of processing costs.

In motion processing, motion vectors are determined by how they can be classified as direct or indirect. Indeed, direct methods that rely on pyramid and block-based searching are commonly used in video encoders. Direct methods often require an increase in processing power and processing costs to increase the accuracy and / or accuracy of the motion vectors determined by these methods.

An indirect method for determining a motion vector is to use statistical functions that are applied over a local or global area within a picture to generate matches between the estimated motion generated in the pictures and the generated motion vectors . It is commonly used when fidelity metrics attempt to identify and eliminate false matches that do not correspond to real motion. However, fidelity metrics often result in inefficient motion vector outliers due to opportunistic best match, which is an error, and the demand for more bits for coding. This limitation tends to reduce video compression quality and efficiency.

Also, existing evaluation methods tend to favor high contrast areas in a picture when relying on a fidelity metric. This often results in poor motion estimates for regions of low texture and generally results in significantly inaccurate motion in these low textures. Fidelity metrics are also often used to determine the contrast, brightness, blur, additional noise, artifacts, and fades, dissolves, and other differences that may occur during compression, Fail to distinguish the motion. These other constraints also tend to reduce video compression quality and efficiency.

The weaknesses of the fidelity metrics in any of these environments can often be mitigated by increasing the motion processing capability, which increases the processing cost. Nevertheless, in environments where fidelity metrics are less effective, motion processing using existing evaluation methods often requires a trade-off between achieving more accurate / precise motion vectors in video compression and lowering processing costs.

In accordance with embodiments of the present invention, there are systems, methods, and computer readable media (CRM) that provide for encoding and decoding utilizing perceptual representations when determining or utilizing motion vectors. The use of a cognitive representation produces a motion vector with improved accuracy and / or accuracy. Recognition representations may be utilized to increase the accuracy and / or accuracy of the motion vector for the area of the low texture in the picture and / or for the picture in the transition sequence. The accuracy and precision of the motion vector increases especially for video sequences that include changes to contrast, brightness, blur, additional noise, artifacts, and other differences that may occur during fade, dissolve, and compression. Utilizing the recognition expressions in determining or utilizing motion vectors yields improved compression efficiency and lowers motion processing requirements and / or processing costs.

According to an embodiment, a system for encoding comprises an interface configured to receive a video signal comprising an original picture within a video sequence comprising a picture. The system includes a processor configured to generate a target recognition representation based on the received original picture, to select a reference picture from the plurality of reference pictures, and to determine motion vector information based on the target recognition representation and the selected reference picture. The determined motion vector information is determined based on the attributes of the reference picture and the target recognition expressions. The system encodes the motion vector information and encodes the pointer associated with the selected reference picture.

According to yet another embodiment, a method for encoding includes receiving a video sequence comprising an original picture within a video sequence comprising a picture; Generating a target recognition representation based on the received original picture; Selecting a reference picture from a plurality of reference pictures; Utilizing the processor to determine motion vector information based on the target recognition representation and the reference picture, the determined motion vector information being determined based on attributes of the reference picture and the target recognition representation; Encoding the determined motion vector information, and encoding a pointer associated with the reference picture.

The method for encoding may be implemented by computer readable instructions stored on non-transitory computer readable media. The instructions may be executed by the processor to perform the method.

According to yet another embodiment, a system for decoding includes an interface configured to receive motion vector information. The motion vector information may be based on a target recognition representation based on the original picture from the video sequence containing the picture, and a reference picture associated with the target recognition representation. The interface is further configured to receive a pointer associated with the reference picture and to receive a residual picture associated with the received motion vector information. The system may also utilize the received pointer to select a reference picture from a plurality of reference pictures, determine a predicted picture based on the received motion vector information and the selected reference picture, and reconstruct a picture based on the predicted picture and the residual picture And generate a decoded picture.

According to yet another embodiment, a method for decoding includes receiving motion vector information, the motion vector information including a target recognition representation based on an original picture from a video sequence comprising a picture, and a reference picture associated with the target recognition representation -; Receiving a pointer associated with a respective reference picture; Receiving a residual picture associated with received motion vector information; Selecting a reference picture from a plurality of reference pictures using each of the received pointers; Utilizing the processor to determine the predicted picture based on the received motion vector information and the respective selected reference picture; And generating a reconstructed picture based on the determined predicted picture and the received residual picture.

The method for decoding may be implemented by computer readable instructions stored on a non-transitory computer readable medium. The instructions may be executed by the processor to perform the method.

The features of the illustrations and disclosure are apparent to those skilled in the art from the following description with reference to the drawings.
1 is a block diagram illustrating a recognition encoding system that utilizes an example representation of a recognition representation.
FIG. 2 is a block diagram illustrating a recognition decoding system that utilizes a cognitive representation according to an example.
3 is a diagram of a photographic image showing the recognition representation and the original picture according to an example.
4 is a flow chart illustrating a calculation in a process for generating an awareness representation according to an example.
Figure 5 is a drawing of a photographic image illustrating a series of recognition expressions based on a companding factor different from the original picture, according to an example.
6 is a pictorial image illustrating the resilience of a perceptual representation of a change in contrast applied to an original picture, according to an example.
7 is a drawing of a photographic image illustrating the reconstruction of a recognized representation of a change in brightness applied to an original picture, according to an example;
FIG. 8 is a block diagram illustrating a motion estimation flow process in a system for encoding utilizing an example recognition perceptual representation. FIG.
9 is a block diagram illustrating a content distribution system according to an example.
Figure 10 is a flow chart illustrating a method for encoding that utilizes a recognizable representation according to an example.
11 is a flow chart illustrating a method for decoding utilizing an example representation of a recognition representation.
12 is a block diagram illustrating a computer system providing a platform for encoding and / or decoding systems, in accordance with an illustrative embodiment.

For simplicity and illustration purposes, the present invention is described by reference to embodiments and examples of the invention. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the examples. However, it is apparent that the present invention can be practiced without being limited to these specific details. In other instances, some methods and structures have not been described in detail in order not to unnecessarily obscure the substrate. Further, different embodiments are described below. The examples may be used or performed in different combinations together. As used herein, the term " comprises, "or " comprising" means including but not limited to. The term "based on" means at least partially based.

As shown in the following example, there is a recognition engine, encoding and decoding system, method, and machine-readable instructions stored in a computer-readable medium (CRM) for encoding and decoding motion vector information based on a cognitive representation do. Recognition representations include frames of the same frames and / or pictures as in the video sequence. The map in the recognized representation may include a calculated value associated with a unit in the same picture as the pixel. The computed values in the map of the recognition representation can be developed based on a model of human perception. Additional details regarding the manner in which recognition and presentation are generated and utilized in encoding and decoding are provided below.

Referring to FIG. 1, a recognition encoding system 100 is shown, which may be found in a device at the head end for distributing content in a compressed bitstream, such as, for example, a transport stream. According to an example, the recognition encoding system 100 receives a video sequence, such as a video sequence 101. The video sequence may be included in the video bitstream. The video sequence 101 may include a frame or picture that may be located or stored as an original picture in a memory associated with the recognition encoding system 100, such as the memory 102. [ The memory 102 may include one or more buffers or higher capacity storage. The picture from the video sequence 101 may be converted into a recognized representation by the recognition engine 104 and stored in the memory 102. The detailed steps and parameters that can generate a recognized representation are described in more detail below, e.g., with reference to FIG.

A target picture such as the target picture 103 may be retrieved from the memory 102 for compression and encoding. The target picture 103 may be an original picture from the video sequence 101. [ Further, a reference picture 106 for determining a motion vector may be retrieved from the memory 102. [ The target picture 103 and the reference picture 106 are signaled to the motion compensator 116 to generate the predicted picture 110 and the motion vector 113. The motion vector 113 may be generated from the recognized representation of the target picture 103 and the reference picture 106, as described below. The recognition representation is shown as a target recognition representation (105) and a reference recognition representation (108).

The pointer 114 may be associated with a reference picture 106. Pointer 114 may identify an attribute associated with reference picture 106 or a reference picture. Pointer 114 may be an identity, an association, an attribute, a location such as a memory address, and so on. The pointer 114 may be encoded and transmitted from the perceptual encoding system 100 for a downstream decoding process based on or associated with the reference picture 106. [

According to the example, the target picture 103 is retrieved from the memory 102 and signaled to the motion compensator 116. A target recognition representation 105 that may be generated by the recognition engine 104 from the target picture 103 is also retrieved from the memory 102 and signaled to the motion compensator 116. The reference picture 106 is selected by the selector 117 from the memory 102 and signaled to the motion compensator 116. A reference recognition representation 108 that may be generated by the recognition engine 104 from the reference picture 106 is retrieved from the memory 102 and signaled to the motion compensator 116. [ The motion compensator 116 may include a motion estimator 109 and a predicted picture generator 115. The motion estimator 109 receives the target recognition representation 105 and the reference recognition representation 108 and utilizes both to determine the motion vector 113. The motion vector 113 may be encoded and transmitted in a compressed video bitstream separately from the pointer 114 or along with the pointer 114. The motion vector 113 may be determined by scanning and identifying the block in the reference recognition representation 108 similar to the block in the target recognition representation 105 and generating a pointer to a similar block.

The predicted picture generator 115 generates the motion vector 113 determined by the motion estimator 109 and the predicted picture 110 using the reference picture 106. [ The subtractor 111 may receive and process the predicted picture 110 along with the target picture 103 to generate the residual picture 112. [ The residual picture 112 is reduced for downstream encoding and transmission to the decoding system. The residual picture 112 may exclude a motion estimation area of the target picture 103, such as an area associated with the motion vector 113. [ The residual picture 112 is an encoded picture that is sent from the recognition encoding system 100 for a downstream decoding process based on or related to the target picture 103. [

Referring to FIG. 2, a recognition decoding system 200 is shown, such as a set-top box, transcoder, handset, personal computer, or other client device for receiving content in a compressed bitstream, such as a transport stream Can be found in the device. According to an example, recognition decoding system 200 receives a residual picture 112, a motion vector 113, and a pointer 114. Any of these may be located or stored in a memory associated with the recognition decoding system 200, such as memory 201. [ The recognition decoding system 200 may utilize the pointer 114 to select a reference picture, such as the reference picture 202, The reference picture 202 corresponds to the reference picture 106 or to the reference picture 106. The relationship between the reference picture 202 and the reference picture 106 can be determined or identified through the pointer 114. [

According to an example, a motion compensator within a recognition decoding system 200, such as motion compensator 205, may receive both a reference picture 202 and a motion vector 113. [ The motion compensator 205 may generate a predicted picture such as the predicted picture 206. [ The predicted picture 206 may be generated based on the reference picture 202 and the motion vector 113. The predicted picture 206 may be signaled to an adder such as adder 207. [ The adder 207 may generate a reconstructed picture such as a reconstructed picture 208. [ The reconstructed picture 208 may be generated based on both the predicted picture 206 and the residual picture 112. [

The recognition expression of the original picture rather than the original picture itself becomes the basis for determining the motion vector. Referring to FIG. 3, the original picture 300 and corresponding recognition representation 301 are provided. Recognition representation (301) mimics the adaptive contrast constancy of human vision. Areas 302 and 303 of the recognition representation 301 illustrate improved imaging of the low contrast regions 302 and 303 associated with a low-level texture appearing in the original picture 300. [ Enhanced imaging that appears in areas 302 and 303 of the recognized representation 301 can improve block-based motion matching in motion estimation of these areas.

Region 304 of the recognition representation 301 shows the area of the original picture 300 that may be associated with the "Mach bands" phenomenon. The MachBand is a recognition phenomenon named after the physicist Ernst Mach and is associated with a bright or dark stripe recognized by the human eye as appearing next to the boundary between the two regions of the image with different brightnesses. The Mach-band effect is due to spatial high-boost filtering performed by the human visual system on the luminance channel of the image captured by the retina. This filtering is mainly performed in the retina itself, by lateral inhibition between the nerves. Mach-band phenomenon, and similar texture masking are performed through intra-retinal filtering that can occur near high contrast edges and features. Area 304 of recognition representation 301 illustrates how a gradient, such as a mathematical phenomenon and similar texture masking, is captured through the recognition representation. These gradations may not be available in other ways in the original picture for block-based motion vector matching.

The region 305 of the recognized representation 301 shows the high contrast feature of the original picture 300 shown as being preserved in the region 305 of the recognized representation 301.

The process for generating a recognized representation from an original picture is now described. Referring to FIG. 4, an example of generating a recognized representation from an original picture is shown in flowchart 400. The original picture has a Y value assigned to each pixel. For example, Y _{i , j} is the luma value of the pixel at the coordinates i, j of the image having the M x N size.

The Y pixel value quoted in the flowchart 400 is associated with the original picture. These Y values are converted into eY values in a spatial detail map. The spatial detail map is a weighting map that forms a picture processed from the original picture. The spatial detail map may be generated by the recognition encoding system 100 using a model of a human visual system that considers the natural image statistics and the response function of the cells in the retina. The weighted map may be a pixel map of the original picture based on the model of the human visual system. The weighted map may include a value or weight for each pixel that identifies the difficulty level for visual recognition and / or the difficulty level for compression. The difficulty level for compression may be a continuous scale that measures the number of bits needed to encode a pixel or region of an image. Similarly, the difficulty level for visual perception is a continuous scale that measures the number of bits needed to encode a pixel or region of an image as being associated with the ability of the viewer to track detail items within a pixel or region. The process for generating a weighted map is described in more detail in U.S. Patent Application No. 12 / 761,581 entitled " System for Reducing Noise in Video Processing "filed April 16, 2010, the entire contents of which are incorporated by reference. .

By way of example, the model associated with the human visual system, which may be used to generate the weighted maps, includes an Integrated Recognition Guide (IPeG) system. The IPeG system uses an ensemble-average statistic of a certain kind, such as scale-invariance of a natural image, to generate an IPeG transform that generates an "uncertainty signal" Lt; / RTI > The IPeG transformation models the behavior of certain cell classes within the human retina. The IPeG transform can be achieved by a 2d spatial convolution followed by a summation step. An improvement in the IPeG transform can be achieved by adding low spatial frequency correction, which can be approximated by decimation followed by interpolation or by other low pass spatial filtering. Pixel values provided in a computer file or provided from a scanning system may be provided to the transform to generate a spatial detail map. The IPeG system is described in U.S. Patent No. 6,014,468 entitled " Apparatus and Methods for Image and Signal Processing "filed on January 11,2000; U.S. Patent No. 6,360,021, entitled " Apparatus and Methods for Image and Signal Processing, " filed March 19, 2002; U. S. Patent No. 7,046, 857, entitled " Apparatus and Methods for Image and Signal Processing, " filed on May 16, 2006, and U.S. Patent No. 6,360,021, entitled "Apparatus and Methods for Image and Signal Processing ", PCT / US98 / 15767, the entire contents of which are incorporated by reference. The IPeG system provides information including a set of signals organizing visual detail items into perceptual importance, and a metric representing the viewer's ability to track a particular video detail item.

The spatial detail map shown in FIG. 4 includes a value eY. For example, eY _{i, j} is the value at i, j of IPeG conversion of Y values in the i, j from the original picture. Each value eY _{i , j} may include a value or weight for each pixel that identifies the difficulty level of visual perception and / or the difficulty level for compression. Each eY _{i , j} may be positive or negative.

As shown in Fig. 4, the sign of the spatial detail map, for example, sign (eY), and an absolute value of the spatial detail map, for example, | eY |, are generated from the spatial detail map. According to an example, the sign information may be generated as follows:

According to another example, the absolute value of the spatial detail map is calculated as follows: | eY _{i, j} | is the minus value of eY _{i , j} .

The compressed absolute value of the spatial detail map, for example, pY, is generated from the absolute value | eY | of the spatial detail map. According to an example, the absent cut-off value information can be calculated as follows:

및

이고, 여기서, CF(압신 인자)는 사용자 또는 시스템에 의해 제공되는 상수이고, λ_Y는 |eY_{i ,j}|의 전체 평균 절댓값이다. "압신"은 "압축" 및 "확장"으로부터 형성된 혼성 단어이다. 압신은 때때로 디지털화라고 명명되는 양자화가 통상적으로 뒤따르는 값들의 세트가 값들의 또다른 세트에 비선형적으로 맵핑되는 신호 프로세싱 동작을 기술한다. 값들의 제2 세트에 균일한 양자화가 이루어질 때, 그 결과는 값들의 오리지널 세트의 불균일한 양자화와 등가이다. 통상적으로, 압신 동작은 더 작은 오리지널 값의 더 미세한(더 정확한) 양자화 및 더 큰 오리지널 값의 더 거친(덜 정확한) 양자화를 초래한다. 실험을 통해, 압신은, 특히, IPeG 변환과 함께 사용될 때, 비디오 프로세싱 및 분석에서 사용하기 위한 인식 맵핑 함수를 발생함에 있어서 유용한 프로세스인 것으로 발견되었다. pY_{i ,j}는 eY_{i ,j} 값의 비선형 맵핑이며, 새로운 값의 세트 pY_{i ,j}는 제한된 동적 범위를 가진다. 위에 보여진 것이 아닌 다른 수학적 표현이 eY_{i ,j}와 pY_{i ,j} 사이의 유사한 비선형 맵핑을 산출하기 위해 사용될 수 있다. 일부 경우들에서, 값 pY_{i ,j}을 더 양자화하는 것이 유용할 수 있다. 계산에서 사용되는 비트수를 유지하거나 감소시키는 것이 이러한 경우일 수 있다.

And

, Where CF (constants) are constants provided by the user or system, and [lambda] _Y is the total average value of | eY _{i , j} |. "Confession" is a hybrid word formed from "Compression" and "Extension". Compression describes a signal processing operation in which the quantization, sometimes referred to as digitization, is typically non-linearly mapped to another set of values. When uniform quantization is performed on the second set of values, the result is equivalent to non-uniform quantization of the original set of values. Typically, the act of pushing results in a finer (more accurate) quantization of the smaller original value and a coarser (less accurate) quantization of the larger original value. Through experimentation, compression has been found to be a useful process in generating awareness mapping functions for use in video processing and analysis, especially when used with IPeG transformations. pY _{i , j} is a nonlinear mapping of eY _{i , j} values, and the new set of values pY _{i , j} has a limited dynamic range. Other mathematical expressions not shown above can be used to yield similar nonlinear mappings between eY _{i , j} and pY _{i , j} . In some cases, it may be useful to further quantize the value pY _{i , j} . It may be in this case to maintain or reduce the number of bits used in the calculation.

Expression recognition may be created by combining and companding the absolute value of the spatial detail map the code space of the detailed map, as _{follows: pY i, j × sign (} eY i, j). _{pY i, j × sign (eY} i, j) The result of eY _i, is that they preferably small absolute value of the _j eY _i, a further part of a larger dynamic range than the large absolute value of _j, but the retention eY _{i, j} Lt; RTI ID = 0.0 > of < / RTI >

Referring to FIG. 5, a flow diagram 500 is shown showing different recognition expressions generated from the original pictures by various different confidence factors. Referring to FIG. 6, a flowchart 600 is shown that includes a recognition representation generated based on the original picture and the same original picture at a lower contrast of 10% of the contrast in the original picture. Recognition expressions for both show comparatively similar reconstructions of the recognition expressions for changes in contrast. Referring to FIG. 7, there is shown a flowchart 700 that includes showing the recognized representation generated based on the original picture and the same original picture at higher brightness, which is 200% of the brightness in the original picture. The perceptual representation for both shows a relatively similar reconstruction of the perceptual representation of the change in brightness.

Referring to FIG. 8, there is shown a flow diagram 800 illustrating a motion estimation flow process executed by a motion estimator, such as motion estimator 109, in a system for encoding using a perceptual representation. In the flowchart 800, the video sequence 801 includes a picture that is signaled to the recognition engine 802 and the first pass motion estimation ASIC 804 in the motion estimator. The recognition engine 802 generates a guide motion vector 803. The guide motion vector 803 is signaled to the first pass motion estimation ASIC 804 where the guide motion vector 803 may be utilized in the pre-analysis process to generate a motion vector "seed" or & , The motion vector "seed" or "hint" may be utilized by the second pass motion estimation ASIC 805 to generate a motion vector such as the motion vector 113.

Referring to FIG. 9, the perceptual representation may be utilized in motion estimation and / or determination and / or utilization of motion vectors according to various video encoding formats such as MPEG-2, MPEG-4 AVC, and the like. In FIG. 9, an example of a content distribution system 900, including an encoding device 910 and a decoding device 940, according to an example, is shown. Encoding device 910 represents any encoding system that may be utilized in the compression or transcoding of a video sequence as discussed above with respect to FIGS. The decoding device 940 represents any of the set top boxes or other receiving devices, such as those discussed above with respect to Figs. Encoding device 910, according to an example, may send to decoder device 940 a compressed bitstream 905 that includes motion vectors and other information associated with the encoding utilizing the perceptual representation.

9, the encoding apparatus 910 includes an interface 920 for the incoming signal 920, a controller 911, a counter 912, a frame memory 913, an encoding unit 914, a transmitter buffer 915, 930), and an interface 935 to an outgoing compressed bitstream 905. The decoding apparatus 940 includes a receiver buffer 950, a decoding unit 951, a frame memory 952, and a controller 953. Encoding device 910 and decoding device 940 are connected to each other via a transmission path to compressed bitstream 905. The controller 911 of the encoding device 910 may control the amount of data to be transmitted based on the capacity of the receiver buffer 950 and may include other parameters such as the amount of data per unit time. The controller 911 can control the encoding unit 914 to prevent the occurrence of a failure of the decoding operation of the received signal of the decoding device 940. [ The controller 911 may include, for example, a microcomputer having a processor, a random access memory, and a read only memory.

For example, an incoming signal 920 supplied by a content provider may include a frame or picture in a video sequence such as video sequence 101. [ The frame memory 913 may have a first area, which is implemented via the encoding unit 914, used to store a picture to be processed through a recognition encoding system, such as the recognition encoding system 100. [ Recognition representations and motion vectors may be derived from the pictures in the video sequence 101, utilizing the controller 911. The second area in the frame memory 913 can be used to read the stored data and output it to the encoding unit 914. The controller 911 may output the area switching control signal 923 to the frame memory 913. [ The area switching control signal 923 may indicate whether the first area is used or the second area is used.

The controller 911 outputs the encoding control signal 924 to the encoding unit 914. [ The encoding control signal 924 causes the encoding unit 914 to begin the encoding operation. In response to the encoding control signal 924 from the controller 911, which includes control information associated with the picture or frame, the encoding unit 914 reads the picture for the high-efficiency perceptual representation encoding process to generate a motion vector, A residual picture is prepared and these are encoded into a compressed bitstream.

Encoding unit 914 may prepare a compressed bitstream 905 encoded in a packetized elementary stream (PES) that includes video packets and program information packets. Encoding unit 914 may use the program time stamp (PTS) and control information to map the compressed picture to a video packet.

The encoded information may be stored in a transmitter buffer 915. The counter 912 may include an information amount counter that is incremented to indicate the amount of data in the transmitter buffer 915. [ As the data is retrieved and removed from the buffer, the information amount counter 912 may be decremented to reflect the amount of data in the buffer. The occupied area information signal 926 is transmitted to the counter 912 to indicate whether data from the encoding unit 914 has been added or removed from the transmitter buffer 915 and thus the counter 912 is incremented or decremented . The controller 911 generates by the encoding unit 914 based on the occupied region information 926 passed to the encoding unit by the controller to prevent overflow or underflow to occur in the transmitter buffer 915 And controls the generation of the packet.

The information amount counter 912 is reset in response to the preset signal 928 generated and output by the controller 911. [ After the information amount counter 912 is reset, the counter counts the data output by the encoding unit 914 and obtains the amount of information generated. Then, the information amount counter 912 supplies the information amount signal 929 indicating the obtained information amount to the controller 911. [ The controller 911 controls the encoding unit 914 so that there is no overflow in the transmitter buffer 915.

The decoding apparatus 940 includes an interface 970 for receiving a compressed bit stream such as a compressed bit stream 905, a receiver buffer 950, a controller 953, a frame memory 952, a decoding unit 951, And an interface 975 for output. The recognition decoding system 200 shown in Fig. 2 may be implemented in the decoding unit 951. Fig. The receiver buffer 950 of the decoding device 940 may temporarily store encoded information including the motion vector, the residual picture, and the pointer received from the encoding device 910 via the compressed bitstream 905. The decoding device 940 counts the amount of received data and outputs the frame or picture number signal 963 applied to the controller 953. [ The controller 953 supervises the number of frames or pictures counted at predetermined intervals, for example, every time the decoding unit 951 completes the decoding operation.

The controller 953 may output a decoding start signal 964 to the decoding unit 951 when the frame number signal 963 indicates that the receiver buffer 950 is in a predetermined capacity or amount. When the frame number signal 963 indicates that the receiver buffer 950 is in a capacity less than the predetermined capacity, the controller 953 waits for the occurrence of a situation in which the number of counted frames or pictures becomes equal to a predetermined amount. When the frame number signal 963 indicates that the receiver buffer 950 is in a predetermined capacity, the controller 953 outputs a decoding start signal 964. The encoded frame, the caption information, and the frame difference map may be decoded in a forged (i.e., increasing or decreasing) order based on a presentation time stamp (PTS) in the header of the program information packet.

In response to the decoding start signal 964, the decoding unit 951 may decode the data 961, which is positive for one frame or picture received from the receiver buffer 950. [ The decoding unit 951 records the decoded video signal 962 in the frame memory 952. [ The frame memory 952 may have a first area in which the decoded video signal is recorded, and a second area used to read the decoded video data and output it to a monitor or the like.

According to an example, the encoding device 910 may be included with or otherwise associated with the head end, and the decoding device 940 may be included with or otherwise associated with the handset or set-top box. These may be used separately or together in a method for encoding and / or decoding associated with using a recognition representation based on an original picture in a video sequence. The various ways in which encoding device 910 and decoding device 940 may be implemented are described in further detail below with reference to FIGS. 10 and 11, which show a flow diagram of methods 1000 and 1100.

Recognition encoding system 100, in other embodiments, may not be included in the same unit that performs the initial encoding as shown in FIG. For example, the recognition encoding system 100 may be provided in a separate device that receives the encoded video signal and that cognitively encodes the video signal for downstream transmission to the decoder. Recognition encoding system 100 may also generate metadata that may be used by downstream processing elements, such as transcoders. The metadata may include a detail entry describing the motion vector estimated from the recognition representation that may be used by the transcoder to control the bit rate.

Way

The method 1000 is a method for encoding that utilizes a recognized representation. The method 1100 is a method for decoding utilizing a cognitive representation. It will be apparent to those skilled in the art that methods 1000 and 1100 illustrate generalized examples and that other steps may be added or existing steps may be eliminated or modified or rearranged without departing from the scope of methods 1000 and 1100. [ The methods 1000 and 1100 are repeatable to continuously encode and decode a picture in a video signal when a picture is received. The description of methods 1000 and 1100 is made specifically with reference to encoding device 910 and decoding device 940 shown in FIG. It should be understood, however, that methods 1000 and 1100 can be implemented in systems and / or devices that are different from encoding device 910 and decoding device 940, without departing from the scope of methods 1000 and 1100.

Referring to method 1000 of FIG. 10, at step 1001, encoding device 910 generates an original picture in a video sequence (e.g., video sequence 101 shown in FIG. 1) at interface 930 Lt; / RTI > For example, the received video signal 920 may be an uncompressed original picture in the video bitstream.

In step 1002, the encoding device 910 utilizes the encoding unit 914 and the controller 911 to generate a recognized representation based on the received original picture. It includes a target recognition representation and a recognition representation that can be used as a reference recognition representation.

In step 1003, the controller 911 selects one or more reference pictures from a plurality of reference pictures from an original picture stored or located in the frame memory 913. [

In step 1004, the encoding unit 914 and the controller 911 determine the motion vector information based on the target recognition representation and the reference picture. The determined motion vector information may be determined based on attributes of the reference picture and the target recognition representation, such as a low contrast feature in the reference picture and / or a matched band in the target recognition representation. The determined motion vector information may include the motion vector 113 shown in FIG.

In step 1005, the encoding unit 914 and the controller 911 encode the original picture using the motion vector information and the reference picture. The residual picture 112 shown in Fig. 1 is an example of an encoded original picture.

In addition, at step 1005, the encoding unit 914 and the controller 911 output the encoded original picture, motion vector information, and a pointer associated with the selected reference picture, such as the pointer 114 shown in FIG.

Referring to method 1100 of Figure 11, at step 1101, decoding device 940 utilizes interface 970 to receive motion vector information from compressed bitstream 905 in receiver buffer 950 do. The received motion vector information is based on a target recognition representation based on the original picture from the video sequence containing the picture and is also based on a reference picture associated with the target recognition representation.

At step 1102, the decoding device 940 utilizes the interface 970 to receive a pointer from the compressed bitstream 905 in the receiver buffer 950. A receive pointer, such as pointer 114, is associated with each reference picture.

At step 1103 decoding device 940 utilizes interface 970 to receive the encoded residual picture associated with the motion vector information received from compressed bitstream 905 in receiver buffer 950.

In step 1104, the controller 953 utilizes the respective receive pointers to select a reference picture from a plurality of reference pictures stored or located in the receiver buffer 950. [

In step 1105, the controller 953 and the decoding unit 951 determine the predicted picture based on the received motion vector information and the respective selected reference picture.

In step 1106, the controller 953 and the decoding unit 951 generate a reconstructed picture based on the determined predicted picture and the received residual picture.

Some or all of the above-described methods and operations may be provided as machine-readable instructions, such as a utility, a computer program, or the like, stored on a computer-readable storage medium that may be non-transient, such as a hardware storage device or other type of storage device . For example, they may exist as program (s) comprised of program instructions in source code, object code, executable code or other formats.

Examples of computer-readable storage media include conventional computer system RAM, ROM, EPROM, EEPROM, and magnetic or optical disks or tapes. Specific examples of tactical items include distribution of programs on a CD ROM. It is therefore to be understood that any electronic device capable of performing the above-described functions may perform the corresponding functions listed above.

12, a platform 1200 that may be used as a computing device in a system for encoding or decoding utilizing recognition representations such as recognition encoding system 100 and / or encoding device 910 is shown. The platform 1200 may also be implemented using an identification representation such as a set top box, handset, mobile phone or other mobile device, an upstream decoding device such as a transcoder, and a recognition decoding system 200 and / or decoding device 940 May be used for other devices and devices that may utilize a recognized representation and / or motion vector. It is understood that the example of the platform 1200 is a generalized example and that the platform 1200 includes additional components and that some of the described components may be removed and / or modified without departing from the scope of the platform 1200.

The platform 1200 includes a display 1202 such as a monitor and is capable of functioning as an interface of the encoding system or device such as interfaces 930 and 935 to the recognition encoding system 100 and the encoding device 910, Such as a local area network (LAN), a wireless 802.11x LAN, a 3G or a 4G mobile, which can perform the functions of an interface of a decoding system or device, such as interfaces 970 and 975 to a system 200 and a decoding device 940. [ Such as a simple input interface and / or network interface to a WAN or WiMax WAN. The platform 1200 further includes a processor 1201, e.g., one or more microprocessors, microcontrollers, digital signal processors (DSPs), combinations thereof, or other such devices known to those skilled in the art. For encoding or decoding, such as recognition encoding system 100, encoding device 910, recognition decoding system 200, and decoding device 940, but not the functions to be performed by display 1202 and interface 1203, The specific operations / functions described herein as being performed by the system described herein may be performed by processor 1201 of the platform by execution of software instructions and routines stored in a computer-readable medium (CRM) 1204 associated with the processor . However, those skilled in the art will appreciate that the operations / functions of processor 1201 may be implemented in hardware, for example, in a programmable (e.g., programmable) memory such as integrated circuits (ICs), application specific integrated circuits (ASICs), PLDs, PLAs, Logic devices, and the like. Based on the present disclosure, those skilled in the art will readily be able to produce and implement such software and / or hardware without failing the experiment. Each of these components may be operatively coupled to bus 1208. [ For example, bus 1208 may be EISA, PCI, USB, FireWire, NuBus, or PDS.

CRM 1204 may be any suitable medium that participates in providing instructions to processor (s) 1201 for execution and may include memory 102 and 913 for encoding system 100 or device 910, Buffer 915 and various memories and buffers described herein, such as memory 201 and 952 and buffer 950 for decoding system 200 or apparatus 940. [ For example, the CRM 1204 may be a non-volatile medium, such as an optical or magnetic disk; Volatile media such as memory; And transmission media such as coaxial cables, copper wires, and optical fibers. The transmission medium may also take the form of acoustic, optical, or radio frequency waves. CRM 1204 may also store other instructions or a set of instructions, including word processors, browsers, emails, instant messaging, media players, and telephony codes.

CRM 1204 may also include an operating system 1205, such as a MAC OS, MS WINDOWS, UNIX, or LINUX; An application 1206, such as a media player, such as a network application, a word processor, a spreadsheet application, a browser, an email, an instant messaging, a game or a mobile application (e.g. And a data structure management application 1207, for example. The operating system 1205 may be multi-user, multiprocessing, multitasking, multithreading, real-time, and the like. Operating system 1205 may also include an input device, such as a keyboard or a keypad, to recognize the input from interface 1203; Sending an output to display 1202 and continuing to track the file or directory on CRM 1204; Controlling peripheral devices such as disk drives, printers, and image capture devices; And to manage traffic on bus 1208. [0035] The application 1206 may include various components for establishing and maintaining network connections, such as code or instructions for implementing communication protocols including TCP / IP, HTTP, Ethernet, USB and FireWire.

As described above, a data structure management application, such as data structure management application 1207, provides various code components for building / updating a computer readable system (CRS) architecture for non-volatile memory. In certain instances, some or all of the processes performed by the data structure management application 1207 may be integrated into the operating system 1205. [ In certain instances, the process may be implemented in digital electronic circuitry, at least in part, in computer hardware, firmware, code, instruction set, or any combination thereof.

Although specifically described throughout the present disclosure, representative examples have utility across a wide range of applications, and the above discussion should not be construed as a limitation and should not be construed as a limitation. The terms, descriptions and drawings used herein are set forth by way of example only, and are not intended as limitations. Those skilled in the art will recognize that many modifications are possible within the spirit and scope of the invention. Although the present invention has been described with reference to exemplary embodiments, those skilled in the art will be able to make various modifications to the described examples without departing from the scope of the examples and equivalents thereof as set forth in the claims that follow.

Claims (20)

A system for encoding,
An interface configured to receive a video signal including original pictures within a video sequence comprising pictures; And
Processor
The processor comprising:
Generate target perceptual representations based on the received original pictures,
Selecting reference pictures from a plurality of reference pictures,
Determine motion vector information based on the target recognition expressions and the reference pictures, the determined motion vector information being determined based on the attributes of the target recognition expressions and the reference pictures,
Encode the determined motion vector information,
And to encode pointers associated with the reference pictures. The method according to claim 1,
Wherein the processor is configured to generate a plurality of reference recognition expressions from the reference pictures and to determine the motion vector information using the plurality of reference recognition expressions. The method according to claim 1,
The processor comprising:
Generates spatial detail maps based on the respective original pictures,
Determines sign information based on each of the generated spatial detail maps,
Determining maximum value information based on the generated spatial detail maps,
Processing the determined code information and the determined cut-off value information to form respective generated target recognition expressions
And generate the target aware representations. The method of claim 3,
Wherein the generated spatial detail maps include values associated with pixels in the original pictures. The method of claim 3,
Wherein the generated spatial detail maps include values determined using a model of human perceptibility of features in the respective original pictures. The method of claim 3,
The processor comprising:
Determine peak values maps based on the generated spatial detail maps;
By generating the absolute space detail maps based on the companding factor, the determined maximum value maps of each of them, and the generated spatial detail maps,
And to determine the bonus information. The method according to claim 1,
Wherein the original pictures in the video sequence are in a transition sequence of pictures in the video sequence. 8. The method of claim 7,
Wherein the transition sequence is characterized by at least one of a changing contrast attribute and a changing brightness attribute of the pictures in the plurality of pictures in the video sequence. As a method for encoding,
The method comprising: receiving a video signal including original pictures in a video sequence including pictures;
Generating target recognition expressions based on the received original pictures;
Selecting reference pictures from a plurality of reference pictures;
Utilizing a processor to determine motion vector information based on the target recognition expressions and the reference pictures, the determined motion vector information being determined based on properties of the target recognition expressions and the reference pictures;
Encoding the determined motion vector information; And
Encoding pointers associated with the reference pictures
/ RTI > A non-transitory computer readable medium (CRM) for storing computer-readable instructions for carrying out the method of claim 9. A system for decoding,
Receiving motion vector information, the motion vector information being based on target pictures based on original pictures from a video sequence including pictures, and reference pictures associated with the target pictures;
Receiving pointers associated with the reference pictures,
An interface configured to receive residual pictures associated with the received motion vector information; And
Using the received pointers to select reference pictures from a plurality of reference pictures,
Determine predicted pictures based on the received motion vector information and the selected reference pictures,
A processor configured to generate reconstructed pictures based on the predicted pictures and the residual pictures;
/ RTI > 12. The method of claim 11,
Wherein the reference pictures are reference recognition representations. 12. The method of claim 11,
Wherein the target recognition expression is generated for each of the generated spatial detail maps based on the respective original pictures,
Determining at least one of the determined code information determined based on the generated spatial detail maps and the determined maximum value information determined based on the generated spatial detailed maps,
≪ / RTI > 14. The method of claim 13,
Wherein the generated spatial detail maps include values associated with pixels in the original picture. 14. The method of claim 13,
Wherein the generated spatial detail maps include values determined using a model of human perception of features in the original picture. 14. The method of claim 13,
The determined cut-
A provisional value map based on the respective generated spatial detail maps, and
A plurality of absolute value maps, and a compressed absolute value space detailed maps generated based on the generated spatial detailed maps,
/ RTI > 12. The method of claim 11,
Wherein the original pictures in the video sequence are within a transition sequence of pictures in the video sequence. 18. The method of claim 17,
Wherein the transition sequence is characterized by at least one of a contrast change attribute and a brightness change attribute of pictures in the transition sequence. As a method for decoding,
Receiving motion vector information, the motion vector information being based on target-recognized representations based on original pictures from a video sequence comprising pictures, and reference pictures associated with the target-recognized expressions,
Receiving pointers associated with the respective reference pictures;
Receiving the residual pictures associated with the received motion vector information;
Selecting reference pictures from a plurality of reference pictures utilizing the respective received pointers;
Utilizing the processor to determine the predicted pictures based on the received motion vector information and the respective selected reference pictures; And
Generating reconstructed pictures based on the determined predicted pictures and the received residual pictures
/ RTI > A non-transitory computer readable medium (CRM) for storing computer readable instructions for carrying out the method of claim 19.

Images