The application divides an application, the national applications of its female case application number is 00819925.6, the international application no of its female case application is PCT/F100/00686, and the applying date is on August 11st, 2000, and the name of innovation and creation is called " method and apparatus that is used for transmitting at telecommunication system frame of video ".
Embodiment
Fig. 1 can use a digital data transmission system of solution of the present invention with the simplified way explanation.Here be the part of a cellular radio system, this system comprises a base station 104, with user terminal 100 and 102 communications, user terminal 100 and 102 can be the terminal of fixed in position or the terminal that maybe can be of portable form in a delivery vehicle with dedicated radio link 108 and 110 for it.Transceiver in this base station is connected to an antenna element, by the radio link of its enforcement to user terminal.This base station also sends base station controller 106 connections that are connected the terminal of locating in addition in the network with one.This base station controller is controlled some connected base stations in a concentrated manner.A control unit in this base station controller is safeguarded and is called control, data service control, mobile management, statistics collection amount, and signaling.
Cellular radio system also can be connected to public telephone network, wherein transcoder belongs to different digital coding format conversion that the situation of this system will be used in the voice between this public telephone network and this cellular radio network to adapting each other, for example is transformed into cellular radio network form (for example 13Kbit/s) and vice versa from the 64Kbit/s of this fixed network.
It will be apparent to one skilled in the art that the present invention also may be used on the telecommunication system of fixing, for example international internet.
The purpose of a frame of video of coding is to reduce the desired figure place of this frame of transmission.Fig. 2 a-2c represents the prior art coding method of moving with plain mode, and it is based on using the mobile vector that early is stored in the reference frame frame piece of prediction (the term piece also uses) in the memory in text.This type of coding is called mutual coding, and it means the similarity between the frame that uses in a string frame under the compressed encoding situation.
In Fig. 2 a-2c example, reference frame 206 be divided into a plurality of or section and early be encoded.Fig. 2 b-2c does not represent a plurality of frame pieces.This reference frame can be before or after the frame that is encoded in this frame string.Fig. 2 a represents the present frame 204 that is encoded, and it also is divided into a plurality of, wherein only represents one 200 for the purpose of clear.The size of these pieces is N * M.These pieces generally are foursquare, and promptly N=M for example by their physical dimension, also can be a rectangle still.Be illustrated in present piece 200 among Fig. 2 c and reference block 214 on physical dimension and do not require similarly, this is because the target that in fact films also changes, for example rotation or distortion.The size and dimension of this piece is determined the situation that each considers following key element that is suitable for, and for example: a little piece size provides a better image, but has reduced the reliability of the mobile vector of determining, because this determines only to be undertaken by several pixels.If use less piece, then also must produce more mobile vector data, this is because then this frame must be divided into more piece.In addition, when the piece size is big, determine that the fast algorithm in the mobile vector is more effective.For example, N=M=16 in some present video encoding standard.
In order to estimate that this moves, a reference block seeking pairing as far as possible at the reference frame 206 of the piece 200 that is used for being encoded is 214 among Fig. 2 c.The adaptability of these reference blocks depends on choice criteria.The purpose of this choice criteria generally is to seek a piece, might in such a way the figure place that requires to transmit this piece that is encoded be reduced to minimum by this piece, and promptly picture quality will keep very good.This choice criteria will depend on application target and can change like this.
Search usually 206 neutralizations of the reference frame in a zone is carried out in certain zone of the point blank described of region of search 210 in by Fig. 2 b and 2c, and this zone is in the frame position of the piece 200 that is encoded in the corresponding present frame 204.By limiting this region of search, time and the desired calculating that search is used will be reduced less than whole reference frame.The size of determining this region of search is slow or fast by determining to move based on the frame of the frame string of early encoding for example to be applicable to application target; If move slowly, then the region of search can be limited less than mobile fast situation.Around the prediction mobile vector, generally carry out this search.
The position of frame piece is that (x y) represents with the coordinate in this piece upper left corner in the example of Fig. 2 a-2c.Coordinate x is illustrated in the pixel of horizontal direction or the position of piece, and coordinate y is illustrated in the pixel of vertical direction or the position of piece.The coordinate of the piece that this is encoded is labeled as 202.In Fig. 2 c, in region of search 210, looked for the best reference frame block 214 of possibility.(x, y) 208 is these positions to coordinate, may be the piece 200 that is encoded there, if it does not move and be identical with coordinate 202 thus.They are represented that like this its purpose is used for explanation.Piece 212 is described piece 200, if it does not move and they are represented that like this purpose is in order to illustrate.The frame piece that is encoded now mobile can be expressed as that (x+u, y+v) 216 to coordinate (x, y) 208 change from coordinate.Usually, ((u, v) mobile vector also can use the definition of mobile vector coefficient and mobile mould to represent to coordinate so the mobile vector 218 of the piece that is encoded can be expressed as coordinate for x, y) as above definition.
Fig. 3 represents prior art mobile coding method with a plain mode, and this coding method is based on the be encoded mobile vector coefficient of the frame piece that is encoded of adjacent block prediction of coding early of frame of use.Be encoded earlier and be used for predicting that the adjacent block of this frame piece 304 that is encoded of the mobile vector coefficient of this frame piece that is encoded is claimed predicting candidate at this.In Fig. 3, this predicting candidate is an adjacent block 300,302,306.Predicting candidate 300,302,306 are used for identification with some location of pixels 308,310,312 mark in Fig. 3 example.Predicting candidate 306 is also to use location of pixels 308 marks on the left side of this encoding block, and predicting candidate 300 is higher than the piece that is encoded also with location of pixels 310 marks, and predicting candidate 302 is at the upper right corner of this encoding block location of pixels 312 marks.As finding out that by Fig. 3 location of pixels 308,310,312nd resides in their piece to press close to very much encoding block 304 ground.Should be understood that the quantity of adjacent block relative with the position be encoded piece can with Fig. 3 in represented inequality.
In situation shown in Figure 3, three predicting candidate all are used for predicting the mobile vector coefficient of this piece 304 that is encoded.Desired mobile vector coefficient for example can be predicted as an intermediate value of the mobile vector coefficient that obtains from these predicting candidate.Also might only use a predicting candidate to predict the mobile vector coefficient of this piece 304, in the case, this coefficient is from the mobile vector coefficient of this prediction piece 306, and the mobile vector coefficient of this prediction piece 300 maybe mobile vector coefficient of this prediction piece 302 obtains.
Can also use the adjacent block of the Zao coding more farther, but should consider in the case to increase with distance usually in the difference of piece aspect moving than proximity distance.
Interframe with frame in coding typically in the encoded video frame connect a combination to obtain the good result of most probable.Intra-coded frame can transmit the accumulation with the possible error that prevents to be caused by this transmission channel from a frame to another frame on proper spacing, thereby causes the failure of frame decoding at last.
Can use the method shown in Fig. 2 a-2c and 3 in Code And Decode.Press the method that encoder uses, necessary frame data are transferred to decoder as a bit stream.
Should point out that as shown in FIG. 3, piece also can segmentation.Be preferably in segmentation in such certain situation, promptly the different piece of frame moves in a different manner there, and for example background keeps identical, and independent target people for example, and car or analog are moving.A frame piece can carry out segmentation by vertically cutting apart it by section, as in the piece 300, does not have the example of piecemeal flatly here, or carry out piecemeal at both direction as at piece 306.Rely on segmentation by using another suitable figure place that partly also may reduce transmission of coded frame.
When the mobile vector coefficient of the frame piece of determining to be encoded, use various known mobile estimation modes.Different patterns is suitable for determining for example representing moving of vertical and level or different moving rotated in expression.Such pattern comprises Move Mode like the plan, one of them pixel, and the moving of frame section or piece by a formulate, wherein Δ X and Δ are y-shaped becomes mobile vector
ΔX(x,y)=a
0+a
1x+a
2y
Δy(x,y)=b
0+b
1x+b
2y (1)
A wherein
0, a
1, a
2, b
0, b
1And b
2Be that mobile vector coefficient and x and y are vector components, wherein x is illustrated in the position of a pixel of horizontal direction or a piece, and y is illustrated in the position of a pixel of vertical direction or a piece, or translative mode, one of them pixel, moving by a formulate of frame section or piece, wherein Δ X and Δ are y-shaped becomes mobile vector
ΔX(x,y)=a
0
Δy(x,y)=b
0 (2)
A wherein
0And b
0Be the mobile vector coefficient, x and y are vector components.Employed Move Mode is stored in the memory of encoder.If some Move Mode is used for coding, then this encoder (for example moves up by moving, the pattern that moves up that indication is used in coding) preference pattern, and be transferred to this decoder with this mobile vector coefficient by the information of the pattern of this use.
Fig. 4 is the flow chart of expression video frame coding method step.In following specification, for example, frame piece means a piece by the YUV pattern that is formed by pixel, section or macro block.Begin to carry out this method at square 400.
In square 402, a frame of video is formed by continuous static frames.For forming this frame of video, static frames is preferably on such speed and transmits, and on this speed, no longer distinguishes the frame that is separated from each other like people's visible system.
In square 404, the coding playback of data of frame piece (for example color of pixel and brightness and from calculating the predication error frame that the mobile vector coefficient obtains) is stored in the memory.H.263 or mpeg encoded the frame playback of data is for example pressed application standard.The definition of mobile vector coefficient and predication error frame is as above described.One or more reference frames are formed by the frame playback of data that is stored in the memory.In addition, the mobile data of the adjacent block of coding early, promptly mobile vector coefficient and the Move Mode that may use are stored in the memory.Most preferably use adjacent block near this piece that is encoded, but also may use sometimes in this frame away from the piece of coding early.
In square 406, the mobile data of the piece that is encoded is determined by the adjacent block of early encoding, and preferably uses the mobile vector coefficient of one or more adjacent block.
In square 408, for example use a reference frame determine the to be encoded playback of data of piece.
This method can be strengthened by the mobile data that only uses this adjacent block to calculate the frame piece that is encoded, and this adjacent block has the same reference frame of the frame playback of data of this piece that is encoded that is used to encode.Usually exist similarity between the frame in this same number of frames string, so likely is to find corresponding to the piece of this piece that is encoded or very similar piece in a frame of early encoding, found a suitable reference block to be used for an adjacent block of coding early thus.For example, the region of search of reference block that will use the mobile vector coefficient of one or more adjacent block and will limit this piece that is encoded in this reference frame is to the environment of being determined by the mobile vector coefficient of this adjacent block.
The situation that has the reference frame identical for the adjacent block of wherein having only an interframe encode with the piece that is encoded, the mobile vector of this adjacent block is selected as a mobile vector candidate, and other candidate for example can be an intermediate value of being calculated by some adjacent block mobile vector.
Should point out that the piece of this piece that is used for encoding is (interframe) coding in the same manner, this piece that is encoded will be encoded with this mode, that is, if adjacent block is in-line coding, then it does not have reference frame.The predicting candidate that if there is no has reference frame, but in-line coding or detection of stored all reference frames in memory of prior art then can be used.
A reference block is the frame piece, like that promptly pushes with the video encoder of the frame playback of data of its coded frame piece and reduces the staff the coding mode use.Select as far as possible with the piece pairing that is encoded and by a frame piece of early encoding and being stored in the one or more reference frame search in the memory as this reference block, predication error frame keeps as far as possible little thus.
In square 410, select this coding method, use this method, the frame playback of data and the mobile data of the frame piece that this is encoded can the most advantageously be encoded, that is and, a predetermined code efficiency realizes with a predetermined picture quality.Should point out, although select the coding method of this piece that is encoded or by the above-mentioned coding method of the mobile data of adjacent block, perhaps any art methods, interframe encode (inter-coding) method for example, the piece that wherein is encoded is directly by the reference block coding that obtains from reference frame.When selecting coding method, its purpose can for example be to make the corresponding normal video image of picture quality of the least possible while of figure place, and the standard that the figure place pairing is used will be even figure place still less will be enough to obtain the picture quality by application target.
For example can be by Lagrange (Lagrangian) function estimation code efficiency, it determines that the most effective section and coding and it are
L(S
K)=D(S
K)+λR(S
K) (3)
Wherein
D (S
K) be the error that produces in expansion (despread) frame.
R (S
K) be requirement transmit this frame figure place and
λ is Lagrangian constant.
Lagrangian has been described the connection between the bandwidth that requires on quality that is transmitted frame and the transmission path; Usually, picture quality is high more, and bandwidth is big more.Selecting best compressed encoding often is trading off between figure place that requires and picture quality.
Arrow 412 is described a repeatability that connects one of this method till this frame is encoded.Till should pointing out that this method preferably repeats all frames in the frame string and is encoded.
Manner of execution finishes in square 414.
Fig. 5 explanation when the adjacent block of the same reference frame of the frame playback of data that only uses that having is used to encode and be encoded piece, prediction mobile vector and obtain the example of a reference block.Should point out, when encoding block, also use art methods usually.At last, this coding method is selected for provides the code efficiency with satisfied picture quality to meet the requirements of use occasion.
The frame page or leaf 500 that is encoded is divided into a plurality of, only represents that wherein several is for clarity sake.Piece 506 is one that is encoded.Frame 502,504,532 and 508 is the reference frames that are stored in the memory.The piece 510 of the adjacent block of coding early of lable number 1 and 514 reference frame are frames 502.Compare with frame 502, piece 506 that is encoded and reference block 510 and 514 do not move in frame 500.The reference frame of the adjacent block 512 of lable number 2 is frames 504 in the drawings, and the reference frame of the adjacent block 516 of lable number 4 is frames 508 in the drawings.Frame 504 and 508 is compared, and the piece in frame 500 is mobile.Frame 532 is not used when coding is encoded arbitrary of adjacent block of piece 506, and promptly it is not a reference frame of arbitrary adjacent block, so do not use this frame 532 to remove to obtain a reference frame of the piece 506 that is used to be encoded in this embodiment.
Then, adjacent block 510 and 514 is used as the predicting candidate of the mobile vector of the piece 506 that is encoded.Whether identically detect in order that observe the type of coding (interframe) of predicting candidate with one that in the present piece of coding, uses.If it is identical, then piece 526 is used as a reference block and the mean value or the intermediate value of mobile vector coefficient that the mobile vector coefficient obtains by the position of calculating by comparison block 510 and 518 obtain, obtain the mobile vector coefficient in view of the above and be used for adjacent block 510, with the position of piece 514 and 520, obtain the mobile vector coefficient in view of the above and be used for adjacent block 514.In addition, compare with reference block 526 by the piece 506 that will be encoded and determine a predication error frame.In the example of Fig. 5, the piece position in frame 500 and 502 does not change, so the mobile vector coefficient is not determined.
Then, detect the type of coding of adjacent block 512, if it is identical with one that uses, then in frame 504, seek a reference block in present piece 506, and be used as the mobile vector coefficient of piece 506 by the mobile vector coefficient that comparison block 512 and 522 obtains.Determine this predication error frame by this reference block 528 then.
Then, detect the type of coding of adjacent block 516.If it is identical with one that uses in present piece 506, then in frame 508, seeks a reference block, and be used as the mobile vector coefficient of piece 516 by the mobile vector coefficient that comparison block 516 and 524 obtains.Determine this predication error frame by this reference block 530 then.
The mobile vector coefficient of piece 506 also can be obtained by the intermediate value or the mean value of the mobile vector coefficient of computing block 510,512 and 514.
Equally, if use some Move Mode, then indicate employed Move Mode.
Piece 534 is in-line codings, so it does not have reference frame, thus when encoding block 506, do not use, unless if find a preferred coding method.
The number of reference frames of the quantity of adjacent block and position and storage can be different from the situation shown in Fig. 5.
Fig. 9 illustrates the prediction mobile vector and obtains second example of a reference block.In this embodiment, the piece that is encoded is only encoded by an adjacent block, as long as have only an adjacent block by interframe encode, has the reference frame identical with the piece that is encoded with this adjacent block.
The frame 900 that is encoded is divided into a plurality of, wherein for clarity sake only represents several.Piece 912 is pieces that are encoded.Frame 902,904 and 906 is the reference frames that are stored in the memory.Early the reference frame (being labeled as 1) of the adjacent block 914 of coding is a frame 902, and the reference frame of piece 910 (being labeled as 2) is a frame 904.Piece 908 is in-line codings, so it does not have reference frame.
Can the example in Fig. 9 will test the reference block that is fit to that obtain to be stored in all reference frames in the memory for the piece 912 that is encoded, and be frame 902,904 in this example, and 906.When whether test frame 902 had a suitable reference block, the mobile vector of piece 914 was used as this mobile vector predicting candidate, because piece 914 only is a piece of three adjacent block, its reference frame is a frame 902.When whether test frame 904 had a suitable reference block, the mobile vector of piece 910 was as the mobile vector predicting candidate, because piece 910 only is a piece of three adjacent block, its reference frame is a frame 904.906 pairs of arbitrary adjacent block of frame are not reference frames, so in the described in this example situation, mobile vector is to use a median prediction of piece 910,914 and 908, for example, in this case, the mobile vector of the piece 908 of in-line coding is set to zero.
Should point out, when prediction is encoded the mobile vector of piece, might not only use the mobile data and the reference frame data of adjacent block, but also use the segment information of macro block, for example the segment information that belongs to of this piece that is encoded.The quantity of the reference block of the quantity of adjacent block and position and storage can be with shown in Fig. 9 inequality.
Fig. 6 represents to implement a used encoder example of said method of encoded video frame.A design principle of video encoder is to make at the piece I that is encoded
n(x, y) 620 and predicting candidate (selected reference block) P
n(x, y) the predication error frame E between
n(x, y) 600 minimums.This predication error frame is defined as poor between piece 620 and this predicting candidate 612 of being encoded, and it obtains in adder 632, like this
E
n(x,y)=I
n(x,y)-P
n(x,y) (4)
Prediction piece 612 is by obtaining by the selected reference frame of use of the preferred embodiments of the present invention and the method for mobile vector.Mobile vector Δ x (x, y), Δ y (x, coefficient y) in mobile vector computing block 630, form as the mobile vector of prediction and the poor coefficient of acquisition with.By the preferred embodiments of the present invention, the mobile vector coefficient of prediction is to use the mobile vector that has with the adjacent block of the piece same reference frame that is encoded to obtain.Other method also is used to predict mobile vector, is a kind of art methods one of in them, and the piece that wherein is encoded is directly to encode by a reference frame.The group of the mobile vector of all pixels in a frame is called a mobile vector field, owing in a frame, have a large amount of pixels, in fact frame is divided into a plurality of of video coding and this mobile message and is sent to the receiver decoder of representing among Fig. 7 one by one.
Select at least one mobile vector or mobile vector encoding block 628, encode from the mobile vector of prediction by its mobile vector field that forms.The Move Mode well known in the prior art of for example wherein representing example in the above is used to coding.The mobile vector of coding or be sent to multiplexer 624 by the mobile vector field that it forms.
In piece 618, a frame is connect one section ground or the formation of a plurality of one by one frame piece by one section.This reference block or predicting candidate have form
Wherein one or more reference frame R
n k(x y) 622 is stored in memory 610 and neutralizes wherein that x and y are vector components, and x represents pixel on the horizontal direction, the position of frame section or piece, and y represents their positions in vertical direction, and
With
Form a pixel, the mobile vector of frame section or piece.
In piece 602, predication error frame E
n(x y) 600 is encoded, and for example by discrete cosine transform (DCT), it generally is expressed as 2 final dimension sequence transformations.Before transferring them to the receiver decoder shown in multiplexer 624 and Fig. 7, the coefficient that is converted to by this DCT is quantized and encodes.
Encoder also comprises the predication error frame E that wherein decodes
n(x, decoder block 606 y).Predication error frame
608 are added to prediction piece P in adder 634
n(x, y) 612 and obtain the piece of a decoding
As a result, obtain a new reference block from the result, or obtain this reference frame 614 and it is stored in the memory 610 by the piece that combination belongs to same page.Transmitter video encoder and receiver video encoder all have same reference frame or piece so that use like this.
In multiplexer 624, comprise the combination of the information of mobile vector information and predication error frame and this frame information 616 and be sent to decoder (Fig. 7).
Can implement above-mentioned video encoder function in many ways, for example by the software on a processor, carried out or by hardware, i.e. the logic setting of isolated component or ASIC (using specific integrated circuit).
Fig. 7 represents an example of decoder, may be implemented in by it to be used to reduce temporary transient redundant said method in the frame of video conveyer.The predication error frame 702 of coding that can receiving video signals 700 separated from one another in demultiplexer 710 and the mobile vector coefficient or the mobile vector field coefficient 712 of coding.
In the memory 716 of decoder, a reference frame R who has at least one adjacent block of encoding morning and/or be divided into polylith
n k(x, y).Same number of frames piece and/or reference frame storing are in the encoder memory.
In
piece 714, form a predicting candidate P with same way as in encoding block 618 (Fig. 6)
n(x, y)
718.In piece 704, form the predication error frame of a decoding
In
adder 720, the predication error frame of decoding
With predicting candidate P
n(x, y) 718 is added together.Produce the piece of a decoding of this form thus
Wherein one or more early coding adjacent block and/or be divided into a plurality of reference frame R
n k(x y) 722 is stored in the
memory 716, and wherein x and y are vector components, and wherein x is illustrated in the position of pixel of horizontal direction or piece, and y is illustrated in the position of pixel of vertical direction or piece, and
With
Form a pixel, the mobile vector of frame section or piece.
Can implement above-mentioned video encoder function in many ways, for example by the software on a processor, carried out or by hardware, i.e. the logic setting of independent component or ASIC (using specific integrated circuit).
Fig. 8 illustrates a terminal of a radio system, can implement the method for the preferred embodiments of the present invention by it.Same task is carried out on the transmitter section ground of the transmitter of user terminal 824 and radio system network element.Terminal 824 for example can be a mobile phone or comprise but do not limit the microcomputer of radio of the present invention part.Described terminal comprises an antenna 800, and it is used for transmitting and receiving signal by a duplexer filter.This terminal also comprises some antennas or a multiaerial system.This terminal also comprises a transmitter 802, it amplifies a transferred signal and sends it to this antenna, a modulator 804, it by selected modulator approach with the data signal modulation carrier wave that comprises desired information, a receiver 806, its amplifies the signal from antenna, and will be transformed into a desired intermediate frequency under it or directly arrive base band, a demodulator 808, the signal of demodulate reception makes this data-signal energy and carrier separation.
This user terminal also has the work of controll block 816 control terminal different pieces, example beginning camera 820 and Video Codec 822, with talk or the data of taking necessary action processing by the user of user's generation, Digital Signal Processing (DSP) for example, digital-to-analog conversion and filtering.In addition, for example in WCDMA, signal spectrum expands to the broadband by a pseudorandom extended code in a transmitter and go expansion in receiver, increases the capacity of channel thus as possible in spread spectrum system.Controll block is also carried out Code And Decode, for example the channel and the coding of talking.In addition, this controll block adopts the signal of transmission and the air-interface standard that signaling information mates employed radio system.User terminal comprises that one or more can be the dissimilar memory (not shown) of other polyliths or independent piece.In Fig. 8 example, controll block also comprises a program of control terminal work.In this figure example, the video coder-decoder 822 that the playback of data of a frame is carried out Code And Decode is independent, but it also can be the part of this controll block 816.
During transmit frame, camera 820 advances memory block 818 with a width of cloth digital image recording.This camera 820 can also directly record this image in the coding decoder 822.Coding decoder is encoded to the frame of selecting by the combination of one of said method or Several Methods.Camera 820, coding decoder 822 and memory 818 can reside in the user terminal 824 or belong to independent camera unit 826, and in the case, camera unit 826 usefulness interfaces 828 are connected to user terminal 824.
The user of user terminal in case of necessity can be on display 814 observer frame.For example Bian Ma frame playback of data is sent to telecommunication system and other phone according to the mode identical with talk or text data.
The operation block of above-mentioned terminal, for example the controll block and the coding and decoding piece 822 of the requirement of frame Code And Decode can be implemented by multiple mode, for example by the software on a processor, carried out or by hardware, for example the logic setting of independent component or ASIC (using specific integrated circuit).The multiple mode that termination function shown in the figure also can be different from this figure is incorporated in a plurality of operation block, and for example, this coding and decoding piece 822 can be divided into an encoder.
The user interface of terminal comprises a loud speaker or earphone 810, microphone 812, a display 814 and may also have a keyboard that they all are connected to controll block 816.
Though below with reference to example the present invention has been described by accompanying drawing, obvious is that the present invention is not limited but correct in every way in the scope of invention disclosed design in claims.The present invention can be with some video encoding standards, for example H.263 or MPEG use together.