CN1143544C - Picture coder, picture decorder, and picture transmission system - Google Patents

Abstract

A transmission system transmits a series of coded frames from a transmitting device to a receiving device, using both intra-frame coding and inter-frame coding. The receiving device decodes the frames and sends acknowledgment signals to the transmitting device. The transmitting device selects the reference frame employed in inter-frame coding according to these acknowledgment signals. The method of reference frame selection can be varied according to an assessment of transmission channel quality. The assessment criteria, or the reference frame selection method itself, can be varied in response to input from a human user.

Description

Image encoder, image decoder and visual transmitting system

Technical field

The present invention relates to the picture coding device, image decoder and visual transmitting system, the data compression performance that this system holds combines with high LOF fault-tolerance.

Background technology

Recently, the system that sends motion video by communication network has obtained developing rapidly, as comprises and use video telephone, video conference and video on demand (VOD) system, in order to reduce the transmission data volume, image be digitized and on transmitting apparatus compressed encoding, and on receiving equipment, decode.

The fundamental type of moving-picture coding can be divided into basic frame coding (Intra-frame coding) and interframe encode (Inter-frame coding) two classes.Basis frame coding each frame of encoding respectively, an or frame is divided into several each piece of encoding respectively, the whole bag of tricks that basic frame is encoded be by standardization, for example JPEG (Joint photographic Experts Group).Basis frame coding comes packed data by lowering in every frame the space redundancy.

Interframe encode not only reduces spatial redundancy and also reduces temporal redundancy, and it is only encoded with the difference of former frame to this frame.Interframe encode can comprise the motion compensation that produces high data compression rate.

Used basic frame coding and all standardization of interframe encode, for example MPEG (MovingPicture Experts Group) and ITU-T (Telecommunication StandardizationSector of International Telecommunication Union).Figure 21 has illustrated the coded system of using in the H.261 suggestion of ITU-T.Basis frame coding is carried out with the interval of rule as shown by hatching in FIG.; Interframe encode is carried out on At All Other Times.In interframe encode, every frame is that benchmark is encoded with back to back former frame, as shown by arrows, predicts out with the precursor frame with some frames that interframe encode is coded, is called the P-frame.And some coded frames of basic frame coding are referred to as the I-frame.In Figure 21, frame a and i are the I-frame, and frame b is to h, and j and k are the p-frame.

Figure 22 has shown the example that appears at problem in the H.261 scheme and similar scheme.If because of some reason receiving system can not decoded frame e, frame f so, g and h also can not decode.Receiving system must be waited for before decoding restarts, and receives next I-frame (frame i) up to it.Therefore, lose a frame and can be directed at tediously long in receiving active images and highly undesirable gap.

LOF can be because of multiple former thereby generation, and some frames can intentionally be lost at receiving terminal, for example because decoder has slower processing speed than encoder and can not continue.Sending in the network of data with packets of information or unit, when network over loading, packets of information or unit can on the way be lost.This phenomenon for example can appear in the local area network (LAN) that adopts famous Ethernet system, or appears in the wide area network of famous ATM(Asynchronous Transfer Mode).In wide area network, packets of information or unit can also arrive not according to sequence especially, for example because send on different network paths.This causes problem again in the decoding of P-frame.

In order to handle the problem of these back, some agreements of some network utilisation, the packets of information that transmitting apparatus sends in this agreement is with a serial number, receiving equipment rearranges packets of information with correct order, determine their arrival, and the request of sending wants dispensing device to resend not have the packets of information that arrives.The famous example of this type protocol is Transmission Control Protocol (Transmission controlProtocol transmits control protocol).

Yet when network work was unstable, packets of information was lost continually, and resending in this type protocol can cause the accumulated delays that generation is big, and this real-time transmission for active images is unaccommodated.When transmitting the active images data, preferably show new data usually, even that device frame-skipping does not go to wait for resending of old data yet.

These problems for example are combined in broadcasting and multicasting scheme in the multileaving scheme, and the multicasting scheme sends identical data to a plurality of acceptance points.If dispensing device is noted from the next request that resends of an acceptance point, will force frequently to other point and send a packets of information that has successfully received at other point, offered load will increase greatly like this.Therefore, broadcasting and multicasting are worked not carrying out under the agreement that resends usually, for example udp protocol (User Datagram protocol User Datagram Protoco (UDP)).But consequently the probability of LOF has increased.

In wireless network, even occur on the special-purpose passage, rather than utilize bag or elements exchange when transmitting, LOF is a serious problem.Wireless transmission makes mistakes highly easily, and when these mistakes surpassed the error correction capability of receiving system, common in fact certain part of obliterated data was handled so that rebuild active data.Therefore, losing of data is tending towards than more in the cable network.

These effects limit the serviceability of encoding scheme illustrated in fig. 21.This scheme is by telephone wire, Integrated Service Digital Network(ISDN) (ISDNS) and between transmitting and receiving device, and being equivalent to physical circuit provides other facility of reliable connection to transmit active images.Transmit for the network by other type, wherein LOF or frame-skipping are inevitably to a certain extent, and the illustrated scheme of Figure 23 is through being usually used in: for example with the JPEG coding by I-frame whole frames of encoding.

When all frames are pressed I-frame coding, appear at frame e among Figure 24 if lose, it feels quality to active images some influence.In Figure 24, only lost frame e; The frame f of back, g and h be the successful decoding of energy because previous frame is not depended in their decoding.

The problem that all is the scheme of I-frame is that the compression ratio of data is not very high, because do not remove the redundancy of time, therefore a lot of network bandwidths is wasted.

Japan Patent Kokai publication No.95571/1995 announces another kind of scheme, is illustrated in Figure 25.Wherein P-frame b is a baseline encoded with front I-frame (a) all to h.In this scheme, a P frame lose the decoding that does not influence other P-frame, the shortcoming of this scheme is a data compression rate, along with because the increase of time gap between P-frame and the I-frame, each subsequent P-frame certainly will descend, and the P-frame is to be benchmark with the I frame.

Summary of the invention

An object of the present invention is to make the P-frame, comprise with front P-frame being the P-frame of baseline encoded, can be decoded after LOF, needn't wait for next I-frame.

Another object of the present invention is to make image quality be adapted to transmit the passage situation.

Purpose in addition of the present invention is to make data compression rate be adapted to transmit the passage situation.

Of the present invention a purpose is arranged again is that the quality control of the active images that send is offered the user, to adapt to the different passage situations that transmit.

Transfer system of the present invention sends series of frames to receiving system from dispensing device.Every frame is selected basic frame coding or interframe encode, and the respective coding processing is performed on dispensing device.When selecting interframe encode, this frame is baseline encoded with the reference frame, and reference frame is the former frame that has been encoded.Coded data by basic frame coding and interframe encode is sent to receiving system.The data that receiving system decoding is encoded, and confirmation signal sent back to dispensing device, dispensing device is selected reference frame according to these confirmation signals.

In first preferred mode of operation, receiving system sends positive confirmation signal, and the frame that dispensing device is selected just confirming is as the reference frame.

In second preferred mode of operation, receiving system sends negative acknowledgement, and dispensing device selects up-to-date coded frame as the reference frame, unless receive negative acknowledgement.When receiving negative acknowledgement, reference frame is set the former frame of getting back to negative acknowledgment, the reference frame that negative acknowledgement can follow desired reference frame number that dispensing device selective reception device is successfully decoded.

In the 3rd preferred mode of operation, negative acknowledgement and positive confirmation signal all send, and the quality of sendaisle is assessed according to these confirmation signals, and the method for reference frame selection changes with assessment.For example, under bad passage situation, reference frame can be selected by the first top optimal way, and under good passage situation, reference frame can be selected by the second top optimal way.Channel quality evaluation criteria and reference frame selecting method can also change with user's input.

Therefore, a first aspect of the present invention provides the encoder in the dispensing device, be used for a series of frame is encoded, and coded data is sent to decoder in the receiving equipment through transmitting passage, and described encoder comprises: be used to receive the receiving system from the affirmation signal of described decoder; Be used for storing the memory device of at least one frame of said series of frames; The reference frame updating device, according to said confirmation signal, select a reference frame of a coded data of successfully being decoded by described decoder in a plurality of frames that from said memory device, store: determine device, which frame that is used to refer in said series of frames is encoded with basic frame, and which frame is encoded with interframe encode; Be coupled to the code device of said memory device and said definite device, thereby to realize producing coded data by the basic frame coding and the interframe encode of said definite device indication, interframe encode is that benchmark is realized by the indicated reference frame of said reference frame updating device; And be coupled to the dispensing device of said code device, be used for the reference frame number of said coded data and the described reference frame of indication is sent to said decoder.

Another aspect of the present invention provides the decoder in a kind of receiving equipment, be used for transmitting passage and receive coded data from the encoder of realizing basic frame coding and interframe encode to obtain a series of frame by one, described decoder comprises: a receiving system is used to receive described coded data and reference frame number from described object machine; Be connected to a decoding device of described receiving system, be used to by the reference frame that described reference frame number the limits described coded data of decoding; With a confirmation signal dispensing device that is connected to described decoding device, be used for sending the affirmation signal of confirming respective frame to described encoder, described confirmation signal comprise described frame number and represent the described decoder described frame of whether successfully having decoded.

The present invention also provides a kind of and from dispensing device the frame of a series of expression active images has been transmitted the method that passage sends to receiving system by one, comprises following each step: sent to the series of frames in the described receiving system in described dispensing device before the storage; In said series of frames, by in basic frame coding and interframe encode, adjudicating, for every frame is selected a coding method; Selecting basic frame Methods for Coding coded frame, need be not benchmark with other frame; Selecting the method coded frame of interframe encode, is benchmark with the reference frame, and said reference frame is the frame that the front has sent; To send to receiving system from said dispensing device by said frame by interframe encode coded data that produces and the reference frame number of representing reference frame; On said receiving system, said coded data is decoded with the reference frame that described reference frame number is represented; Send confirmation signal for said dispensing device from said receiving system, described confirmation signal comprises the frame number of described frame, and represents the described receiving system described frame of whether successfully decoding; And, on said dispensing device, select said reference frame according to said confirmation signal, to select to be received a frame that device successfully decodes as the reference frame.

Description of drawings

Fig. 1 is the calcspar of moving picture device among first embodiment.

Fig. 2 is the calcspar of decoding moving image device among first embodiment.

Fig. 3 has shown the example of interframe encode relation among first embodiment.

Fig. 4 has shown another example of interframe encode relation among first embodiment.

Fig. 5 has shown the example of LOF among first embodiment.

Fig. 6 is the calcspar that has shown a kind of variation of moving picture device among first embodiment.

Fig. 7 is the calcspar of the reference frame updating block of moving-picture coding device among second embodiment.

Fig. 8 has shown the example of LOF among second embodiment.

Fig. 9 has shown the example that confirmation signal is lost among second embodiment.

Figure 10 is the calcspar of the reference frame updating block of moving picture device among the 3rd embodiment.

Figure 11 has shown the example of LOF among the 3rd embodiment.

Figure 12 is the calcspar of the reference frame updating block of moving picture device among the 4th embodiment.

Figure 13 has shown the example of LOF among the 4th embodiment.

Figure 14 has shown the example that the LOF back has confirmation signal to lose among the 4th embodiment.

Figure 15 is the calcspar of moving picture device among the 5th embodiment.

Figure 16 is the calcspar of decoding moving image device among the 5th embodiment.

Figure 17 has shown the example of LOF in the 5th embodiment.

Figure 18 is the calcspar of moving picture device among the 6th embodiment.

Figure 19 is the calcspar of moving picture device among the 7th embodiment.

Figure 20 is the calcspar of decoding moving image device among the 7th embodiment.

Figure 21 explanation is the interframe encode relation in first prior art scheme.

Figure 22 has shown the example of LOF in first prior art scheme.

Figure 23 explanation is the interframe encode relation in the 2nd prior art scheme.

Figure 24 has shown the example of LOF in the 2nd the prior art scheme.

The relation of Figure 25 explanation interframe encode in the 3rd prior art scheme.

Embodiment

Embodiments of the invention will be narrated with reference to described illustrative figure.

These embodiment for example can be used as semiconductor integrated circuit and realize.Here omitted the explanation on the circuit grade.As the method that designs and make required integrated circuit those persons skilled in the art are understood.The present invention can also realize by the method for software.

First embodiment illustrates the general structure of encoder in the moving image transmitting system of the present invention, and has shown a scheme of reference frame selection, and other scheme will elaborate in the embodiment of back.

Fig. 1 is the functional block diagram of moving-picture coding device 300 among first embodiment.This places the encoder of dispensing device to comprise active images input unit 301, coding unit 302, decoding unit 303, frame memory unit 304, reference frame memory unit 305, basis frame/interframe determining unit 306, coded data transmitting element 307, confirmation signal receiving element 308 and reference frame updating block 309.

Active images input unit 301 is the active images data of for example importing from video frequency camera, supply coding unit 302 frame by frame.Coding unit 302 coding incoming frame data, and coded data supply decoding unit 303 and coded data transmitting element 307.Coding unit 302 is realized basic frame coding and interframe encode under the control of basic frame/interframe determining unit 306.To interframe encode, coding unit 302 references are stored in the reference frame in the reference frame memory unit 305, and preferably adopt motion compensation only the difference between present frame and the reference frame to be encoded.

Decoding unit 303 is decoded from the coded data of coding unit 302 receptions, and decoded frame and frame number are write in the frame memory unit 304, frame memory unit 304 these decoded frames of storage.The reference frame that 305 storages of reference frame memory unit are copied from frame memory unit 304 by reference frame updating block 309.

Basis frame/interframe determining unit 306 is selected basic frame coding or interframe encode, and selecting to be notified to coding unit 302, decoding unit 303 and coded data transmitting element 307.Common basic frame coding is selected (for example per 30 frames once) with fixed intervals, and interframe encode is being selected At All Other Times.And basic frame/interframe determining unit 306 can receive one and force the refresh signal (RFSH) of selecting basic frame coding, and refresh signal is sent by decoder, and this describes in detail in the back.When selecting basic frame coding, also notification reference frame update unit 309.

Coded data transmitting element 307 sends the coding moving image data for an active images decoder or the different a lot of active images decoders that receive the place by transmitting the passage (not shown).Coded data transmitting element 307 multiplex heading messages are to output stream.Heading message comprises reference frame number (as the P frame), and basic frame/interframe mark and other information that needs are as transmitting routing iinformation.

Confirmation signal receiving element 308 receives positive confirmation signal (ACK) from the active images decoder, and notice has had the reference frame updating block 309 of each frame of just confirming.In multileaving, confirmation signal receiving element 308 notice is at least by some acceptance point, preferably the reference frame updating block 309 of these frames of just having been confirmed by all acceptance points.

The notice that reference frame updating block 309 response basic frame/interframe determining units 306 and confirmation signal receiving element 308 receive is upgraded reference frame memory unit 305.Wait for and to encode by coding unit 302 according to the basic frame coding notification reference frame update unit 309 that basic frame/interframe determining unit 306 receives, decoding unit 303 decoding also writes on present frame in the frame memory unit 304, then, this frame is copied to reference frame memory unit 305 from frame memory unit 304, and the whole frame data in the delete frame memory unit 304.According to the frame number that receives positive acknowledgement frame in the confirmation signal receiving element 308, reference frame updating block 309 waits for that coding unit 302 removes to finish to use the frame that is stored in the reference frame memory unit 305, remove the current frame of encoding, 304 positive acknowledgement frames copy reference frame memory unit 305 to from the frame memory unit then, and comprise the frame data of positive acknowledgement frame in the delete frame memory unit 304.

Fig. 2 is the functional block diagram of active images decoder 400 among first embodiment.This places the decoder of receiving equipment to comprise coded data input unit 401, reference frame comparing unit 402, reference frame updating block 403, reference frame memory unit 404, frame memory unit 405, decoding unit 406, confirmation signal transmitting element 407, active images output unit 408 and refresh signal transmitting element 409.

Coded data input unit 401 receives coded data and the heading information that moving-picture coding device 300 sends, and coded data supply decoding unit 406, from heading information, coded data input unit 401 extracts basic frame/interframe mark and (being the P-frame) reference frame number, basic frame/interframe mark is offered decoding unit 406, reference frame number is offered decoding unit 406 and reference frame comparing unit 402.

Reference frame comparing unit 402 compares the reference frame number that receives in the coded data input unit 401 with the reference frame number that is stored in the reference frame memory unit 404.If the reference frame number that receives is upgraded than the reference frame number of storage, reference frame comparing unit 402 passes to reference frame updating block 403 to the reference frame number that is received with the reference frame update request.

According to the reference frame update request that receives, reference frame updating block 403 is pressed the reference frame number that is received and is sought this frame in frame memory unit 405.If find this frame, reference frame updating block 403 copies this frame and frame number thereof in the reference frame memory unit 404 to, thereby upgrades reference frame.

Reference frame memory unit 404 storage one width of cloth reference frame and its frame numbers.The frame number of the frame data of frame memory unit 405 storage decodings and some up-to-date frames.No matter when, reference frame updating block 403 has upgraded reference frame by copying reference frame memory unit 404 to from frame memory unit 405, and the frame that then is copied and the frame of all fronts are deleted from frame memory unit 405.

Decoding unit 406 is according to the coded data of appended basic frame/interframe mark decoding from 401 receptions of coded data input unit.When this mark indication interframe encode, decoding unit 406 checks that at first the reference frame number coupling that receives is stored in the reference frame number in the reference frame memory unit 404, has only these data of just decoding when these two frame numbers are complementary; Decoding in this case is the inter-frame decoding according to the reference frame of storage in reference frame memory unit 404.

During decoding processing, decoding unit 406 has been carried out for example this error checking of cyclic redundancy check (CRC).If check and to pass through, think that this frame successfully finished decoding, decoding unit 406 is delivered to active images output unit 408 to decoded data.If this frame is the I-frame, decoding unit 406 is also write reference frame memory unit 404 to decoded data and frame number, so upgrade reference frame and frame number is offered confirmation signal transmitting element 407.If this frame is the P-frame, decoding unit 406 is write frame memory unit 405 to decoded data and frame number.In some cases, decoding unit 406 also offers confirmation signal transmitting element 407 to the frame number of the P-frame of successfully decoding.Can utilize different situations, if frame number is supplied with confirmation signal transmitting element 407 when the supposition coded data surpasses a certain threshold size in the following explanation.

According to the frame number that receives from decoding unit 406, confirmation signal transmitting element 407 confirms that (ACK) signal sends to moving-picture coding device 300 to a forward.Ack signal comprises the frame number that receives from decoding unit 406.

Active images output unit 408 is given monitor display unit or similar device from decoding unit 406 output decoder data, refresh signal transmitting element 409 receives the refresh requests from user's control input device 420, and delivers to moving-picture coding device 300 refreshing (RFSH) signal accordingly.

Next step, the operation of first embodiment will illustrate with reference to figure 3,4 and 5.In these figure and later figure, confirmation signal is represented with straight arrows, and is represented referring-to relation with curve arrow.

The situation that Fig. 3 shows, wherein Code And Decode processing and data transfer process are to carry out than higher speed.First frame (a) is encoded as the I-frame, and successfully be sent out and by active images decoder 400 decoding, decoder 400 also returns the reference frame memory unit 404 of the data-storing of decoding at it to moving-picture coding device 300 to the forward confirmation signal, shown in first arrow of mark ACK.

Frame a is also decoded by decoding unit 303 in moving-picture coding device 300.The data of decoding are write frame memory unit 304, copy reference frame memory unit 305 then to, use as the reference frame so that make in encoder reference frame memory unit 305 and 404 include same decoded data.

Then some frames (b, c, d and e) successfully are encoded and decode according to this reference frame (a).Frame b, c and d a at same frame is not very different, thereby the size of coded data is little and active images decoder 400 does not return confirmation signal.And frame e has enough difference, thereby causes encoded data size that certain increase is arranged, and active images decoder 400 correspondingly returns a positive confirmation signal to frame e.

Moving-picture coding device 300 received this confirmation signal before next frame (f) coding beginning, reference frame updating block 309 so can in time copy the decoded data of frame e to reference frame memory unit 305 is so that be basic coding frame f with frame e.Thereby reduce the size of the coded data of frame f.

When the coded data of active images decoder 400 received frame f, it is that frame number (frame a number) than storage in reference frame memory unit 404 upgrades that reference frame comparing unit 402 is sought appended reference frame number (frame e number), and domination reference frame updating block 403 from frame memory unit 405 copy frame e to reference frame memory unit 404.Then, decoding unit 406 is according to frame e decoded frame f successfully.

Frame g and h are baseline encoded and successfully decoding with frame e also.

In multiple spot links, have only some acceptance point at least, when preferably all acceptance point returns positive confirmation signal for frame e, just reference frame is changed framing e from frame a.

Fig. 4 illustrates a kind of situation, and wherein the active images decoder returns a positive confirmation signal for frame d, but because coding, decoding or transfer rate are all slow, and moving-picture coding device 300 just receives this positive confirmation signal after frame e has begun coding.Therefore, frame e is baseline encoded and decoding according to the reference frame a that exists.Then, reference frame changes framing d, and following each frame (f, g and h) is baseline encoded and decoding with frame d.

Fig. 5 illustrates the situation of LOF, frame b, and c, d and e are baseline encoded with first I-frame, frame b is successfully decoded to d, and returns positive confirmation signal for frame d.The coded data of frame e is lost in transmission or is destroyed, and represents with the X-mark in the drawings.As a result, coded data or can not receive has perhaps received but can not successfully decode, and as represented with cross-hauling, this does not influence following frame f, g and h, and these frames are successfully Code And Decode of benchmark with positive acknowledgement frame d.Active images decoder 400 needn't be waited for next I-frame (i) in order to continue decoding again after frame e loses.

Obtain undesirable image damage if now examine the user of active images output by active images output unit 408, for example owing to there is not the mistake of detection, at this moment he or she can push button or carry out similar operations, promptly sends a refresh command to refresh signal transmitting element 409.Then, send a refresh signal for moving-picture coding device 300, force next frame as I-frame coding, thereby recover good image quality.

In first embodiment, the P-frame lose the decoding that at all can not hinder subsequent frame because the P-frame that has only forward to confirm can become reference frame.With the prior art difference of Figure 23, decoding can not waited for the arrival of next I-frame and continue.Simultaneously, successfully decoded and forward ground when confirming when the P-frame, therefore it can also, reduce temporal distance between coding and the reference frame with the reference frame of doing the subsequent frame coding, compares with prior art shown in Figure 26, improved data compression rate.

Especially, when system situation allows all frames successfully to be sent and to decode in fact, if in order to send confirmation signal, the encoded data size threshold value is arranged on an enough low level, the data compression performance of first embodiment will be near the situation of scheme explanation among Figure 22, and wherein every frame is baseline encoded with the former frame.

When the passage situation poorly is, or threshold value is when higher, and positive confirmation signal will be little frequent, and reference frame upgrades and will often not take place, and therefore, data compression rate is with corresponding decline.Yet, providing same when preventing LOF, first embodiment still can obtain higher data compression rate than the art methods of Figure 26.

Like this, first embodiment allows data compression rate to become with the passage situation, makes the good passage situation of transfer system utilization can obtain high compression rate, even and under disadvantageous passage situation, still keep good image quality.

The variation scheme of Fig. 6 explanation first embodiment when moving-picture coding device 500 does not have decoding unit.The other parts of encoder are identical with Fig. 1, and each several part has identical reference number.Except active images are directly write the frame of each reception the frame memory unit 304, this operation that changes scheme is same as above.Except the structure and operation of having simplified moving-picture coding device 500, the advantage that this variation scheme has relates to the precision of motion compensation.Its shortcoming be the moving-picture coding device with the active images decoder not with identical just digital coding and decoding P-frame.

About other variation, decoding unit 303 or the active images input unit 301 among Fig. 6 in Fig. 1 can be suitable for directly the I-frame data being write reference frame memory unit 305, rather than at first data are write frame memory unit 304, copy reference frame memory unit 305 to by reference frame updating block 309 then.Arrange to have simplified the operation of moving-picture coding device like this.

About other variation, a frame memory unit can be formed in reference frame memory unit 305 and frame memory unit 304, and removing to indicate which storage frame with a pointer is reference frame.By eliminating the data copy from a memory to another memory, the operation of moving-picture coding device has further been simplified in arrangement like this, has reduced the requirement of memory.

The frame memory cell that this is combined into can be as a circular buffer, when memory is full, writes until old on the data with new data is topped, thereby needn't delete old data.Same method, when unit 304 not with reference frame memory unit 305 in conjunction with the time, can be used for frame memory unit 304.

About also having other variation, decoding unit 406 in Fig. 2 in the active images decoder 400 can be suitable for estimating decoding processing, and when the sign of low quality image or other decoding problem occurs, send refresh command for refresh signal transmitting element 409, then, refresh signal transmitting element 409 does not need to import from the user and can send refresh signal.When previous frame can not successfully be decoded, refresh signal also can send.

Also have a kind of variation scheme, moving-picture coding device 300 is divided into a lot of pieces to every frame, encodes every respectively.In the P-frame, every being baseline encoded corresponding to the piece in the reference frame, can keep an independent reference frames number to each piece, so that different masses can be that benchmark is encoded with different reference frames in same frame, and each piece can be given a confirmation signal independently.Because piece is littler than frame, the probability of mistake has also reduced, and has improved the turnover rate of whole reference frame and the compression ratio of data, and the influence of losing is limited on the concrete piece in a frame.

Second embodiment

Except active images decoder 400 confirms that each successful decoded frame, second embodiment is identical with first embodiment.

Fig. 7 has illustrated the internal structure of reference frame updating block 309A, comprising a frame number setup unit 310 and data copy cell 311.This reference frame updating block 309A example illustrates a preferred internal structure of reference frame updating block 309 among first embodiment.

Frame number is provided with the frame number that unit 310 setting check signal receiving elements 308 receive the frame of just confirming.These frame numbers are set at internal work memory field (not shown), and pass to data copy cell 311.

304 copies have by frame number and these frames of the frame number that unit 310 receives are set to reference frame memory unit 305 data copy cell 311 from the frame memory unit.

The operation of second embodiment will be described below.

Reference frame updating block 309A upgrades reference frame according to the ack signal that active images decoder 400 sends.In Fig. 8, for example active images decoder 400 is represented acknowledgement frame a by the arrow of mark ACK, b and c.Moving-picture coding device 300 receives ack signal for frame b after the coding of frame c has begun.So frame c is a baseline encoded with frame a still, but frame d is a baseline encoded with frame b.Equally, frame e is a baseline encoded with frame c.Active images decoder 400 is benchmark decoded frame b and c with frame a, is benchmark decoded frame d with b, and is benchmark decoded frame e with frame c.

The decoding of frame d is unsuccessful, so active images decoder 400 does not send ack signal for frame d.Therefore, moving-picture coding device 300 does not change reference frame when coded frame f, and the usefulness frame c that continues is as the reference frame.Like this, active images decoder 400 can remove decoded frame f.

Frame e is successfully decoded and is confirmed, so frame g is a baseline encoded with frame e, frame g also just can successfully decode.

In the 2nd embodiment, as at first embodiment, frame d loses the decoding that does not influence follow-up each frame, and they can not wait for next I-frame and successfully decoding.Owing to lose the minimum that is declined to become that causes image quality.Data compression rate also improves, and as frame c, d, e, g and h are baseline encoded with the frame before 2 frames all, and frame f is a baseline encoded with the frame before three frames only.With frame c replace frame d as the reference frame of frame f only slight usually influence data compression rate.

Fig. 9 has shown by sending ack signal, the example of losing from active images decoder 400 to moving-picture coding device 300 on passage.In this example, frame d is successfully encoded.But corresponding ack signal is on the way lost.Moving-picture coding device 300 works with mode identical among Fig. 8, is baseline encoded frame f with frame c, is baseline encoded frame g or the like with frame e.All these frames can successfully be decoded, and data compression rate is identical with Fig. 8.

Therefore, the propagation of second embodiment both direction between encoder to dropout and data has high fault-tolerant power.

The 3rd embodiment is different from the embodiment of front, is that its active images decoder 400 sends negative acknowledgment (NACK) signal, rather than positive confirmation signal.When a frame is lost in transmission or during the destruction that is subjected to correcting, sent a negative acknowledgement.Negative acknowledgement comprises the frame number of the frame that does not receive or the frame number of the erroneous frame that can't correct that received.

Except this difference, the operation of the active images decoder in the 3rd embodiment is identical with the mode among first embodiment with structure shown in Figure 2.

Moving-picture coding device 300 has structure shown in Figure 1, but the operation difference of reference frame updating block.

Figure 10 has illustrated the internal structure of reference frame updating block 309b among the 3rd embodiment, and suffix B represents this structure with structure difference among the 2nd embodiment, and its part is frame number setup unit 310A, data copy cell 311A and frame number determining unit 312.Suffix A also represents with the 2nd

The difference of embodiment.

Frame number setup unit 310A supplies with frame number determining unit 312 to the frame number of the negative acknowledgment frame that receives from confirmation signal receiving element 308, and receive the frame number of determining recently that returns from frame number determining unit 312, frame number setup unit 310A (not shown) in inner memory is set each up-to-date definite frame number, and these up-to-date definite frame numbers are passed to data copy cell 311A.

When receiving the frame number of negative acknowledgment from frame number setup unit 310A, seek in the frame number that frame number determining unit 312 store from frame memory unit 304, older and not by the up-to-date frame number of negative acknowledgment to find out than the frame number of frame number setup unit 310A reception.This frame number returns to frame number setup unit 310A as up-to-date definite frame number.

Data copy cell 311A before coding unit 302 begins to encode next frame, copies each frame in the frame memory unit 304 to reference frame memory unit 305 at frame by the decoding unit among Fig. 1 303 usually in being written to frame memory unit 304.And according to the up-to-date definite frame number that receives from frame number setup unit 310A, data copy cell 311A is from frame memory unit 304, and the frame that will have this frame number copies reference frame memory unit 305 to.

No matter when be, when a new reference frame is copied into reference frame memory unit 305, delete frame data from frame memory unit 304 not, the operation of the other parts of moving-picture coding device 300 is as first embodiment, do not having under the situation of negative acknowledgement, in frame memory unit 304, preserve frame data and reach sometime, preferably long as far as possible.

Below, the operation of the 3rd embodiment is described with reference to Figure 11.

As long as moving-picture coding device 300 do not receive negative acknowledgement, reference frame updating block 309B supposes that all transmit frames successfully decodes, and to keep the renewal reference frame be up-to-date coded frame.Therefore, common every frame is a baseline encoded with the former frame that is right after, shown in the top solid arrow of Figure 11, the frame b in Figure 11, c, d and e encode with such method.

Frame a, b and c are successfully decoded, but frame d lost in transmission or destroyed, and active images decoder 400 returns a negative acknowledgement (NACK).This NACK signal is that benchmark arrives moving-picture coding device 300 when frame e is encoded at coding unit 302 with frame d.In case coding unit 302 has been finished the coding that relates to frame d, reference frame updating block 309B is not promptly copied to reference frame memory unit 305 by the latest frame of negative acknowledgment as yet before the frame d and upgrades reference frame by with frame c from frame memory unit 304.Here, frame d and e can delete from frame memory unit 304.Because very clear, active images decoder 400 is failed decoded frame d also can not decoded frame e.

Frame f is a baseline encoded with frame c now, shown in dashed lines arrow in top among Figure 11.Therefore, active images decoder 400 energy decoded frame f, frame e can not be decoded, because it is that benchmark is encoded with frame d, but frame e does not originally lose in transmitting or destroys.Therefore, 400 couples of frame e of decoding moving image device do not have negative acknowledgement and return when moving-picture coding device 300 coded frame f.

Therefore, data copy cell 311B copies frame f to reference frame memory unit 305 from frame memory unit 304, and frame g is a baseline encoded with frame f.Equally, frame h, i, each frame is a baseline encoded with back to back precursor frame among j and the k.

With second embodiment relatively, general the 3rd embodiment obtains higher data compression rate, because every frame is a baseline encoded with back to back precursor frame, rather than is benchmark with the frame before 2 frames.When LOF occurring, below a frame or a plurality of frame may not decode (as frame e among Figure 11), but needn't wait for next I-frame again and decode and become possibility very soon.

In multileaving, if be confirmed to be negative sense definite number of some acceptance point at least, confirmation signal receiving element 308 is demarcated a frame of negative acknowledgment.For example, if a frame is a negative acknowledgment at a point at least, just confirms and demarcate as negative sense.

The 4th embodiment is identical with the 3rd embodiment substantially, but all frames that 400 pairs of active images decoders can not successfully be decoded send negative acknowledgment (NACK) signal.No matter be to transmit mistake or frame itself is lost, still the precursor frame transmits mistake or loses.When losing possibility when existing on the confirmation signal passage, this mode is preferably, and each NACK signal had both been specified the frame number of the frame that can not decode, specified a desired reference frame number again.Desired reference frame number is the frame number of the latest frame of successfully decoding.

Following explanation is limited to the structure and the operation of reference frame updating block in the moving-picture coding device.

The internal structure of Figure 12 description references frame update unit 309C, suffix C represents that its structure is different from the 3rd embodiment.Its part comprises frame number setup unit 310C, data copy cell 311B, frame number determining unit 312A, alternative delete cells 313 and alternative number table holding unit 314.

Frame number setup unit 310C passes to frame number determining unit 312A to each negative acknowledgment frame number.If frame number determining unit 312A determines that the negative acknowledgment frame number is as described below, frame number setup unit 310C passes to data copy cell 311B and alternative delete cells 313 to desired reference frame number.If the frame number of negative acknowledgment does not determine that frame number setup unit 310C is failure to actuate.

When from frame number setup unit 310C, receiving the frame number of negative acknowledgment, frame number determining unit 312A seeks the frame number of negative acknowledgment in the alternative number table that is stored in alternative number table holding unit 314, and whether notice frame number setup unit 310C about existing the frame number of negative acknowledgment in alternative number table, and the frame number that report occurs on alternative number table is just thought and is determined.

When receiving a desired reference frame number, alternative delete cells 313 is deleted all reference frame number than the requirement that requires reference frame number to upgrade from alternative number table.When deleting frame number from alternative number table, corresponding frame data are also deleted from frame memory unit 304.

Alternative number table holding unit 314 storage and upgrade table alternative number.When each frame during by coding unit 302 coding, its frame number is added in alternative number table.When a frame was deleted from frame memory unit 304 and reference frame memory unit 305, its frame number was just removed from alternative number table.

The operation of data copy cell 311B is same as data copy cell 311A among the 3rd embodiment.Ordinary solution code element 303 copies each frame to reference frame memory unit 305 once this frame of having decoded as early as possible from frame memory unit 304.Yet when when frame number setup unit 310C receives desired reference frame number, data copy cell 311B 304 copies the frames with that frame number to reference frame memory unit 305 from the frame memory unit.

Below, the operation of the 4th embodiment will be illustrated with reference to Figure 13 and 14.

Figure 13 explains the situation identical with Figure 11 situation, and wherein frame d loses in transmission, and frame d and e can not be decoded, and the NACK signal is returned for this 2 frame d and e, and the frame number of two NACK signal designated frame c is as desired reference frame number.

Arrive up to these NACK signals, reference frame updating block 309C thinks that all frames are successfully decoded, and upgrades the reference frame on each frame.Therefore, frame b is that benchmark is encoded with back to back precursor frame all to e.

When the negative acknowledgment of reference frame updating block 309C received frame d, frame a is encoded and sends to frame d, and on alternative number table, frame e is encoded now.Frame number determining unit 312A determines that frame d shows at alternative number, so frame number setup unit 310C sends the frame number of frame c to data copy cell 311B and alternative delete cells 313.When finishing the coding of frame e, data copy cell 311 copies frame c to reference frame memory unit 305 from frame memory unit 304, the frame number of frame d and e in alternative number table of alternative delete cells 313 deletions.Frame d is deleted from frame memory unit 304.Frame e had better not write in the frame memory unit 304; If write, should delete immediately.

Frame f is a baseline encoded with frame c, and during frame f coding, reference frame updating block 309c receives a negative acknowledgment for frame e, and frame e is because of being baseline encoded with the frame d that loses, so can not be decoded.Frame number determining unit 312A seeks frame e in alternative table, but because it be can not find by 313 deletions of alternative list deletion above-mentioned unit.Because the negative acknowledgment of frame e can not be determined by frame number determining unit 312A, frame number setup unit 310C does not provide any frame number for data copy cell 311B and alternative delete cells 313.When frame f had been encoded, it was sent to active images decoder 400 and is added to and shows for alternative number, and data copy cell 311B copies frame f to reference frame memory unit 305 immediately.

Therefore, frame g is a baseline encoded with back to back precursor frame f, and the coding of follow-up each frame continues after this manner.

Figure 14 illustrates that a negative acknowledgement has been sent out but the situation that do not have to arrive.As former case, frame d is lost in transmission or is destroyed, and causes frame d and e not to decode.And at this moment, the NACK signal of frame d is also lost in transmission, so reference frame updating block 309C can not be familiar with frame d and can not be decoded, and continues the processing of common mode, is baseline encoded frame f with frame e.Frame a to e all be placed on alternative number the table in.

Active images decoder 400 sends another NACK signal for frame e, the frame number that is accompanied by frame number c is as desired reference frame number, and the NACK signal is received in moving-picture coding device 300, data copy cell 311B copies to the reference frame memory unit 305 from frame memory unit 304 frame c, alternative delete cells 313 is deleted (frame number d, e and f) all frame numbers that upgrade than frame c from alternative number table.Frame g is a baseline encoded with frame c.

Active images decoder 400 also sends the NACK signal for frame f, is accompanied by frame number c once more as desired reference frame number.Because frame number f deletes from alternative number table, and reference frame updating block 309c is no matter this NACK signal copies frame g rather than frame c to reference frame memory unit 305 from frame memory unit 304.Therefore, frame h can be a baseline encoded with frame g.

Because losing of frame d and its negative acknowledgement, 3 frames (d, e and f) can not be decoded, but system still can recover to use.Each frame can all successfully be decoded after frame g reached, so, avoided long-time image to worsen.And all frames except that frame g are baseline encoded with back to back precursor frame, so kept high data compression rate.

Whether the 4th embodiment that is not both between third and fourth embodiment checks that with frame number determining unit 312 frame of negative acknowledgment appears on alternative number table, as long as the frame number of negative acknowledgment not on alternative number table then no matter the NACK signal, the 4th embodiment avoids keeping an old reference frame according to the NACK signal that repeats redundantly.In Figure 13, frame g is a baseline encoded with frame c, and in the 3rd embodiment (Figure 11), frame g is a benchmark coding earlier with frame c.Therefore, when LOF occurring, the 4th embodiment can obtain better data compression rate than the 3rd embodiment.The advantage of the 4th embodiment along with between moving picture device 300 and the decoder 400 back and forth the increase of delivery time seem more important.

As the variation scheme of the 4th embodiment, active images decoder 400 can omit desired reference frame number from the NACK signal, and moving-picture coding device 300 can handle and also do not have negative acknowledgment, as last precursor frame of requirement reference frame.When the transmission channel height of NACK signal was reliable, this variation scheme was suitable.

About other variation is that active images decoder 400 can send ack signal and can send the NACK signal again, and the reference frame updating block 309C in moving-picture coding device 300 can handle and once received the up-to-date frame number of ack signal as want reference frame number.

Transfer system in the 5th embodiment is that every frame sends the plus or minus confirmation signal, and aligns the reference frame update mode suitably according to the passage situation.

Under good passage situation, reference frame number is generally to each frame update.When receiving negative acknowledgment, be provided with and get back to as at third and fourth embodiment.This mode of operation will be called as the NACK mode.

Under inappropriate passage situation, reference frame number only is updated receiving when just confirming, as first and second embodiment, this working method is called as the ACK mode.

Therefore, except the reference frame updating block, the moving-picture coding utensil has passage assessment unit and mode converting unit.

Figure 15 has shown the structure of moving-picture coding device 600 in the 5th embodiment, and the same section utilization with the identical sidenote numeral of Fig. 1, is had the sidenote numeral of alphabetic word tail to similar part usefulness.Following explanation concentrates on the part that does not have among the different part of same first embodiment or first embodiment.These parts comprise confirmation signal receiving element 308A, reference frame updating block 309D, passage assessment unit 315 and update mode converting unit 316.

Confirmation signal receiving element 308A confirmation of receipt signal, and the plus or minus confirmation passed to reference frame updating block 309D and passage assessment unit 315.If determine that one confirmation signal can't arrive in the time, confirmation signal receiving element 308A regards associated frame by negative acknowledgment as.

In multileaving, confirmation signal receiving element 308A is from each destination confirmation of receipt signal, calculates by being some point at least, and preferably the frame number of those frames of just being confirmed of have a few is treated by the frame of negative acknowledgment to remaining frame number conduct.

In the represented update mode of update mode converting unit 316, reference frame updating block 309D upgrades reference frame, and the notice that basic frame/interframe determining unit 306 and confirmation signal receiving element 308A receive is reacted.

In the ACK mode, when receiving the just affirmation of a concrete frame number from confirmation signal receiving element 308A, reference frame updating block 309D copies corresponding frame data to reference frame memory unit 305 from frame memory unit 304.At this moment, the data of this frame number front can be deleted from frame memory unit 304, and negative acknowledgment is not produced effect.

In the NACK mode, reference frame updating block 309D just begins usually the every frame in the frame memory unit 304 to be copied to reference frame memory unit 305 to before frame is encoded down at coding unit 302.And when when confirmation signal receiving element 308A receives a negative acknowledgment, nearest positive acknowledgement frame copies reference frame memory unit 305 to from frame memory unit 304, thereby set the reference frame of back, the information that identifies up-to-date positive acknowledgement frame remains among the reference frame updating block 309D, and is updated receiving when just confirming from confirmation signal receiving element 308A.At this moment, have frame number and just confirming that than up-to-date the old frame of frame number can delete from frame memory unit 304.

When from basic frame/interframe determining unit 306, receiving basic frame coding notice, reference frame updating block 309 copies the data of the present frame in the frame memory unit 304 (I frame) to reference frame memory unit 305, and all frame data in the delete frame memory unit 304.

The affirmation signal evaluation channel quality that passage assessment unit 315 receives according to confirmation signal receiving element 308A, and the result notification update mode converting unit 316 of assessment.Have only 2 evaluation grades in the following description, promptly good and bad.If when M or more a plurality of negative acknowledgment were for example arranged in N latest frame, channel quality was cited as bad.And if when being less than the M frame by negative acknowledgment in the N frame, channel quality is cited as that (M here, N is a positive integer, wherein N 〉=M).

Can also be with other evaluation criteria.For example when L successive frame was negative acknowledgment, channel quality was cited as bad, and L is a suitable positive integer here.

The assessment conversion update mode that update mode converting unit 316 is made according to passage assessment unit 315.For example, when assessment result to beginning some successive frames had been, mode can convert the NACK mode to, and when assessment result be bad time to some successive frames of back, mode changes into the ACK mode.

Figure 16 has illustrated the structure of active images decoder 700 in the 5th embodiment.To with part identical among Fig. 2 with same reference number, and to similar part with originally with reference to back word adding alphabetic word tail.The structure of the active images decoder in the present embodiment is basically with identical among first embodiment.But there is the function of some identical parts different, will narrates below these differences with operation.

Reference frame comparing unit 402A compares with the reference frame number that writes down in the reference frame memory unit 404 reference frame number that receives by each frame, and works as reference frame number and do not send the reference frame update request to reference frame updating block 403 simultaneously.And in first embodiment, only forwards to reference frame number be updated to newer frame number.Among the 5th embodiment, also allow the reference frame number of backward directions is updated to older frame number.

In first embodiment, the frame data that decoding unit 406A decoding is received write reference frame memory unit 404 to decoded I-frame, and decoded P-frame is write frame memory unit 405.Decoding unit 406A also checks the mistake in the decoding, for example uses Cyclical Redundancy Check.And Notification Validation signal transmitting unit 407 whether success or unsuccessful each frame that receives.

Below, the operation of the 5th embodiment is described.This explanation will concentrate on the operation of moving-picture coding device 300.This is ACK mode or NACK mode and different with its selected mode.For the purpose of simplifying the description, when receiving positive confirmation signal, channel quality is assessed as, and when receiving negative acknowledgement, channel quality is cited as bad, even when channel quality because a frame be evaluated as bad time, working method just changes into the ACK mode from the NACK mode.And when channel quality had been chosen as because of four successive frames, working method changed into NACK from the ACK mode.

The picture coding device 300 that comes into play is worked in the NACK mode, and referring to Figure 17, it is each frame of baseline encoded with the precursor frame.Frame a, b and c are successfully encoded, and appear at frame d and lose.When the NACK signal record received this losing shown in dotted arrow, stoping frame e was that baseline encoded is too late with frame d, but frame f reference frame number is configured to turn back to the latest frame frame e that has just confirmed.Therefore, frame f is that benchmark is encoded with frame c.

Active images decoder 400 can not decoded frame e, but can decoded frame f.Therefore, 400 couples of frame e of active images decoder send a negative acknowledgment, represent with another dotted arrow in Figure 17; Just confirming and frame f is sent one, in Figure 17, representing with solid arrow.

Meanwhile, moving-picture coding device 300 changes into the ACK mode because of the negative acknowledgment of frame d.Therefore, the NACK signal of frame e is left in the basket, reference frame is let alone not change, and frame g is that benchmark is encoded with frame c.

When frame g is encoded, receive the positive confirmation signal of frame f, therefore, next frame h is that benchmark is encoded with the f frame.Same, it is that benchmark is encoded with frame g that the positive confirmation signal that receives for frame g makes frame i.

Another is lost and appears at frame h, because the NACK signal of frame h received before the coding of frame j, so when frame j was encoded, reference frame number was not updated.Frame j can be encoded, and is that benchmark is successfully decoded with frame g.During frame j coding, for frame i receives an ack signal, so next frame k is that benchmark is encoded with frame i.

Also have one to lose and appear at frame k.To make frame m be baseline encoded and successfully decoded with frame j to the NACK signal as a result.Is that benchmark is encoded and successfully decoded and the just affirmation of frame l makes frame n with frame l.

Frame l, m, n and o are all successfully decoded.Their positive confirmation signal makes moving-picture coding device 300 rotate back into the NACK mode later at coded frame q.Therefore, frame r is a baseline encoded with frame q, and frame s is a baseline encoded with frame r.

Under good channel condition, when nothing was lost, the every frame of reference frame number was updated, so, even the transmission delay of big round trip is arranged, also can obtain a good data compression rate.If accidental go out active, because system operates under the NACK mode, and next frame or more following frames just can not be decoded.And the degeneration of the image quality that causes often subjectively is left in the basket, because the image quality that this transmission is made good use of is controlled with long losing arbitrarily at interval.

If the passage situation degenerates, moving-picture coding device 300 changes into the ACK mode from the NACK mode, and beginning only with positive acknowledgement frame as the reference frame.So that a P-frame lose that not hinder other frame decoded.In this mode, system allows to lose continuously and can be because of this frame that causes that long-play can not be decoded of losing.Therefore, image degradation is limited to owing to it loses the inevitable degeneration that causes itself.Under relatively poor passage situation, the 5th embodiment provides the improvement that is approximately one to five decibel of image quality among third and fourth embodiment.Do not do explanation in the prior art in Figure 22.

After the frame losing in ACK or NACK mode, the P-frame of successfully decoding can restart and not wait for next I-frame.

Data compression rate is adapted to the situation of passage sensitively.When channel quality descended, 300 operations of moving-picture coding device increased the ratio of ACK mode time, therefore, and the corresponding decline of data compression rate, as long as and have any time to be spent in the NCK mode, data compression rate will be than better among first and second embodiment.

The advantage of the 5th embodiment is to obtain by removing to assess channel quality with the affirmation signal that exists.Therefore, do not need to send fresh information for the purpose of passage assessment.

Another advantage of the 5th embodiment is the update mode that does not depend on reference frame in the operation of receiver side.Be moving-picture coding device 300 necessary conversion regimes, and need not give receiver side send mode notification signal.Consider rearmost point,, must get strong error detection and correcting measuring and be received to guarantee this signal if the mode notification signal is sent out.Therefore, the 5th embodiment not only will be left the required bandwidth chahnel of send mode notification signal, and will be left mistake and prevent needed bandwidth.

Several variations to the 5th embodiment are illustrated below.

First variation, every frame is divided into some pieces.Each piece provides independently reference frame number.This scheme illustrated in the variation of first embodiment.If this scheme is used for the 5th embodiment, passage can be assessed based on the affirmation signal of the reception of piece in a series of connection frames of same position.

The another kind of variation, in multileaving, confirmation signal receiving element 308A passes to passage assessment unit 315 to each confirmation signal that receives from different destinations, rather than notify simply passage assessment unit 315 about the frame of whether being had a few by the decoding of success.This variation can make passage assessment unit 315 assess channel condition more sensitively.

Also have another to change, the evaluation stage more than 2 passages can be provided, every grade of usefulness is mode of operation independently, and for example very bad level can be added on good and the bad level recited above.When thinking a very bad level, quantizing the size in step in encoding process can increase, to reduce the amount of coded data of every frame, the percentage of the frame that increase can successfully be decoded.

Also have another variation, 400 of active images decoders send ack signal.Can not in the time of plan, receive that ack signal treats as negative acknowledgment.This variation is suitable for the transmission passage with the measurable round trip delivery time.

Other has a kind of variation, and the mode information information that serves as a mark is added on the data that sent by moving-picture coding device 300.Active images decoder 400 can utilize this information to delete unwanted data from frame memory unit 405.For example, in the ACK mode, can be deleted than the data that current reference frame is older.

A kind of variation is arranged again, and when 400 decisions of active images decoder remove conversion regime.From confirmation signal, moving-picture coding device 300 only can determine whether that every frame (or piece) has uncorrectable error, and active images decoder 400 can be determined the recoverable error rate, thereby estimates the passage situation more sensitively.

Also have a kind of variation, the I-frame is directly write reference frame memory unit 305 by decoding unit 303, rather than copies reference frame memory unit 305 by reference frame updating block 309D to from frame memory unit 304.

Other has a kind of variation, and reference frame memory unit 305 and frame memory unit 304 are combined into single memory unit, and reference frame wherein is by pointer management.

A kind of variation is arranged again, and passage assessment unit 315 has a lot of standards in order to assess channel status, and when all these standards had all been thought, this passage was assessed as, and one (or a plurality of) in these standards are chosen as bad time, and this passage is assessed as bad.Other method, when in the standard any one had been chosen as, this state was assessed as.And be assessed as bad time for bad or only when the standard indication of some quantity changes at least, assessment is changed when all standards.

The 6th embodiment

The 6th embodiment is similar with the 5th embodiment, and also available manual operation moving-picture coding device and control mode are changed.Therefore, the structure of encoder is slightly different with the 5th embodiment.The structure of active images decoder is identical with the 5th embodiment with operation, so no longer narration.Figure 18 is the calcspar of moving-picture coding device 600 among the 6th embodiment.For adopting identical reference number with part that equate among the 5th embodiment (Figure 15) or of equal value, these parts will no longer be narrated.

Standard modification unit 317 receives the standard modification order of the negative input of operation, determines suitable new passage evaluation criteria and notifies passage assessment unit 315A from these orders.

Passage assessment unit 315A except having the passage evaluation function described in the 5th embodiment, is applicable to the standard that provided by standard modification unit 317 is provided into the passage evaluation criteria.

Mode is revised the mode of the negative input of unit 318 reception operations and is revised order, and order update mode converting unit 316A converts indicated mode to.

Update mode converting unit 316A except having the function of the update mode converting unit 316 described in the 5th embodiment, is suitable for the conversion regime according to revise order that unit 318 receives from mode.Then, reference frame updating block 309D operates up to for example receiving a reset signal with command mode.So update mode converting unit 316A restarts according to passage situation conversion update mode.

The operation of the 6th embodiment will be described below, and this explanation is limited to the operation of carrying out according to the operator's of moving-picture coding device 800 input.Other operational circumstances is as the 5th embodiment.

The operator of moving-picture coding device 800 upgrades with two kinds of action type control reference frames and handles.

A kind of action type is the state standard that changes evaluated transmission passage.An example of this action type is to change bad evaluating standard, for example at least 3 negative acknowledgments of last 10 frames is become at least 7 negative acknowledgments of last 12 frames.Like this, the operator can influence the reference frame update mode, and still allows the moving-picture coding device to deacclimatize the passage situation of variation simultaneously.

Another kind of action type is to force moving-picture coding device 800 to be operated under the mode of appointment.As ACK mode or NACK mode, and do not consider the passage situation.

Except the advantage with the 5th embodiment, the 6th embodiment carries out selection for the operator directly or indirectly, and therefore control reference frame update mode controls image quality and data compression rate.The permission operator likes according to the individual or takes action according to the passage situation of appointment like this.The 6th embodiment also is used for test and estimates various evaluation criterias and reference frame update mode.

The 7th embodiment is also similar with the 5th embodiment, and it is liked according to moving-picture coding device operator's among the 6th embodiment, gives active images decoder user to select.Slightly different with among the 5th embodiment of the moving-picture coding device here and active images decoder.

Figure 19 has illustrated the structure of moving-picture coding device 900, for the identical reference number that uses with part same among the 6th embodiment as shown in figure 18.Following explanation is limited to different piece between them.

The standard modification unit 317 of standard modification unit 317A in the 6th embodiment.But receive the standard modification signal that sends from active images decoder 1000, rather than from being attached to the order of the input unit input on the moving-picture coding device 900.

Mode is revised unit 318A and is similar among the 6th embodiment mode and revises unit 318.But receive from the mode of active images decoder 1000 transmissions and revise signal, rather than from being attached to the order of the input unit input on the moving-picture coding device 900.

Figure 20 has illustrated the structure of active images decoder 1000, uses with reference number identical among Figure 16 for identical parts, and following explanation is limited to the increase part.

Standard modification signal transmitting unit 410 receives and passes through the standard modification order of input unit 420 by user's input, and sends corresponding standard modification signal to moving-picture coding device 900 by transmitting passage.

Mode is revised signal transmitting unit 411 and is received by input unit 420 and revise order by the mode of user's input similarly, and sends corresponding mode and revise signal to moving-picture coding device 900 by transmitting passage.

The operation of the 7th embodiment will be described below.Explanation will be limited to the performed operation of order according to user's input of active images decoder 1000.

The user can carry out three kinds of action types.A kind of action type is as importing refresh command among first embodiment.Other 2 kinds of action types are that input standard modification order and mode are revised order.These orders are similar among the 6th embodiment the corresponding command by operator's input of moving-picture coding device 900.

Any one type in these three kinds of action types all makes active images decoder 1000 send corresponding signal for moving-picture coding device 900.Refresh signal is forced moving-picture coding device 900 to remove coding and is sent the I-frame.Standard modification and mode are revised signal, and to revise order as standard modification and mode among the 6th embodiment processed.

The 7th embodiment has with the same advantage of the 5th embodiment, and the user who adds to active images decoder 1000 outward has the advantage of more control on image quality.The special advantage of the 7th embodiment is that standard modification order and mode are revised order and can be sent according to the actual image quality of feeling of the user of active images decoder 1000.

In the 6th embodiment, directly the ability of dedicated tunnel evaluation criteria and reference frame update mode is under various criterion and mode, and test and evaluation system performance are useful.

As a kind of variation, the 6th embodiment can combine with the 7th embodiment.Carry out selection control by evaluation criteria and reference frame update mode for the user of 2 ends.

Although for sending the active images data, the present invention is described.It can be used for using any data of basic frame and inter-frame coded system coding.If a frame is divided into the absolute coding piece, the reference frame update method of being invented can be added to every respectively, and is as above said.And word " frame " can be interpreted into one of indication, as a part of image.

Many variations illustrated in the above among the embodiment.Concerning those skilled in the art that, will know within following claim scope of the present invention and still can do further variation.

Claims (73)

1. the encoder (300) in dispensing device is used for a series of frame is encoded, and coded data is sent to decoder (400) in the receiving equipment through transmitting passage, and described encoder comprises:

Be used to receive receiving system (308) from the affirmation signal of described decoder;

Be used for storing the memory device (304,305) of at least one frame of said series of frames;

Reference frame updating device (309), according to said confirmation signal, from said memory device (305), select a reference frame of a coded data of successfully being decoded by described decoder in a plurality of frames of storage:

Determine device (306), which frame that is used to refer in said series of frames is encoded with basic frame, and which frame is encoded with interframe encode;

Be coupled to the code device (302) of said memory device (304,305) and said definite device (306), thereby to realize producing coded data by the basic frame coding and the interframe encode of said definite device indication, interframe encode is that benchmark is realized by the indicated reference frame of said reference frame updating device (309); And

Be coupled to the dispensing device (307) of said code device (302), be used for the reference frame number of said coded data and the described reference frame of indication is sent to said decoder.

2. according to the encoder (300) of claim 1, it is characterized in that said series of frames constitutes active images.

3. according to the encoder (300) of claim 1, it is characterized in that the every frame in the said series of frames is represented an images.

4. according to the encoder (300) of claim 1, it is characterized in that the every frame in the said series of frames is represented the part of an images.

5. according to the encoder (300) of claim 1, it is characterized in that, said encoder sends said coded data to a plurality of decoders (400), described receiving system (308) receives said confirmation signal from said a plurality of decoders (400), the frame that said reference frame updating device (309) is selected just to be confirmed by said at least some decoders (400) is as said reference frame.

6. according to the encoder (300) of claim 1, it is characterized in that, said encoder (300) sends said coded data to a plurality of decoders (400), described receiving system (308) receives said confirmation signal from said a plurality of decoders (400), said reference frame updating device (309) is selected not by a frame of at least some decoder negative acknowledgments of said decoder as said reference frame.

7. according to the encoder (300) of claim 1, it is characterized in that, said confirmation signal comprises indicates the successfully positive confirmation signal of the frame of decoding, and when said each positive confirmation signal was received, the frame that said reference frame updating device (309) is selected to be indicated by one of said positive confirmation signal was as said reference frame.

8. according to the encoder (300) of claim 1, it is characterized in that, said confirmation signal comprises the negative acknowledgement of the frame that indication is not successfully decoded, and when each said negative acknowledgement is received, said reference frame updating device (309) is chosen in frame that the frame by one of said negative acknowledgement indication is encoded before as said reference frame.

9. encoder according to Claim 8 (300), it is characterized in that, described confirmation signal comprises the negative acknowledgement of the reference frame of a frame that expression can not successfully be decoded and an expectation, when receiving each described negative acknowledgement, the frame that described reference frame updating device (309) is selected to be represented by described each negative acknowledgement is as described reference frame.

10. encoder according to Claim 8 (300) is characterized in that, described reference frame updating device (309) comprising:

A table holding unit (314) to preserve an alternative reference frame table, is added to the frame of said coding unit coding in the said alternative reference frame table; And

A delete cells (313), when frame of one of said confirmation signal indication was not successfully decoded, this indicated frame of deletion reached all some newer frames in said series of frames from said alternative reference frame table; And

Wherein said reference frame updating device (309) selects one of frame on the said alternative reference frame table as said reference frame.

11. encoder according to Claim 8 (300), it is characterized in that, described frame of successfully not decoded is not stored in the described storage device (305), and described reference frame updating device (309) selects the frame of a up-to-date storage in the described storage device (305) as the reference frame.

12. encoder according to Claim 8 (300), be characterised in that when receiving each described negative acknowledgement described reference frame updating device (309) is left out frame and all thereafter frames of being represented by described each negative acknowledgement from described storage device (305).

13. encoder (300) according to claim 1, it is characterized in that, said confirmation signal comprises that indication is not by the negative acknowledgement of successful decoded frame and desired reference frame, when each said negative acknowledgement was received, said reference frame updating device (309) was selected by the indicated desired reference frame of one of said negative acknowledgement as said reference frame.

14. encoder (300) according to claim 13, be characterised in that when a described confirmation signal represents that a frame is not successfully decoded described delete cells (313) is left out frame and all newer frames of being represented by described confirmation signal from described storage device (305).

15., be characterised in that the negative acknowledgement of the reference frame of a frame that described confirmation signal comprises that expression can not successfully be decoded and an expectation according to the encoder (300) of claim 14.

16. the encoder (300) according to claim 1 is characterized in that, said reference frame updating device (309) comprising:

A table holding unit (314) to preserve an alternative reference frame table, is added to the frame of said coding unit coding in the said alternative reference frame table; And

A delete cells (313), when frame of one of said confirmation signal indication was not successfully decoded, this indicated frame of deletion reached all some newer frames in said series of frames from said alternative reference frame table; And

Wherein said reference frame updating device (309) selects one of frame on the said alternative reference frame table as said reference frame.

17. the encoder (300) according to claim 16 is characterized in that, said confirmation signal comprises the negative acknowledgement of frame that indication can not be successfully decoded and a desired reference frame.

18. encoder (300) according to claim 16, it is characterized in that, if said frame and said desired reference frame that can not be successfully decoded appear on the said alternative reference frame table, said reference frame updating device (309) selects said desired reference frame as said reference frame.

19。Encoder (300) according to claim 16, it is characterized in that, when the said frame that can not successfully decode is not on said alternative reference frame table the time, said reference frame updating device (309) selects up-to-date frame on the said alternative reference frame table as said reference frame.

20. the encoder (600) according to claim 1 is characterized in that, also comprises:

An apparatus for evaluating (315) is assessed the quality of said transmission passage according to said confirmation signal; And

A mode conversion equipment (316) that is coupled to said apparatus for evaluating (315) is used for controlling said reference frame updating device (309), removes to select reference frame by different way according to said quality.

21. the encoder (600) according to claim 20 is characterized in that: said apparatus for evaluating (315) is translated by the negative acknowledgment frame in the middle of the frame of counting some up-to-date transmission and is estimated said quality.

22. the encoder (600) according to claim 20 is characterized in that, said apparatus for evaluating (315) is assessed said quality by counting continuous negative acknowledgment frame.

23. encoder (600) according to claim 20, it is characterized in that, said these frames are represented the partial images in the active images, and said apparatus for evaluating (315) is assessed said quality by the counting frame number that negativity is confirmed on the same position in some consecutive image.

24. the encoder (600) according to claim 20 is characterized in that, said apparatus for evaluating (315) is assessed said quality according to a lot of standards.

25. the encoder (600) according to claim 20 is characterized in that, said apparatus for evaluating (315) is assessed said quality by the good of formation with bad assessment.

26. encoder (600) according to claim 25, it is characterized in that, when said quality had been chosen as, the frame that said mode conversion equipment (316) the said reference frame updating device of control (309) goes to select up-to-date coding was as said reference frame, unless receive a negative acknowledgement.

27. encoder (600) according to claim 25, it is characterized in that, when said quality is chosen as bad time, said mode conversion equipment (316) the said reference frame updating device of control (309) is only selected by the definite frame of successfully having been decoded of said confirmation signal, as said reference frame.

28. encoder (600) according to claim 20, it is characterized in that, comprise a mode modifier (318) that is used for receiving a mode order, and force said reference frame updating device to remove to select said reference frame by in the mode shown in the said mode order from the user.

29. the encoder (600) according to claim 20 is characterized in that, comprises a standard modification device (318) that is used for revising from user's acceptance criteria order, and according to said standard modification order, revises the quality standard of said apparatus for evaluating assessment.

30. the decoder (400) in the receiving equipment is used for transmitting passage by one and receives coded data from the encoder (300) of realizing basic frame coding and interframe encode to obtain a series of frame, described decoder comprises:

A receiving system (401) is used for receiving described coded data and reference frame number from described object machine (300);

Be connected to a decoding device (406) of described receiving system, be used to by the reference frame that described reference frame number the limits described coded data of decoding; With

Be connected to a confirmation signal dispensing device (407) of described decoding device (406), be used for send confirming the affirmation signal of respective frame to described encoder (300), described confirmation signal comprise described frame number and represent the described decoder described frame of whether successfully having decoded.

31. the encoder (400) according to claim 30 is characterized in that said series of frames is formed active images.

32. the decoder (400) according to claim 30 is characterized in that, said confirmation signal comprises that each frame of expression is by the positive confirmation signal of successfully decoding.

33. the decoder according to claim 30 is characterized in that, said confirmation signal comprises the negative acknowledgement that each frame of expression is not successfully received.

34. the decoder (400) according to claim 30 is characterized in that, said confirmation signal comprises the negative acknowledgement that each frame of expression is not successfully decoded.

35. the decoder (400) according to claim 30 is characterized in that, said confirmation signal comprises the negative acknowledgement that expression corresponding frame positive confirmation signal of successfully having been decoded and corresponding frame are not successfully received.

36. decoder (400) according to claim 30, it is characterized in that, said confirmation signal comprises the corresponding frame of the expression negative acknowledgement of successfully not decoded, and said each negative acknowledgement indicates desired reference frame to be used for said encoder (300) each frame of back is carried out interframe encode.

37. decoder (1000) according to claim 30, it is characterized in that said encoder (900) is according to the quality of the said sendaisle of a certain criterion evaluation, and basis is corresponding to the mode of said quality, for utilizing interframe encode to select reference frame, said decoder (1000) also comprises:

A standard modification sender unit (410) with reception user's input, and sends a corresponding signal for said encoder (900), orders said encoder (900) to revise said standard according to said input.

38. decoder (1000) according to claim 30, it is characterized in that said encoder (900) is according to the quality of the said sendaisle of a certain criterion evaluation, and basis is corresponding to the mode of said quality, for utilizing interframe encode to select reference frame, said decoder (900) also comprises:

A mode is revised sender unit (411), with reception user's input, and sends a signal for said encoder (900), orders said encoder (900) to revise said mode according to said input.

39. the decoder (400) according to claim 30 is characterized in that each frame in the described series of frames is represented an image.

40. the decoder (400) according to claim 30 is characterized in that each frame in the described series of frames is represented the part of an image.

41. one kind is transmitted method that passage send to receiving system with the frame of a series of expression active images by one from dispensing device, comprises following each step:

In described dispensing device, sent to the series of frames in the described receiving system before the storage;

In said series of frames, by in basic frame coding and interframe encode, adjudicating, for every frame is selected a coding method;

Selecting basic frame Methods for Coding coded frame, need be not benchmark with other frame;

Selecting the method coded frame of interframe encode, is benchmark with the reference frame, and said reference frame is the frame that the front has sent;

To send to receiving system from said dispensing device by said frame by interframe encode coded data that produces and the reference frame number of representing reference frame;

On said receiving system, said coded data is decoded with the reference frame that described reference frame number is represented;

Send confirmation signal for said dispensing device from said receiving system, described confirmation signal comprises the frame number of described frame, and represents the described receiving system described frame of whether successfully decoding; And

According to said confirmation signal, on said dispensing device, select said reference frame, to select to be received a frame that device successfully decodes as the reference frame.

42. the method according to claim 41 is characterized in that, each frame in the said series of frames is represented an image.

43. the method according to claim 41 is characterized in that, each frame in the said series of frames is represented the part of an image.

44. the method according to claim 41 is characterized in that, said coded data is sent on a lot of receiving systems, and said confirmation signal is received from said a lot of receiving systems.

45. the method according to claim 44 is characterized in that, the step of said selection reference frame comprises:

Select a frame, this frame is just confirmed by some said receiving system at least.

46. the method according to claim 44 is characterized in that, selects the said step of said reference frame to comprise:

Select a frame, this frame is not also by some said receiving system negative acknowledgment at least.

47. the method according to claim 41 is characterized in that, said confirmation signal comprises successfully the decode positive confirmation signal of each frame of expression, and selects the said step of said reference frame to comprise:

Selection is by a frame of one of said positive confirmation signal expression.

48. the method for claim 41, wherein said confirmation signal comprise the expression negative acknowledgement of each frame of successfully not decoding, and select the said step of said reference frame to comprise:

Be chosen in a frame by coding before the frame of one of said negative acknowledgement expression.

49. the method according to claim 41 is characterized in that, said confirmation signal comprises the negative acknowledgement and the desired reference frame of each frame that expression is not successfully decoded, and the step of said selection reference frame comprises:

Select said desired reference frame as said reference frame.

50. method according to claim 49, it is characterized in that, described confirmation signal comprises the negative acknowledgement of the reference frame of a frame that expression can not successfully be decoded and an expectation, and the step of described selection reference frame comprises: the reference frame of selecting described expectation is as described reference frame.

51. the method according to claim 49 is characterized in that, the step of described selection reference frame is further comprising the steps of:

Preserve an alternative reference frame table;

The frame of said coding unit coding is added in the said alternative reference frame table;

Deletion is by the frame of negative acknowledgment from said alternative reference frame table; And

Select said reference frame from said alternative reference frame table.

52. the method according to claim 49 is characterized in that, and is further comprising the steps of:

Storage is chosen as the described frame of described reference frame at present in a storage device (305) of described dispensing device;

The step of wherein said selection reference frame comprises, does not select the frame of a up-to-date storage in the described storage device (305) as the reference frame when described frame of successfully not decoded is not stored in the described storage device (305).

53. the method according to claim 49 is characterized in that, and is further comprising the steps of:

Storage is chosen as the described frame of described reference frame at present in a storage device (305) of described dispensing device; With

From described storage device (305), leave out by the frame of one of described negative acknowledgement expression with than the newer frame of frame by the expression of one of described negative acknowledgement.

54. the method according to claim 49 is characterized in that, selects the step of said reference frame also to comprise following each step:

Keep an alternative reference frame table;

Being added in the said table with the coded frame of basic frame coding with the coded frame of interframe encode;

Each frame of deletion negative acknowledgment from said table;

From said table, delete some frames simultaneously than negative acknowledgment frame update; And

From said table, select said reference frame.

55. the method according to claim 54 is characterized in that, said confirmation signal comprises the frame that expression can not successfully be decoded and the negative acknowledgement of desired reference frame.

56. the method according to claim 55 is characterized in that, selects the step of said reference frame to comprise:

If the said frame that can not successfully decode and said desired reference frame are on said table, select said desired reference frame.

57. the method according to claim 55 is characterized in that, the step of said selection reference frame comprises:

If the said frame that can not successfully decode is selected the latest frame on the said table not on said table.

58. the method according to claim 57 is characterized in that, and is further comprising the steps of:

Storage is chosen as the described frame of described reference frame at present in a storage device (305) of described dispensing device; With

From described storage device (305), leave out by the frame of negative acknowledgment and than described by the newer frame of the frame of negative acknowledgment.

59. the method according to claim 58 is characterized in that, described confirmation signal comprises the negative acknowledgement of the reference frame of a frame that expression can not successfully be decoded and an expectation.

60. the method according to claim 41 is characterized in that, also comprises the following steps:

Assess the quality of said transmission passage; And

Corresponding to said quality, select said reference frame according to different modes.

61. the method according to claim 60 is characterized in that, said dispensing device is carried out the step of said quality of evaluation according to confirmation signal.

62. the method according to claim 61 is characterized in that, the step of said quality of evaluation comprises: determine interim one, counting negative acknowledgment frame.

63. the method according to claim 61 is characterized in that, the step of said quality of evaluation comprises: count continuous negative acknowledgment frame.

64. the method according to claim 61 is characterized in that, said these frames are represented some visual parts in the said active images, and the step of said quality of evaluation comprises:

The frame of the negative acknowledgment of the same position of counting in some consecutive image.

65. the method according to claim 60 is characterized in that, the step of said quality of evaluation comprises the good and bad assessment of formation.

66. the method according to claim 66 is characterized in that, the step of said selection reference frame comprises:

When said quality evaluation has been, select up-to-date coded frame; Unless receive a negative acknowledgement.

67. the method according to claim 66 is characterized in that, selects the step of said reference frame to comprise:

When said quality evaluation is bad time, determine to select some frames with the affirmation signal of successfully having decoded.

68. the method according to claim 60 is characterized in that, also comprises from the user receiving first order, points out the step of the standard of the said quality that will be used to assess.

69. the method according to claim 68 is characterized in that, said first order receives from the user on said dispensing device.

70. the method according to claim 68 is characterized in that, said first order receives from the user on said receiving system.

71. the method according to claim 60 is characterized in that, also comprises from the user receiving the 2nd order, specifies the step of the mode of selecting said reference frame.

72. the method according to claim 71 is characterized in that, said the 2nd order is on said dispensing device, receives from the user.

73. the method according to claim 71 is characterized in that, said second order receives from the user on said receiving system, and it also comprises next step:

Send said the 2nd order from said receiving system to dispensing device.

Images