JP3810830B2 - Decoding processing method of encoded video signal and decoding apparatus using the same - Google Patents

Description

[0001]
[Industrial application fields]
The present invention relates to a decoding processing method for a video signal encoded for reducing the amount of data and a decoding apparatus using the decoding method.
[0002]
[Prior art]
The digitized video signal has a huge amount of information, particularly when it is a video signal of a moving image. For this reason, if such a video signal is recorded as it is on a recording medium such as a magnetic disk for a long time, a very large storage capacity is required and the cost is increased. Also, when trying to transmit in real time by wire or wireless, the bit rate of video data is high, so a very broadband transmission path is required, and its realization is not easy. In view of this, several encoding methods have been proposed in the past for efficiently encoding video data by signal processing to reduce the amount of data.
[0003]
As such an encoding method, there are MPEG1 ("OPTRONICS"; see 1992 No.5 pp.86-98) and MPEG2 ("Journal of the Television Society"; Vol.48 No.1 pp.44-49). (Hereinafter, when expressed simply as MPEG, both are indicated). In MPEG, in order to reduce the amount of video data, the following methods are mainly used in combination as appropriate.
[0004]
The first method is to calculate the number of bits required for encoding by taking a difference between an image to be encoded and images before and after that (hereinafter referred to as a reference image) and converting the difference into an information signal having a small amplitude. It is a way to reduce.
[0005]
In MPEG, an image unit to be encoded is called a picture. In the case of MPEG1, one picture is composed of one frame of the original image. In the case of MPEG2, one picture is composed of one frame unit or one field unit of the original image, and which unit is used can be selected at the time of encoding.
[0006]
There are three types of pictures, I, P, and B pictures, depending on the image reference method. This is shown in FIG.
[0007]
In the figure, the image that is the start point of the arrow represents the reference image, and the image that is the end point represents the picture being decoded. The I picture does not refer to an image. This is because all the information necessary for decoding is encoded in the picture. The P picture uses the I picture or P picture decoded immediately before as a reference picture. A B picture uses an I or P picture existing immediately before and after it as a reference picture.
[0008]
One picture is divided by a section having a certain size called a macroblock. Furthermore, the macro block is composed of a unit image (hereinafter referred to as a block) of 8 pixels × 8 pixels. For example, when the color video signal has a format of Y: Cb: Cr = 4: 1: 1, as shown in FIG. 14A, four blocks of luminance signals Y0 to Y3, color difference signals Cb and Cr, A macro block is composed of six blocks, one for each block.
[0009]
As shown in FIG. 14B, these macroblocks are given serial numbers called macroblock addresses in order from the upper left corner of the image. Here, for example, when one picture is composed of a frame image of 720 × 480 pixels, there are (720 × 480 pixels) ÷ (8 × 8 × 4) = 1350 macroblocks. . Accordingly, macroblock addresses 0 to 1349 are set in sequential macroblocks.
[0010]
The macro block is a minimum unit for taking a difference from the reference image. The image reference method can be changed in units of macroblocks. For this reason, information indicating this and information indicating a part actually used for reference in the reference image (this is referred to as motion vector information) are encoded in the encoded data of each macroblock.
[0011]
A second method for reducing the amount of data is a method of converting the video data into the spatial frequency domain by performing DCT processing (discrete cosine transform processing) on the video data. In the case of a video signal, generally, the ratio of energy concentration to the low frequency component of the spatial frequency is high, and the high frequency component has a small amplitude. Therefore, it is possible to optimize the allocation of the number of bits by increasing the number of bits allocated to the encoding as the low frequency component and decreasing the high frequency component, thereby reducing the total number of bits. Is possible. In MPEG1 and 2, DCT is performed on a block basis as described above to reduce data.
[0012]
A third method for reducing the amount of data is to send data representing the value and the number of the data instead of repeatedly sending the data of the same value when a plurality of data of the same value are consecutive. This is a method for reducing the amount of data. In MPEG, this method is used for encoding DCT coefficients.
[0013]
A fourth method for reducing the amount of data is variable-length coding in which codes having different lengths are assigned according to the appearance probabilities of the respective values. By assigning a shorter code to a value having a higher appearance frequency, it is possible to reduce the total number of bits. In MPEG, variable length coding is used for coding parameters such as macroblock addresses and motion vectors in addition to DCT coefficients. In each of these, a correspondence table between variable length codes and values to be decoded is determined in advance in the standard.
[0014]
Next, the hierarchical structure of the code will be described.
[0015]
MPEG encoded data has a hierarchical structure called a layer, and as shown in FIG. 15, there are six layers from a sequence layer to a block layer.
[0016]
The sequence layer of the highest layer is a code unit that starts with a sequence header, includes one or more group of picture layers, and ends with a sequence end code. In the sequence header, parameters common to a series of sequence layers such as a picture size and a frame rate are encoded. Note that the header of the sequence layer can be repeatedly inserted into the sequence as needed on the encoding side in preparation for the case where decoding is started in the middle of the sequence layer, such as channel switching.
[0017]
The group of picture layer starts with a group of picture header and includes a plurality of picture layers.
[0018]
This picture layer includes one picture, and includes a plurality of slice layers starting from a picture header. As described above, one picture corresponds to one frame image or one field image. In the picture header, a parameter for distinguishing whether the picture is an I, P, or B picture is encoded.
[0019]
The slice layer starts with a slice header and includes a macroblock layer for each of a plurality of macroblocks. Information indicating the vertical position of the slice on the screen is encoded in the slice header, and information indicating the horizontal position of the slice on the screen is encoded in the macroblock layer immediately after the slice header. ing. When these two pieces of information are combined, the absolute address of the macroblock address can be obtained.
[0020]
The macroblock layer includes blocks constituting the macroblock, and includes a macroblock header and a layer for every six blocks constituting the macroblock shown in FIG. 14A, that is, a block layer. It is out. The macro block header is encoded with the motion vector information and information indicating the macro block address.
[0021]
The block layer of the lowest layer includes a DCT coefficient encoded for each pixel in the block.
[0022]
Among the above layers, the headers of layers higher than the slice layer from the sequence layer to the slice layer start from a code called a start code. The start code is a code having a length of 3 bytes (= 24 bits), and it is prohibited that the same pattern other than the start code appears in the encoded data. Further, when the encoded data is divided from the top in units of 1 byte, the start code is arranged so that the start of the start code starts just from the start (byte alignment). Therefore, even if the continuity of the encoded data is lost due to an error etc., it is easy to find the start code, and use this as a clue to perform normal decoding with at least the header of the slice layer or higher. It is possible to return to processing.
[0023]
By the way, in MPEG, a sequence error code indicating that an error is included in encoded data is prepared. This is, for example, inserted by a transmission medium when an error occurring during transmission of encoded data cannot be corrected. However, the MPEG standard does not define any means for indicating the occurrence of an error other than the sequence error code. Also, there is no provision for distinguishing between various types of errors that can be considered and for evaluating the severity of errors. Furthermore, there is no mention of how to deal with and how to recover when an error actually occurs.
[0024]
[Problems to be solved by the invention]
An error may occur during decoding of the encoded video signal due to some cause such as an error during transmission or a mistake during encoding. When such an error is detected, it is possible to search for the start code described above and use it as a clue to return to normal decoding processing. However, if the severity (error level) of the error that has occurred is not evaluated, and the process always returns to the normal decoding process through the same process, the following problems occur.
[0025]
That is, for example, let us consider a case where a serious error occurs in a system in which recovery is always performed by detecting a start code in a layer above the slice layer after an error occurs. A serious error here is a case where the continuity of encoded data is lost, for example, when the channel is switched to completely different encoded data by channel switching.
[0026]
If important parameters essential for decoding differ before and after channel switching, such as picture size, it is practically difficult to continue decoding further. Nevertheless, since this system has no means for evaluating the error level, it returns to the normal decoding process every time it detects the start code of the slice header of the slice layer that is the lowest layer having the start code. End up. Then, after that, the decoding process is broken and the error process is started again. As a result, significant image deterioration is caused.
[0027]
Further, for example, consider a system that always recovers by detecting a start code in a sequence header after an error occurs. In such a system, even if a serious error as described above occurs, the process does not return to the decoding process until the next sequence header is detected, so that, for example, the previous reference image is continuously displayed until then. If this is done, image distortion can be kept to a minimum.
[0028]
However, in such a system, a problem occurs when a relatively minor error occurs. The minor error here is, for example, a case where several bits of the DCT coefficient are incorrect.
[0029]
In such a case, only the slice containing the error is affected at most by the error. Therefore, if the next slice is restored immediately, the decoding process can be continued normally thereafter. Nevertheless, since this system has no means for evaluating the error level, it does not return until the start code in the next sequence header is detected. Therefore, the return to the normal decoding process is delayed as compared with the case of returning at the next slice, and it becomes impossible to display a normal image for a long time.
[0030]
As described above, if the error level is not evaluated and error detection is returned to normal decoding processing using the same method, the return to the appropriate layer according to the error level is achieved in the minimum necessary time. As a result, an abnormal image may be displayed for a long time.
[0031]
An object of the present invention is to quickly return the decoding operation at an appropriate layer according to the error level, and to reduce the time for displaying an abnormal image due to an error to the minimum necessary time. A decoding processing method and a decoding apparatus using the decoding processing method are provided.
[0032]
[Means for Solving the Problems]
In order to achieve the above object, according to the present invention, when an error is detected at the time of decoding, the degree of the detected error is evaluated to a plurality of levels, and a recovery process from the error is switched based on the evaluation.
[0033]
[Action]
If there is a code error during the decoding of the encoded video signal, when this error is detected, it is evaluated whether the detected error corresponds to a level representing the importance. Depending on the result of this evaluation, the stage of return to normal decoding processing is varied. In other words, when the evaluation level of the detection error is low, the process returns from the lower layer of the encoded video signal, and when the evaluation level of the detection error is high, the process returns from the upper layer.
[0034]
In this way, by changing the return stage to the normal decoding process according to the evaluation level of the detected error, it is possible to promptly return at the appropriate layer according to the error evaluation level.
[0035]
【Example】
Embodiments of the present invention will be described below with reference to the drawings.
In the embodiment described below, encoded data of a video signal conforming to the MPEG described above is assumed as an input signal.
[0036]
FIG. 1 is a block diagram showing an embodiment of a decoding processing method of an encoded video signal and a decoding device using the same according to the present invention, wherein 1 is a decoding device, 2 is an external memory, and 3 is an input terminal. 4 is an output terminal, 5 is an input buffer memory, 6 is a buffer memory for decoding, 7 is a variable length decoding unit, 8 is an IDCT (Inverse Discrete Cosine Transform) unit, 9 is a motion compensation unit, 10 is a display unit, 11 is A memory controller, 12 is a timing unit, and 13 is a data bus.
[0037]
In the figure, encoded data of a video signal conforming to MPEG is input from an input terminal 3 and temporarily stored in an input buffer memory 5. When there is a data request signal from the memory controller 11, the encoded data is read from the input buffer memory 5 and sent to the memory controller 11 via the data bus 13. The memory controller 11 sends the encoded data together with the address and control signal to the external memory 2 and stores the encoded data in a predetermined area of the external memory 2.
[0038]
Further, when there is a data request signal from the decoding buffer memory 6, the memory controller 11 reads the encoded data from the external memory 2 by the address and control signal, and transfers it to the decoding buffer memory 6. The encoded data temporarily stored in the decoding buffer memory 6 is decoded by the variable length decoding unit 7, the IDCT unit 8, and the motion compensation unit 9.
[0039]
The decoded frame or field unit image (decoded image) is supplied to the memory controller 11 via the data bus 13 and is thereby stored in a predetermined area of the external memory 2. On the one hand, the decoded image stored in the external memory 2 is read as a display image, supplied to the display unit 10 via the data bus 13, subjected to predetermined display processing, and output from the output terminal 4. On the other hand, when a reference image is necessary for the decoding process such as a P picture or a B picture, the memory controller 11 encodes the P or B picture to be decoded from the external memory 2 as described above. Data is read out and transferred to the decoding buffer memory 6, and a decoded image is read out from the external memory 2 and transferred as a reference image to the motion compensation unit 9 via the data bus 13.
[0040]
Also, in the case of an encoded video signal conforming to the MPEG standard, due to the peculiarity of the start code pattern described above, if there are more than 5 consecutive bytes of zero in the encoded data, at least one byte is included. The minute is known to be zero data inserted as a dummy. When 5 bytes are found in succession, the input buffer memory 5 uses this fact to output encoded data so as to mechanically remove one byte, and the subsequent stage is output. The processing load in the decryption processing unit is reduced.
[0041]
The decoding buffer memory 6 outputs a data request signal to the memory controller 11 when an empty space is generated therein, and receives and accumulates encoded data transferred from the external memory 2 under the control of the memory controller 11. When a data request signal is received from the variable length decoding unit 7, the data stored therein is output accordingly. However, if no encoded data remains in the external memory 2, a data empty signal is output to notify the variable length decoding unit 7 that the decoding buffer memory 6 is empty.
[0042]
As will be described in detail later, the variable length decoding unit 7 receives the encoded data from the decoding buffer memory 6 and analyzes it, and decodes the above parameters and DCT coefficients in the encoded data. This parameter is sent to each of the decoding processing blocks 8 to 10 and 12 in the decoding device 1 as information necessary for the decoding processing. The DCT coefficient is sent to the IDCT unit 8.
[0043]
The IDCT unit 8 performs inverse DCT processing on the DCT coefficient data received from the variable length decoding unit 7 and sends the IDCT coefficient obtained thereby to the motion compensation unit 9.
[0044]
The motion compensation unit 9 first reproduces a motion vector from the parameters decoded by the variable length decoding unit 7, gives it to the memory controller 11, and performs predetermined decoding from among the decoded images stored in the external memory 2. The converted image is read as a reference image. Then, the reference image and the output of the IDCT unit 8 are added to reproduce the decoded image. The reproduced decoded image is stored in the external memory 2 via the memory controller 11 as a display image or a reference image for decoding the next image.
[0045]
The display unit 10 reads the decoded image from the external memory 2 via the memory controller 11 in accordance with the display timing, converts the decoded image into final image data, and outputs the final image data from the output terminal 4.
[0046]
The timing unit 12 is supplied with a display pixel clock (pelCLK) and a horizontal synchronization signal (Hsync) and a vertical synchronization signal (Vsync) synchronized with the display pixel clock (pelCLK). When these horizontal synchronizing signal (Hsync) and vertical synchronizing signal (Vsync) are not supplied, they are generated internally from the display pixel clock (pelCLK). Then, based on these signals, a timing / control signal is generated and sent to each of the processing blocks 5 to 11, and the processing blocks are controlled to perform decoding processing in cooperation with each other.
[0047]
FIG. 2 is a schematic diagram showing a specific example of the internal configuration of the external memory 2 in FIG.
[0048]
In the figure, the external memory 2 has three frame memory areas 14 to 16 and an encoded data buffer area 17, and each of the frame memory areas 14, 15 and 16 stores a decoded image one by one. Thus, the encoded data from the input buffer memory 5 (FIG. 1) is temporarily stored in the encoded data buffer area 17. Write / read control in these regions 14 to 17 is performed by the memory controller 11.
[0049]
FIG. 3 is a block diagram showing a specific example of such a memory controller 11, wherein 11a is a coded data buffer control unit, 11b is a reference image read control unit, 11c is a decoded image write control unit, and 11d is a decoding unit. The converted image read control unit 11e is a bus width conversion unit.
[0050]
In this figure, the encoded data buffer control unit 11a sends a data request signal to the input buffer memory 5 (FIG. 1) when there is an empty space in the encoded data buffer area 17 (FIG. 2) in the external memory 2. Then, after the encoded data sent in response thereto is processed by the bus conversion unit 11e, an address and a control signal are output to the external memory 2 so as to be stored in the encoded data buffer area 17. When the encoded data buffer control unit 11a receives a data request signal from the decoding buffer memory 6 (FIG. 1), it reads the encoded data from the encoded data buffer area 17 (FIG. 2) accordingly. Then, the data is transferred to the decoding buffer memory 6 after being processed by the bus converter 11e.
[0051]
The reference image reading control unit 11b receives the motion vector signal from the motion compensation unit 9 (FIG. 1), and outputs an address and a control signal to the external memory 2 based on the motion vector signal, and from the frame memory areas 14 to 16 (FIG. 2). A predetermined reference image is read out.
[0052]
The decoded image writing control unit 11c stores the decoded image from the motion compensation unit 9 (FIG. 1) in any of the frame memory areas 14 to 16 (FIG. 2) in the external memory 2. 2 outputs an address and a control signal. Also, the decoded image reading control unit 11d outputs an address and a control signal for reading a desired decoded image from any one of the frame memory areas 14 to 16 in accordance with the display timing.
[0053]
FIG. 4 is a timing chart showing the relationship between the decoding processing timing and the decoded image writing / reading and display timing in the frame memory areas 14 to 16 in the external memory 2, where Vsync is the vertical synchronization signal of the display system. Indicates. Also, here, the encoded video signal shown in FIG. 13 is decoded, and the encoded data has pictures in the order of I1, P5, B2, B3, B4, P9, B6, B7, and B8. Here, of course, I1 is an I picture, P5 and P9 are P pictures, B2 to B4 and B6 to B8 are B pictures, and these numbers indicate the display order.
[0054]
In FIG. 2, in the external memory 2, the frame memory areas 14 and 15 are assigned for writing an I picture or a P picture as a reference image, and the frame memory area 16 is assigned for writing a B picture. Here, it is assumed that each picture is composed of an image for one frame.
[0055]
Decoding processing of the I1 picture is performed in the first one frame period, and the decoded image I1 (hereinafter, the same code as the picture before decoding) is written in the frame memory area. In the next one frame period, the decoded image I1 is read from the frame memory area 14, the P5 picture is decoded using this as a reference image, and the resulting decoded image P5 is written in the frame memory area 15. In the next one frame period, these decoded images I1 and P5 are read out as reference images, and the B2 picture is decoded. The decoded image B2 obtained by this is written in the frame memory area 16. In the same manner, each picture is decoded.
[0056]
Next, reading from the external memory 2 for display will be described.
[0057]
When a decoded picture of a picture is written to the frame memory area 14, 15 or 16, this reading must be performed before a new decoded picture is written to the same frame memory area. . For example, when attention is paid to the B2 picture in FIG. 4, after the B2 picture is decoded and written in the frame memory area 16, the B3 picture is subsequently decoded in the frame memory area 16 in the next one frame period. Will be written to.
[0058]
Here, if the start of reading for display of the decoded image B2 is made to wait until the decoding processing of the B2 picture is completely completed, this reading will decode the next B3 picture and its decoded image B3. Therefore, it is necessary to prepare another frame memory area and store the decoded images B2 and B3 in separate frame memory areas.
[0059]
However, in practice, it is not necessary to wait for the display (reading) until the decoding process (writing) of the B2 picture is completely completed, and it is sufficient to sequentially read the part after the decoding process. In this way, the B picture decoding process and display can be performed without any problem if there is only one frame memory area for the B picture.
[0060]
However, in the encoded video signal, since there is a bias in the amount of code within one picture, there is local unevenness in the progress of the decoding process. Therefore, in this embodiment, as shown in FIG. 4, a margin is taken by starting display (reading) of the decoded image B2 with a delay of one field from the start of decoding processing (writing) of the B2 picture. ing.
[0061]
As described above, in order to perform decoding while minimizing the capacity of the external memory 2 and efficiently using it, it is essential to control the decoding in synchronization with the vertical synchronization signal Vsync of the display system. Therefore, the decoding process ignoring the timing of the vertical synchronization signal Vsync of the display system is not realistic.
[0062]
The timing unit 12 in FIG. 1 controls the decoding process as described above, and generates and outputs a timing / control signal for this purpose. Such timing / control signals will be described with reference to FIG. However, here, it is assumed that the video signal based on the decoded image is a video signal compliant with the NTSC television signal, and Vsync and Hsync in the figure are vertical and horizontal synchronization signals for displaying the decoded image, respectively. is there.
[0063]
The timing / control signal indicates that a control signal (PictStart) indicating the start of decoding processing of a layer higher than the picture layer shown in FIG. 15 and decoding of one macroblock including a slice layer are started. It consists of a control signal (MbStart).
[0064]
When the picture to be decoded corresponds to one frame, as shown in FIG. 5 (1), one control signal (PictStart) and one control signal (MbStart) are the number of macroblocks for each frame as shown in FIG. Are output in a sufficient number for decoding. Similarly, when the picture to be decoded corresponds to one field, as shown in FIG. 5 (2), one control signal (PictStart) and one control signal (MbStart) are provided for each field as shown in FIG. A sufficient number of macroblocks are output to be decoded. 5 (1) and 5 (2) adaptively change depending on whether the picture to be decoded is a frame image or a field image.
[0065]
FIG. 6 is a block diagram showing a specific example of the variable length decoding unit 7 in FIG. 1, in which 7a is a barrel shifter, 7b is a decoding table, 7c is a buffer memory, 7d is a buffer memory controller, and 7e is a start code detection. 7f is a timing controller, and 7g is a decoding controller.
[0066]
In the figure, encoded data from the decoding buffer memory 6 (FIG. 1) is supplied to a barrel shifter 7a. The barrel shifter 7a supplies a part of the fetched encoded data to the decoding table 7b. Since the code to be decoded is generally a variable-length code, the next code data is cued by receiving the code length from the decoding table 7b. When the encoded data in the barrel shifter 7a is used up, a data request signal is sent to the decoding buffer memory 6 (FIG. 1) to request the next encoded data.
[0067]
The decoding table 7b decodes the parameters and DCT coefficients included in the encoded data. As shown in FIG. 7, the decoding table 7b is composed of n tables. Table 1 shows the DCT coefficients. n is for decoding specific codes other than DCT coefficients such as parameters and code lengths. Which one of them is selected is determined by a decoding control signal to be described later.
[0068]
If the code corresponding to the supplied encoded data does not exist in the selected table, it is determined that an error has occurred and an error detection signal is output.
[0069]
In FIG. 6, the code length data of each code obtained from the decoding table 7b is supplied to the barrel shifter 7a. The DCT coefficients of the data output from the decoding table 7b are once stored in the buffer memory 7c. When 64 DCT coefficients corresponding to one block are prepared, the DCT coefficients are output to the IDCT unit 8 (FIG. 1).
[0070]
The buffer memory controller 7d includes a counter therein, counts the number of DCT coefficients output from the decoding table 7b, outputs a buffer memory control signal, and controls the buffer memory 7c as described above. If more than 64 DCT coefficients are sent, it is determined that an error has occurred and an error detection signal is output.
[0071]
The start code detector 7e is supplied with encoded data, and outputs a start code detection signal when a pattern specific to the start code (see FIG. 15) is detected.
[0072]
The timing controller 7f and the decoding controller 7g control the operation of the variable length decoding unit 7 as described above.
[0073]
FIG. 8 is a block diagram showing a specific example of the timing controller 7f, where 7f1 is a decoding timing control unit, 7f2 is an MB address count unit, 7f3 is an MB address extraction unit, and 7f4 is an MB address comparison control unit. .
[0074]
In the figure, the decoding timing control unit 7f1 is supplied with a timing / control signal from the timing unit 12 (FIG. 1) and is suitable for processing blocks in the variable length decoding unit 7 including the decoding controller 7g (FIG. 6). A decoding timing control signal that gives an instruction to start and stop the operation is generated and output at an appropriate timing. That is, the start of the picture layer is instructed by the control signal (PictStart) of the timing / control signal, and the start of the macroblock is instructed by the control signal (MbStart), and this decoding timing control signal is generated.
[0075]
The MB address counting unit 7f2 is cleared by the control signal (PictStart) of the timing / control signals, and counts the control signal (MbStart), thereby generating a macroblock address at the decoding timing. On the other hand, the MB address extracting unit 7f3 extracts and reproduces the macroblock address on the encoded data based on the parameters obtained as a result of analyzing the encoded data in the decoding table 7b (FIG. 6). The decoding process is performed using the timing / control signal as a timing reference. In a normal decoding process, what is the control signal (MbStart) of the timing / control signal from the control signal (PicStart)? The macroblock with the determined macroblock address is processed. For this reason, when the decoding process is normally performed, these two macroblock addresses should match. Therefore, the MB address comparison control unit 7f4 compares the both, and if the latter (ie, the macroblock address extracted from the encoded data) is delayed, it is determined that an error has occurred and an error detection signal is sent. Output. If the latter has progressed too much, a decoding timing control signal for temporarily suspending the decoding process of this macroblock is output until it matches.
[0076]
FIG. 9 is a block diagram showing a specific example of the decoding controller 7g in FIG. 6, in which 7g1 is a decoding state counter, 7g2 is a branch destination calculation unit, 7g3 is an error detector, and 7g4 is an error level counter.
[0077]
In the encoded data, codes defined in advance in the encoding standard are arranged in an order determined. However, in this order, not only the codes are simply arranged, but the arrangement order and presence of specific codes change depending on conditions. Data representing the condition itself is also encoded in the encoded data.
[0078]
In order to appropriately decode the encoded data having such properties, in this embodiment, a decoding state counter 7g1 is provided in the decoding controller 7g as shown in FIG. It manages and controls the state of the decryption process including The value held in the decoding state counter 7g1 indicates the current decoding state, and it is simultaneously used for controlling each processing block (FIG. 6) in the variable length decoding unit 7 as a decoding control signal.
[0079]
The branch destination calculation unit 7g2 determines the next decoding state based on the current decoding state indicated by the decoding control signal and the parameter value decoded up to that point, and instructs this to the decoding state counter 7g1. In this way, the codes can be analyzed in an appropriate order based on the condition data included in the encoded data.
[0080]
When an error detection signal is received from the decoding table 7b (FIG. 7), the buffer memory 7d, the timing controller 7f (FIGS. 6 and 8), or an error detector 7g3 described later, the process branches to a routine for error processing. The decoding state counter 7g1 is instructed to do so. Further, when a start code is detected at a position other than the position determined in the encoded data standard, it is determined as an error and the process is branched to an error processing routine.
[0081]
The error detector 7g3 checks the value of the parameter decoded in the decoding table 7b (FIG. 6), and checks whether it is within a predetermined range. If not, it is determined that an error has occurred and an error detection signal is output. Even when a sequence error code is detected in the encoded data, it is determined that an error has occurred and an error detection signal is output.
[0082]
When the decoding controller 7g branches to an error processing routine due to error detection, the error level counter 7g4 determines the error level based on the decoding control signal from the decoding state counter 7g1. An error level signal representing the determination result is distributed to each processing block in the decoding apparatus 1.
[0083]
Next, processing when an error occurs in the decoding controller 7g will be described in more detail.
[0084]
As described above, when an error is detected by any of the error detection methods, the process branches to an error processing routine in accordance with an instruction from the branch destination calculation unit 7g2. A specific example of this error processing routine will be described with reference to FIG.
[0085]
First, when an error is detected (step 101), the error level is set to 1 (step 102). Specifically, the error level counter 7g4 (FIG. 9) detects the decoding state from the decoding control signal from the decoding state counter 7g1 (FIG. 6), and sets the error level to 1 (hereinafter, the error level is set). The same mechanism is used).
[0086]
Next, it is checked whether or not the control signal (PictStart) (this is regarded as a vertical synchronizing pulse for decoding) is detected at this time (step 103), and if it is detected, the error level is set to 2 It is changed (step 106), and if not detected, it is checked whether or not a start code that causes a return to normal decoding processing has been detected (step 104). If a start code is detected, that is, if a start code is detected before the decoding vertical sync signal (PictStart) is detected, this error has no significant influence and is normally decoded as having low significance. Return processing is performed (step 113).
[0087]
When neither the decoding vertical synchronization signal (PictStart) nor the start code is detected, the process returns to step 103, and the operations of steps 103 and 104 are repeated until either the decoding vertical synchronization signal (PictStart) or the start code is detected.
[0088]
The return processing routine (step 113) is a routine for returning at a level higher than the slice layer. As described above, when the error level is 1, the layer having the start code in the hierarchical structure of the encoded data can be restored with the slice that is the lowest layer.
[0089]
If the decoding vertical synchronization signal (PictStart) is detected before the start code is detected, a situation such as the movement of another picture during the error occurs, which is more serious than an error level 1 error. It becomes. For this reason, for such an error, the error level is set to 2 (step 106), and the decoding vertical synchronization signal (PictStart) or the start code is detected as in the previous steps 103 and 104. Is repeated (steps 107 and 108).
[0090]
In this case, if a start code is detected prior to the decoding vertical synchronization signal (PictStart), it is further checked whether it is greater than or equal to the picture header (step 109). (Step 114), and if it is a layer lower than the picture header, that is, a slice header, the process returns to step 107 again.
[0091]
In this way, when the error level is 2, it is possible to return from the start of the hierarchy higher than the picture layer.
[0092]
Further, when the decoding vertical synchronization signal (PictStart) is detected prior to the start code (step 107), the most serious error that does not even detect the start of the picture has occurred. For an error, the error level is set to 3 (step 110). Then, it waits until a start code is detected (step 111). When a start code is detected, it is checked whether it is greater than or equal to the sequence header (step 112). If so, a return process is performed (step 115). If the header is a layer lower than the sequence layer, the process returns to step 111 again.
[0093]
In this way, when the error level is 3, it is possible to recover only at the sequence layer or higher.
[0094]
The error level signal obtained as described above is distributed to each block shown in FIG.
[0095]
Next, error concealment processing when an error occurs will be described.
[0096]
In FIG. 1, when the error level is 1, the motion compensation unit 9 uses the reference image data immediately before the same position as the currently processed macroblock, instead of decoding the macroblock image by normal processing. Use to hide this macroblock error. For this purpose, a zero motion vector is given to the memory controller 11 to read the reference image. In this way, in the case of level 1, which is the smallest error, error concealment is performed in units of macroblocks, so that even when the normal decoding process is resumed, the effect of the error on the screen is minimal. It will be over.
[0097]
When the error level is 2, the display unit 10 hides the error by displaying the immediately preceding decoded picture (reference image) instead of displaying the picture currently being decoded. For this purpose, the signal specifying the display image to be sent to the memory controller 11 is switched so as to specify the immediately preceding reference image when an error occurs. In this way, in the case of error level 2, error concealment can be performed in units of pictures.
[0098]
If the error level is 3, it is determined that it is the most serious error and it is difficult to continue decoding further, and once the input buffer memory 5, the decoding buffer memory 6, and the memory 2 The encoded data buffer area 17 (FIG. 2) is cleared. The encoded data buffer area 17 is cleared by sending an error level signal having an error level of 3 to the encoded data buffer controller 11a (FIG. 3) in the memory controller 11. Thereafter, decoding is newly performed on the encoded data newly input from the input terminal 3. In addition, the display unit 10 continues to display the previous picture (reference image) that has already been decoded until the variable length decoding unit 7 finds the sequence header and returns to normal decoding processing.
[0099]
In this way, in this embodiment, the error level is evaluated in a plurality of stages according to the time from the error to return, and thereby the process of returning according to each error level is made different. Is possible. In addition, since appropriate error concealment according to each error level can be performed, it is possible to minimize the time during which an image is disturbed due to an error.
[0100]
FIG. 11 is a block diagram showing another specific example of the decoding controller 7g in FIG. 6. 7g5 is an error counter, and the same reference numerals are given to the portions corresponding to those in FIG. To do.
[0101]
In this figure, this specific example is obtained by adding an error count counter 7g5 to the configuration shown in FIG. 9, and this error count counter 7g5 receives an error detection signal from the timing controller 7f (FIG. 6). The counter value is incremented by 1, the number of errors is counted, and the counted value is given to the branch destination calculation unit 7g2.
[0102]
Next, a specific example of the error processing routine in the decoding controller 7g shown in FIG. 11 will be described with reference to FIG.
[0103]
This error processing routine differs from that shown in FIG. 10 in that the error level is not evaluated by a signal based on the time of the decoding vertical synchronization signal (PictStart), but the number of times errors have occurred in the past. It is in the point made to do in.
[0104]
First, when an error is detected by any of the error detection methods, the count value of the error number counter 7g5 is increased by 1, and the process branches to an error processing routine (step 201). Here, the number of times an error has occurred in the past is checked, and if it is once (that is, this time for the first time), the error level is set to 1 (step 203), and the process waits until a start code is detected (step 203) Step 204). When the start code is detected, it is reflected in the return processing routine (step 213).
[0105]
This return processing routine (step 213) is a routine for returning at the slice layer or higher, and when the error level is 1, the layer having the start code in the hierarchical structure of the encoded data is the lowest layer. Recovery can be achieved by slicing.
[0106]
On the other hand, when the number of error occurrences is greater than 1 (step 202) and the number of past error occurrences is 2 (ie, this time is the second) (step 206), the error level is set to 2 (step 207), and waits until a start code is detected (step 208). When the start code is detected, it is checked whether or not it is greater than or equal to the picture header (step 209). If so, the process proceeds to return processing (step 214), and a layer lower than the picture header, that is, a slice header. If so, the process returns to step 208 again. This is different from when the error level is 1.
[0107]
In this way, when the error level is 2, it is possible to recover only at the picture layer or higher.
[0108]
If the number of error occurrences is greater than 2 (step 202), the error level is set to 3 (step 210) and the process waits until a start code is detected (step 211). If a start code is detected, it is further checked whether it is greater than or equal to the sequence header (step 212). If so, the process proceeds to return processing (step 215), and the header of the layer lower than the sequence layer If so, the process returns to step 211 again.
[0109]
In this way, when the error level is 3, it is possible to recover only at the sequence layer or higher.
[0110]
Note that the error concealment process corresponding to the error level is the same as that described above.
[0111]
As described above, in this specific example, the error level is evaluated in a plurality of stages based on the number of times an error has occurred, and the process of returning can be made different according to each error level. In addition, since appropriate error concealment according to each error level can be performed, it is possible to minimize the time during which an image is disturbed due to an error.
[0112]
As mentioned above, although the Example of this invention was described, this invention is not limited only to this Example.
[0113]
That is, in the above embodiment, the input encoded data conforms to the MPEG, but is not limited thereto, and may be data encoded by another encoding method having the same properties.
[0114]
In FIG. 4, the relationship between the decoding processing in the above embodiment and the use and display timing of the frame memory areas 14 to 16 provided in the memory 2 is composed of one picture for each picture. Although the case has been described as an example, the present invention is not limited to this, and the same applies to the case where each picture is composed of an image for one field.
[0115]
Further, in FIG. 5, the control signal from the timing unit 12 (FIG. 1) has been described by taking the NTSC television signal as an example. However, the present invention is not limited to this, and the same applies to television signals of other systems. is there.
[0116]
Furthermore, in the above embodiment, the error level is evaluated in three stages. However, the error level is not limited to this, and may be further finely or conversely divided in two stages, and may be changed as necessary. .
[0117]
Furthermore, in the error processing routine shown in FIG. 10, the decoding vertical synchronization signal (PictStart) is used as a signal for evaluating the error level. However, the present invention is not limited to this. (Vsync) itself or other signals closely related to this may be used.
[0118]
Furthermore, in the above embodiment, when the error level is 2 or more, the already decoded picture (reference image) is displayed. However, the present invention is not limited to this. For example, a plain image or other image is displayed. An arbitrary image may be displayed.
[0119]
【The invention's effect】
As described above, according to the present invention, the error level is evaluated in a plurality of stages according to the time until the error is recovered or the number of times the error has occurred, and the process of returning according to each error level is different. Since it is possible to perform appropriate error concealment according to each error level, it is possible to minimize the time during which an image is disturbed by the influence of an error.
[Brief description of the drawings]
FIG. 1 is a block diagram showing an embodiment of a decoding processing method of an encoded video signal and a decoding apparatus using the same according to the present invention.
2 is a schematic diagram showing a specific example of an internal configuration of a memory in FIG. 1. FIG.
FIG. 3 is a block diagram showing a specific example of the memory controller in FIG. 1;
4 is a timing chart showing the relationship between the decoding processing timing in the embodiment shown in FIG. 1 and the writing / reading and display timing of a decoded image in a frame memory area in the memory. FIG.
FIG. 5 is a diagram showing a specific example of a control signal output from the timing unit in FIG. 1;
6 is a block diagram showing a specific example of the variable-length decoding unit in FIG. 1. FIG.
7 is a block diagram showing a specific example of a decoding table in FIG. 6. FIG.
8 is a block diagram showing a specific example of the timing controller in FIG. 6. FIG.
9 is a block diagram showing a specific example of a decoding controller in FIG. 6. FIG.
FIG. 10 is a flowchart showing a specific example of an error processing routine instructed by the decoding controller shown in FIG. 9;
11 is a block diagram showing another specific example of the decoding controller in FIG. 6. FIG.
12 is a flowchart showing a specific example of an error processing routine instructed by the decoding controller shown in FIG.
FIG. 13 is a diagram illustrating an example of an encoded code according to MPEG.
14 is a diagram showing a configuration of a macroblock in the encoded data shown in FIG.
FIG. 15 is a diagram illustrating a hierarchical structure of layers in MPEG code data.
[Explanation of symbols]
1 Decryption device
2 memory
3 Input terminal
4 output terminals
5 Input buffer memory
6 Decoding buffer memory
7 Variable length decoding unit
8 IDCT unit
9 Motion compensation unit
10 Display unit
11 Memory controller
12 Timing unit

Claims (14)

In a decoding processing method of a video signal encoded by an encoding method having a hierarchical structure starting with a different start code for each layer,
A start code search step that enables starting or restarting decoding from a predetermined hierarchy by searching for a specific start code;
A decoding step of decoding the encoded video signal;
An error detection step for detecting an error that occurs because normal decryption processing is not performed;
An error evaluation step for evaluating the degree of the detected error to a plurality of levels according to the elapsed time from the detection of the error or the number of errors further detected after the detection of the error;
Depending on the error evaluation levels the evaluation, the more the error evaluation level is low, return a more decoding from a lower layer encoded signal, the more the error evaluation level is higher, a higher encoded signal A decoding step is resumed from a different layer by the start code search step so that the decoding process is restored from the layer of the original layer, thereby returning to a normal decoding process. A method of decoding an encoded video signal. In claim 1,
The error detection step detects the occurrence of the error by detecting a specific code pattern inserted in the encoded sequence of the encoded video signal. Method. In claim 1,
The error detecting step detects the occurrence of the error by detecting a code other than a code predefined as a normal code from a coded sequence of the coded video signal. Video signal decoding processing method. In claim 1,
The error detection step detects the occurrence of the error by detecting that the number of specific codes existing in the encoded video signal is outside a predetermined range. Video signal decoding processing method. In claim 1,
The decoding step is to decode a specific code present in the encoded video signal using a different table for each type of code,
The error detection step detects the occurrence of the error by detecting that a code not defined in the table is input in at least one of the tables. Decryption processing method. In claim 1,
The error detection step detects the occurrence of the error by detecting that a value of a specific parameter obtained by decoding the encoded video signal is outside a predefined range. A method of decoding an encoded video signal characterized by In claim 1,
The decoding step is controlled by a pulse signal having a predetermined period synchronized with a display timing of a decoding process of the encoded video signal,
When a plurality of error evaluation levels set by the error evaluation step are expressed as n (n = 1, 2, 3,...),
If an error is detected by the error detection step and the pulse signal is detected before returning to normal decoding processing by the return step in a state where the error evaluation level of the error is set to n, the error evaluation is performed. A method of decoding an encoded video signal, wherein the level is reset to n + 1. In a decoding device for a video signal encoded by an encoding method having a hierarchical structure starting with a different start code for each layer,
Start code search means for enabling decoding start or restart from a predetermined hierarchy by searching for a specific start code;
Decoding means for decoding the encoded video signal;
Error detection means for detecting an error that has occurred because normal decryption processing is not performed;
Error evaluation means for evaluating the degree of the detected error to a plurality of levels according to the elapsed time from the detection of the error or the number of errors further detected after the detection of the error;
Depending on the error evaluation levels the evaluation, the more the error evaluation level is low, return a more decoding from a lower layer encoded signal, the more the error evaluation level is higher, a higher encoded signal And a return means for restarting the decryption process from a different layer by the start code search means so as to return the decryption process to the normal decryption process. An apparatus for decoding an encoded video signal. In claim 8 ,
The decoding apparatus for an encoded video signal, wherein the error detection unit detects the occurrence of the error by detecting a specific code pattern inserted in a code string of the encoded video signal. In claim 8 ,
The error detecting means detects the occurrence of the error by detecting a code other than a code predefined as a normal code from a coded sequence of the coded video signal. Video signal decoding device. In claim 8 ,
The error detection means detects the occurrence of the error by detecting that the number of specific codes existing in the encoded video signal is outside a predetermined range. Video signal decoding device. In claim 8 ,
The decoding means is for decoding a specific code present in the encoded video signal using a different table for each type of code,
The error detection means detects the occurrence of the error by detecting that a code not defined therein is input in at least one of the tables. Decryption device. In claim 8 ,
The error detection means detects the occurrence of the error by detecting that a value of a specific parameter obtained by decoding the encoded video signal is outside a predefined range. An apparatus for decoding an encoded video signal characterized by the above. In claim 8,
The decoding means is controlled by a pulse signal having a predetermined period synchronized with the display timing of the decoding process of the encoded video signal,
When a plurality of error evaluation levels set by the error evaluation means are expressed as n (n = 1, 2, 3,...),
If an error is detected by the error detection means and the pulse signal is detected before returning to normal decoding processing by the return means in a state where the error evaluation level of the error is set to n, the error evaluation An apparatus for decoding an encoded video signal , wherein the level is reset to n + 1.

Images