JP2001298736A - Information processing unit and method, learning unit and method, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing unit that can generate pixels with higher quality independently of whether or not pixels are missing. SOLUTION: A class synthesis circuit 507 classifies target data to any of classes preset in advance on the basis of a plurality of data around the target data. A coefficient storage class code selection circuit 509 stores conversion information with respect to the target data corresponding to each class. An estimate prediction arithmetic circuit 510 converts the target data to data with higher quality on the basis of the conversion information. A class synthesis circuit 507 classifies the target data to different classes depending on whether or not the target data are missing.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to an information processing apparatus and method, a learning apparatus and method, and a recording medium.
In particular, the present invention relates to an information processing apparatus and method for creating data, a learning apparatus and method, and a recording medium.

[0002]

2. Description of the Related Art In many compression systems for data transmission, variable-length coding represented by entropy coding is used to improve a compression ratio. The feature of this entropy coding is that the compression is realized by changing the length of the code word according to the content of the data to be compressed. For example, data with a high frequency of occurrence is assigned a short codeword, and data with a low frequency of occurrence is assigned a long codeword. As a result, the information amount of the entire data to be transmitted is reduced.

When erroneous data is received on the receiving side due to a communication error, the length of the code word always changes according to the contents of the data. This phenomenon is generally referred to as “error propagation”.

Further, MPEG (Moving Picture Experts Grou)
The compression method of moving images represented by p) reduces the amount of data by reducing temporal redundancy by, for example, transmitting only the differences between frames, by reducing the spatial redundancy by discrete cosine transform. Reduce.

[0005] The data of a moving image compressed by such a compression method is decompressed on the receiving side, but if data of the transmitted moving image is lost due to a communication error or the like, the loss of the data is spatial and The effect is more pronounced as the compression ratio is expanded in time and the compression ratio is higher.

[0006] The condition of the communication path in the mobile environment is inferior to that of a fixed station or a wire line, and burst-like data loss called "packet loss" occurs relatively frequently. Therefore, in communication of an image using compression in such a mobile environment, loss of data on a communication path is prevented by using a strong error correction code.

[0007] Pixels that cannot be corrected by the error correction code and are missing are analyzed for correlation with surrounding pixels and linearly interpolated using pixels in the direction of strong correlation, or simply in the direction of strong correlation. Replaced with pixel. Alternatively, a missing pixel is detected by detecting a motion amount or a motion vector of a peripheral pixel, and is linearly interpolated using a pixel in a direction in which a motion element has a strong correlation. Alternatively, the missing pixels are concealed using a median filter.

[0008]

However, the pixels thus linearly interpolated or replaced are:
Since the resolution is reduced and continuity with surrounding pixels is lost,
The reconstructed image remained uncomfortable. When a median filter is used, a sense of incongruity also remains.

[0009] The present invention has been made in view of such a situation, and has as its object to generate higher quality pixels regardless of whether or not pixels are missing. .

[0010]

According to a first aspect of the present invention, there is provided an information processing apparatus for classifying target data into one of a plurality of preset classes based on a plurality of data around the target data. Means, selecting means for selecting conversion information corresponding to the classified class, and conversion means for converting the data of interest into higher-quality data based on the conversion information. The data of interest is classified into different classes depending on whether or not it is missing.

The conversion information is information for generating the data of interest based on a plurality of peripheral data for a missing class in which the data of interest is missing, and converting the information to a non-missing class in which the data of interest is not missing. On the other hand, it can be used as information for converting attention data into higher quality data.

The conversion information can be information for converting the data of interest into data with reduced noise for the non-missing class.

The conversion information may be information obtained by learning in advance.

The conversion information can be a linear or non-linear, or a prediction coefficient used for a first-order or multi-order estimation formula.

The classifying means can classify the data of interest into one of a plurality of preset classes based on a plurality of class taps around the data of interest.

The conversion means can convert the target data into higher quality data based on the prediction taps corresponding to the classified classes.

The class classification means can classify the data of interest into one of a plurality of preset classes based on a plurality of data spatially surrounding the data of interest.

The class classification means can classify the data of interest into one of a plurality of preset classes based on a plurality of data temporally surrounding the data of interest.

The data of interest can be image data.

In the information processing method according to the present invention, a class classification step of classifying the data of interest into one of a plurality of preset classes based on a plurality of data around the data of interest. A selection step of selecting conversion information corresponding to the class, and a conversion step of converting the data of interest into higher-quality data based on the conversion information, and in the class classification step, whether or not the data of interest is missing , The attention data is classified into different classes.

The conversion information is information for generating a target data based on a plurality of peripheral data for a missing class in which the target data is missing, and is used for a non-missing class in which the target data is not missing. On the other hand, it can be used as information for converting attention data into higher quality data.

The conversion information may be information for converting the data of interest into data with reduced noise for the non-missing class.

The conversion information can be information obtained by learning in advance.

The conversion information can be a prediction coefficient used for a linear or non-linear, or a first- or multi-order estimation formula.

In the class classification step, the data of interest can be classified into one of a plurality of preset classes based on a class tap which is a plurality of data around the data of interest.

In the conversion step, the data of interest can be converted to higher quality data based on the prediction taps corresponding to the classified classes.

In the class classification step, the data of interest can be classified into one of a plurality of preset classes based on a plurality of data spatially surrounding the data of interest.

In the class classification step, the data of interest can be classified into one of a plurality of preset classes based on a plurality of data temporally surrounding the data of interest.

The data of interest can be image data.

According to a twenty-first aspect of the present invention, a program for a recording medium includes a class classification step of classifying target data into one of a plurality of preset classes based on a plurality of data around the target data. A selecting step of selecting conversion information corresponding to the selected class; and a conversion step of converting the data of interest into higher-quality data based on the conversion information. The feature is that the data of interest is classified into different classes depending on the above.

A learning device according to a twenty-second aspect of the present invention provides a classifying means for classifying target data into one of a plurality of preset classes based on a plurality of data around the target data, and a class corresponding to the class. Conversion information generating means for generating conversion information for converting the data of interest into higher quality data, wherein the class classification means classifies the data of interest into different classes according to whether or not the data of interest is missing. It is characterized by doing.

A learning method according to a twenty-third aspect of the present invention corresponds to a class classification step of classifying target data into one of a plurality of preset classes based on a plurality of data around the target data. A conversion information generating step of generating conversion information for converting the data of interest into higher quality data. In the class classification step, the data of interest is classified into different classes depending on whether or not the data of interest is missing. It is characterized by being performed.

An information processing apparatus according to claim 1,
In the information processing method according to the first aspect and the recording medium according to the first aspect, the attention data is classified into one of a plurality of preset classes based on a plurality of data around the attention data. The conversion information corresponding to the classified class is selected, the attention data is converted to higher quality data based on the conversion information, and the attention data is classified into a different class depending on whether or not the attention data is missing in the class classification. being classified.

[0034] In the learning apparatus according to the twenty-second aspect and the learning method according to the twenty-third aspect, the attention data is classified into one of a plurality of preset classes based on a plurality of data around the attention data. In accordance with the class, conversion information for converting the attention data into higher quality data is generated, and in the class classification, the attention data is classified into different classes depending on whether or not the attention data is missing. .

[0035]

FIG. 1 is a diagram for explaining an image communication system for communicating images according to the present invention. Transmission device 1
Compresses the input video signal by the method according to the present invention, encodes it, converts it into a packet format, and transmits it to the receiving device 2 via a transmission path. The receiving device 2 receives the packet transmitted via the transmission path, decodes the data included in the packet by the method according to the present invention, decompresses the data, and outputs it as a video signal. In a transmission path, data included in a packet may be lost due to congestion or the like.
Further, when the processing capacity of an ATM switch (not shown) constituting the transmission path is exceeded, the packet itself may be lost.

FIG. 2 is a block diagram showing a configuration of one embodiment of the transmitting apparatus 1 according to the present invention. The Y / C separation chroma decoder 11 outputs an analog composite video signal,
Alternatively, when any one of the analog signal (Y / C signal) such as the luminance signal Y and the chroma signal C is input, the signal is converted into the luminance signal Y and the color signals U and V.
Into a component signal (Y / U / V signal) as shown in FIG. The selector 12 has Y / C
One of the Y / U / V signal supplied from the separation chroma decoder 11 and the signal input in the Y / U / V signal format from the beginning is selected and supplied to the thinning unit 13. The Y / U / V signal has, for example, a standard density (SD) of 4: 2: 2 and a field frequency of 60H.
z and interlaced format signals.

The thinning section 13 thins out the Y signal, U signal, and V signal supplied from the selector 12 by a method described later, and supplies the thinned signal to the format conversion circuit 14. The thinning unit 13 includes a pre-filter 41, an A / D conversion circuit 42, a thinning filter 43, an external memory 44, a UV
A vertical half circuit 45 and an external memory 46 are provided.

The pre-filter 41 receives the input Y signal,
This is a low-pass filter that outputs only a predetermined frequency band of each of the U signal and the V signal. Prefilter 4
The output of 1 is supplied to the A / D conversion circuit 42.

The A / D conversion circuit 42 samples the input Y signal, U signal, and V signal, and converts each of them into, for example, 8-bit data. The analog Y signal input to the A / D conversion circuit 42 is sampled to be converted into Y data including 528 pixels in the horizontal direction and 480 pixels in the vertical direction, thereby obtaining a thinning filter 43.
Is output to

The analog U signal input to the A / D conversion circuit 42 is sampled and output to the thinning filter 43 as U data including 176 pixels in the horizontal direction and 480 pixels in the vertical direction. The analog V signal input to the A / D conversion circuit 42 is sampled so that the thinning filter 43 is converted into V data including 176 pixels in the horizontal direction and 480 pixels in the vertical direction.
Is output to

Therefore, the output of the A / D conversion circuit 42 is
The signal has a format of 3: 1: 1.

The thinning filter 43 temporarily stores the input Y data, U data and V data in the external memory 4.
4 and temporarily store the Y data, U data, and V data, which are thinned to 水平 in the horizontal direction and １ ／ in the vertical direction, respectively.
5

The UV vertical half circuit 45 temporarily stores the Y data, U data, and V data supplied from the thinning filter 43 in the external memory 46, and temporarily stores the U data and V data. Are respectively thinned in the vertical direction by ２, and the Y data, U data, and V data are output to the format conversion circuit 14.

Therefore, the signal output from the thinning section 13 is a signal of 3: 0.5: 0.5.

The format conversion circuit 14 includes the thinning unit 1
3, the Y data, U data, and V data supplied from
It is supplied to the encoding circuit 15.

The encoding circuit 15 temporarily stores the Y data, U data, and V data supplied from the format conversion circuit 14 in the external memory 16.
6 is shuffled by changing the storage and read addresses of the Y data, U data, and V data, and ADRC (Adaptive Dynamic Range Codin) is performed.
g) Encode by the method, further shuffle, and output to the packetizing circuit 17.

The ADRC system is a variable length coding system for outputting a variable length code adapted to a dynamic range (DR) of input image data. Image data is
The block is divided into blocks (ADRC blocks to be described later) including a plurality of pixels, and a dynamic range that is a difference between the maximum value and the minimum value of the pixel values included in each block is detected. Based on the dynamic range of the block, each pixel value is requantized with a smaller number of bits than the original number of quantization bits (for example, 8 bits). As the dynamic range is smaller, requantization can be performed with a smaller number of bits, and it is possible to further reduce the amount of data by removing only the redundancy of the pixel value of the pixel while suppressing an increase in quantization distortion. is there.

In the ADRC method, the number of quantization bits is selected in relation to the dynamic range, which is the difference between the maximum value and the minimum value of the pixel values included in a block. In order to determine the magnitude relationship of the dynamic range,
A threshold value corresponding to movement or the like is used. For example, when assigning any one of 2 bits, 3 bits, or 4 bits as the number of bits for requantization, thresholds T1 and T2 stored in a quantization table for each motion or dynamic range (however, T1 <T2) is used. The same quantization table is used on the receiving side.

In a block having a dynamic range of (T1-1) or less, two bits are assigned to a code corresponding to a pixel value. In a block whose dynamic range is equal to or more than T1 and equal to or less than (T2-1), three bits are assigned to a code corresponding to a pixel value. In a block having a dynamic range of T2 or more, 4 bits are assigned to a code for a pixel value. Let q be the number of bits assigned to the code for the pixel value.

The encoding is performed in units of 88 ADRC blocks generated by dividing two frames into 30 units (this unit is called a buffer). Here, one quantization table storing the threshold values T1 and T2 is selected for one buffer so that the code generated for one buffer is 16104 bits or less. The selected quantization table is specified by a table index indicated by TI. By making the code generated for one buffer less than or equal to 16,104 bits, the information of the encoded image data can be changed to 8 Mbp.
s.

When the dynamic range is greater than 2 to the qth power, the code Q for the pixel value is [(L-MIN +
0.5) × 2q / DR]. [] Indicates truncation below the decimal point. L represents a pixel value, MIN
Represents the minimum value of the pixel values of the pixels in the block. DR
Represents the dynamic range of the block. When the dynamic range is equal to or less than 2 to the power of q, the code Q for the pixel value is calculated by L-MIN.

The encoding circuit 15 includes, as encoded data, a table index TI for designating a quantization table, a dynamic range DR, a minimum value MIN of pixel values of pixels in a block, a motion flag MF indicating motion, and The code Q corresponding to the pixel value is output. The table index TI, the dynamic range DR, the minimum value MIN of the pixel values of the pixels in the block, and the length (number of bits) of the motion flag MF indicating the motion are fixed at 8 bits. On the other hand, the length of the code Q for the pixel value changes.

The packetizing circuit 17 temporarily stores the encoded data supplied from the encoding circuit 15 in the external memory 1.
8 and temporarily stored encoded data,
The data is divided into 1.6 K bits, packetized by adding a header and the like, and supplied to the transmission circuit 19. The transmission circuit 19 modulates the packet supplied from the packetizing circuit 17 by a predetermined transmission method and transmits the modulated packet via a transmission path.

The PLL circuit 20 generates a reference signal synchronized with the image, and supplies the control circuit 21 with the reference signal. The control circuit 21 includes a thinning unit 13, a format conversion circuit 14, an encoding circuit 15, an external memory 16,
A control signal based on the reference signal supplied from the PLL circuit 20 is supplied to the packetizing circuit 17, the external memory 18, and the transmitting circuit 19 to control the entire operation of the transmitting device 1.

Referring to FIG. 3, the process of compressing the image signal input to transmitting apparatus 1 will be described. The interlaced Y signal having a field frequency of 60 Hz is converted by the A / D conversion circuit 42 into Y data of 528 pixels in the horizontal direction and 480 pixels in the vertical direction per field. The interlaced U and V signals having a field frequency of 60 Hz are converted to U data and V data of 176 pixels in the horizontal direction and 480 pixels in the vertical direction, respectively. The information of the image signal input to the A / D conversion circuit 42 corresponds to 166 Mbps, the information of the image data output from the A / D conversion circuit 42 becomes 104 Mbps, and the information amount is compressed.

528 per frame in interlaced mode with a field frequency of 60 Hz
The Y data including 480 pixels in the vertical direction are compressed by the decimation filter 43 into data having a frame frequency of 30 Hz and including 264 pixels in the horizontal direction and 240 pixels in the vertical direction per one frame in the progressive system. U data and V data having a field frequency of 60 Hz and including 176 pixels in the horizontal direction and 480 pixels in the vertical direction per frame of the interlaced system are converted into 30H by the decimation filter 43.
Progressive 1 with a frame frequency of z
88 pixels per frame horizontally and 2 pixels vertically
Each of them is compressed into data including 40 pixels.

U data and V data containing 88 pixels in the horizontal direction and 240 pixels in the vertical direction per frame are converted into data including 88 pixels in the horizontal direction and 120 pixels in the vertical direction by the UV vertical 1/2 circuit 45. Each is compressed. The information of the image data output from the UV vertical half circuit 45 is 21 Mbps, and the information amount is compressed as compared with the information of the signal input to the transmission device 1.

As described above, by performing the thinning-out process in the thinning-out section 13, the configuration of each circuit at the subsequent stage can be simplified, and the bit rate in the transmission line can be set to a practically small value.

U data and V data having a frame frequency of 30 Hz and including 88 pixels in the horizontal direction and 120 pixels in the vertical direction per one frame of the progressive system are converted by the format conversion circuit 14 into 8 data.
The data of 8 pixels × 120 pixels is converted into two combined data.

Y data having 264 pixels in the horizontal direction and 240 pixels in the vertical direction per frame;
The encoding circuit 15 encodes U data and V data of pixels × 120 pixels × 2. The information of the encoded image data is 8 Mbps, and the information amount is compressed as compared with the information of the analog signal input to the transmission device 1.

As described above, the transmitting apparatus 1 compresses and encodes an image signal.

FIG. 4 is a block diagram showing the configuration of the thinning filter 43. Interlaced Y data is
The signal is supplied to a horizontal half circuit 61 along a horizontal line. The horizontal half circuit 61 is a delay circuit (register)
71-1 to 71-N, multiplication circuits 72-1 to 72-
N and an adder circuit 73.

The Y data sequentially input to the horizontal half circuit 61 which is a half band filter is supplied to a delay circuit 71-1.
Through 71-N, each is delayed by one pixel in the horizontal direction, and sequentially output to the subsequent stage. The multiplication circuits 72-1 to 72-N multiply the input Y data (pixels) by 1 / N and output the result to the addition circuit 73. The addition circuit 73 adds the data supplied from the multiplication circuits 72-1 to 72-N and outputs the result to the vertical half circuit 62.

The vertical half circuit 62, which is a half-band filter, includes a latch circuit 81 and a field FIFO (Fast FIFO).
In Fast Out) 82, an adder circuit 83, and a latch circuit 84. Data output from the horizontal half circuit 61 is supplied to the latch circuit 81. When the clock signal is input, the latch circuit 81 latches the input data and outputs the latched data.

The operation of the horizontal half circuit 61 will be described with reference to FIG. Y sequentially input in the horizontal direction of the frame
The data (pixels indicated by white circles in FIG. 5) is
-1 to 71-N, and are multiplied by coefficients by the multiplying circuits 72-1 to 72-N, respectively. The outputs of the multiplication circuits 72-1 to 72-N are added to the addition circuit 73 and then output to the latch circuit 81. Latch circuit 81
, A clock signal instructing a latch is input at a timing corresponding to the black square in FIG. In the example of FIG. 5, one clock is input to the latch circuit 81 every time data of two pixels is transferred in the horizontal direction. For example, when N = 2 and the value of each coefficient is 1/2, , Adjacent two
The average value of the pixels is latched by the latch circuit 81 at the timing indicated by the black square in FIG. As described above, the number of pixels latched by the latch circuit 81 depends on the thinning filter 4
3 is の of the number of pixels of the data input.

The value latched by the latch circuit 81 is
It is supplied to the field FIFO 82 and the adder 83. The field FIFO 82 stores the data of the pixels of the first field, which is supplied from the latch circuit 81 and is thinned out in the horizontal direction by ２, and delays the data by one field.
Output to the addition circuit 83. The addition circuit 83 adds the data supplied from the latch circuit 81 and the field FIFO 82 and supplies the result to the latch circuit 84. When the enable signal is input, the latch circuit 84 latches the input data.

The operation of the vertical half circuit 62 will be described with reference to FIG. The Y data of an arbitrary pixel in the first field (a black square pixel on the first field shown in FIG. 6) and the one line below the line at the same position in the horizontal direction of the screen as the pixel in the first field After the data of the pixels of the second field (the black square pixels on the second field located at the lower right of the black square pixels on the first field shown in FIG. 6) are added by the adding circuit 83 , Latch circuit 84
Is output to Therefore, the average value of the pixels in the first field and the second field (white 4 in FIG. 6) is stored in the latch circuit 84.
(Corresponding to a pixel indicated by a square) is latched.

As described above, the output of the latch circuit 84 is decimated to 1/2 of the pixel data output from the latch circuit 81 between the first field and the second field. Between the third field and the fourth field, the data is thinned out to 1/2 and becomes progressive Y data having a frame frequency of 30 Hz.

The U data and V data are similarly thinned out in the horizontal and vertical directions by １ ／, and become progressive format data having a frame frequency of 30 Hz.

As described above, the Y data, U data, and V data of the image output from the thinning filter 43 are data in a progressive format having a frame frequency of 30 Hz which is thinned out.

Next, the UV vertical half circuit 45 will be described. FIG. 7 is a block diagram showing the configuration of the UV vertical 1/2 circuit 45. To the UV vertical 1/2 circuit 45, progressive U data and V data having a frame frequency of 30 Hz, which are thinned out, are input from the thinning filter 43. The input data is line F
By the FIFOs 91-1 to 91-6, one line at a time,
It is sequentially delayed and supplied to the subsequent stage. Multiplication circuit 92-1
Multiplies the input data by a coefficient and outputs the result to the addition circuit 93. The multiplication circuits 92-2 to 92-7 multiply the data input from the line FIFOs 91-2 to 91-6 by coefficients and output the result to the addition circuit 93.

The adder circuit 93 includes multiplying circuits 92-1 through 9-9
The data supplied from 2-7 is added to the latch circuit 9
4 The latch circuit 94 latches the input data when the clock signal is input.

The operation of the UV vertical half circuit 45 will be described with reference to FIG. Data of pixels (pixels indicated by white squares in FIG. 8) at the same position in the horizontal direction of the screen on seven consecutive lines are input to the multiplying circuits 92-1 to 92-7. The multiplication circuits 92-1 to 92-7 multiply input data by coefficients. The addition circuit 93 adds the outputs of the multiplication circuits 92-1 to 92-7 and outputs the result. The latch signal is input to the latch circuit 94 once every two lines (the timing indicated by a black circle in FIG. 8). Thus, the UV vertical half circuit 45 is
The number of lines of input U data and V data is 1
/ 2 and output.

The UV vertical half circuit 45 passes the Y data.

Next, the encoding circuit 15 will be described.
FIG. 9 is a block diagram illustrating a configuration of the encoding circuit 15. The control signal supplied from the control circuit 21 is input to the timing signal generation circuit 101.
The timing signal generation circuit 101 generates a timing signal based on the input control signal, and supplies the timing signal to the block shuffle circuit 102, the ADRC encoding circuit 103, and the inter-segment shuffle circuit 104.

The Y data, U data, and V data converted by the format conversion circuit 14 are input to the block shuffle circuit 102. The block shuffle circuit 102 receives the supplied Y data, U data, and V data.
The data is temporarily stored in the external memory 16. The block shuffle circuit 102 converts the Y data, U data, and V data stored in the external memory 16 into an AD
Rearranged in the form of RC block, shuffled in the segment range described later, and in ADRC block units,
It is supplied to the RC encoding circuit 103.

The ADRC encoding circuit 103 converts the Y data, U data,
And V data are encoded based on the ADRC scheme and supplied to the inter-segment shuffle circuit 104. The inter-segment shuffle circuit 104 temporarily stores the supplied data encoded by ADRC in the external memory 22.

The inter-segment shuffle circuit 104 shuffles the ADRC data stored in the external memory 22 by changing the storage and read addresses. The shuffling in the inter-segment shuffling circuit 104 increases the possibility that the receiving apparatus 2 can receive any of the dynamic range DR, the minimum value MIN, and the motion flag MF even if data is lost in the transmission path. Even if data loss occurs in the transmission path, the receiving device 2
The purpose is to separate each code Q to facilitate extraction.

Referring to FIG. 10, the unit of processing of encoding circuit 15 and packetizing circuit 17 will be described. Pixels are selected from a certain frame 0 and the next frame 1 by a method described later, and ADRC blocks of 1320 even-numbered segments (comprising 4 × 16 pixels) and 1320
A number of odd segment ADRC blocks (consisting of 4 × 16 pixels) are generated. The block shuffle circuit 102 selects 88 ADRC blocks from the generated even-numbered segment and odd-numbered segment ADRC blocks. The ADRC block contains the selected 8
In units of eight, the ADRC encoding circuit 103 calculates
In the ADRC, the dynamic range DR, the minimum value MIN, the motion flag MF, the table index TI, and the variable-length data (V
L).

Encoded data (hereinafter, referred to as a buffer) corresponding to the 88 ADRC blocks output from the ADRC encoding circuit 103 are grouped into five by the inter-segment shuffle circuit 104 (hereinafter, referred to as a buffer).
Segment). Three segments (shown as segment 0, segment 2, and segment 4 in the figure) corresponding to the even-numbered segments are shuffled between segments by the inter-segment shuffle circuit 104. Three segments corresponding to the odd-numbered segments (shown as segment 1, segment 3, and segment 5 in the figure)
Is shuffled between segments by the inter-segment shuffle circuit 104.

The segments shuffled between the segments are arranged in the order of segment 0, segment 1, segment 2, segment 3, segment 4, and segment 5.

The packetizing circuit 17 converts the encoded data stored in the segments arranged in the order of segment 0, segment 1, segment 2, segment 3, segment 4, and segment 5 into eight ADRCs.
Data (dynamic range DR,
The minimum value MIN, the motion flag MF, the table index TI, and the code Q) are collected and stored in a packet.

The operation of the block shuffle circuit 102 will be described with reference to FIGS. FIG.
It is a figure explaining a C block. Horizontal 8 from each frame
A block of adjacent 64 pixels of 8 pixels × 8 pixels is selected, and the leftmost and uppermost pixel (0-1 or 1-1 in the figure) of the block and As a reference, a pixel at a position shifted horizontally by two pixels from the pixel (a pixel indicated by 0-3 or 1-3 in the figure), a pixel at a position shifted vertically by two pixels from the pixel ( A pixel indicated by 0-17 or 1-17 in the figure) or a pixel shifted by one pixel horizontally and one pixel vertically from the pixel (denoted as 0-10 or 1-10 in the figure) Pixels) are referred to as even-numbered pixels. Further, similar selection is repeated with these pixels as a reference, and the selected pixels are set as even-numbered pixels.

The remaining pixels in each frame are odd pixels.

As shown in FIG. 11, even pixels (4 × 8 pixels) included in an 8 × 8 pixel block i2 of an even frame (frame 0) and 8 × 8 pixels of an odd frame (frame 1) are included.
A collection of even-numbered pixels (4 × 8 pixels) included in an 8-pixel block is defined as an ADRC block of an even-numbered segment. At the same time, odd pixels (4 × 8 pixels) included in an 8 × 8 pixel block of an even frame (frame 0) and odd pixels (4 × 8 pixels) included in an 8 × 8 pixel block of an odd frame (frame 1) Pixels) are taken as an odd-numbered segment ADRC block. The ADRC block is composed of every other pixel on the original frame of two frames. Pixels located on the original frame next to the pixels of the ADRC block of the even-numbered segment constitute a corresponding ADRC block of the odd-numbered segment.

FIGS. 12 to 14 are diagrams showing the relationship between the pixels in the frame and the ADRC block. As shown in FIG. 12, the pixel of the frame 0 is divided into blocks of 64 horizontal 8 pixels × 8 vertical pixels. Similarly, FIG.
As shown in FIG. 3, the pixels of the frame 1 (the frame next to the frame 0) are divided into blocks of 8 horizontal pixels × 8 vertical pixels of 64 pixels. A block of 64 pixels located at the leftmost and uppermost position in the block of frame 0 (FIG. 1)
2 is a 64-pixel block indicated by A1 in FIG. 2), and an even number of a 64-pixel block (64-pixel block indicated by A1 in FIG. 13) located at the leftmost and uppermost position in the frame 1 block As shown in FIG. 14, the ADRC block of the even-numbered segment constituted by the pixels is arranged as the segment 0 at the leftmost and uppermost position.

The AD of the odd-numbered segment formed by the odd-numbered pixels of the 64-pixel block indicated by A1 in FIG. 12 and the 64-pixel block indicated by A1 in FIG.
As shown in FIG. 14, the RC block is arranged as the segment 3 on the right side of the ADRC block of the even segment of the segment 0 in the drawing.

An ADR of an even-numbered segment composed of a 64-pixel block indicated by A2 in FIG. 12 and an even-numbered pixel of the 64-pixel block indicated by A2 in FIG.
As shown in FIG. 14, the C block is a segment 0 composed of a 64-pixel block indicated by A1 in FIG. 12 and an odd-numbered pixel of the 64-pixel block indicated by A1 in FIG. It is arranged on the right side of the ADRC block in the figure. A block of 64 pixels labeled A2 in FIG. 12 and a block of 6 labeled A2 in FIG.
As shown in FIG. 14, the ADRC block of the odd-numbered segment composed of the odd-numbered pixels of the 4-pixel block is shown as a segment 3 as a 64-pixel block indicated by A2 in FIG. 12 and A2 in FIG. It is arranged on the right side in the figure of the ADRC block of segment 0, which is composed of even-numbered pixels of a block of 64 pixels.

The ADR of the even-numbered segment constituted by the even-numbered pixels of the block of 64 pixels indicated by B1 in FIG. 12 and the block of 64 pixels indicated by B1 of FIG.
As shown in FIG. 14, the C block is a segment 4 composed of a 64-pixel block indicated by A1 in FIG. 12 and an even-numbered pixel of the 64-pixel block indicated by A1 in FIG. Are arranged below the ADRC block of FIG. A block of 64 pixels indicated as B1 in FIG. 12 and 6 blocks indicated as B1 in FIG.
As shown in FIG. 14, the ADRC block of the odd segment constituted by the odd pixels of the block of 4 pixels is indicated as the segment 1 by the block of 64 pixels indicated by A1 in FIG. 12 and by A1 in FIG. The lower side in the figure of the ADRC block of segment 3 (ADRC of segment 4) constituted by odd-numbered pixels of a block of 64 pixels
To the right of the block).

The ADR of the even segment constituted by the even-numbered pixel of the block of 64 pixels indicated by B2 in FIG. 12 and the block of 64-pixel indicated by B2 of FIG.
As shown in FIG. 13, the C block is composed of a 64-pixel block indicated by B1 in FIG. 12 and an odd-numbered pixel of the 64-pixel block indicated by B1 in FIG. It is arranged on the right side of the ADRC block in the figure. A block of 64 pixels labeled B2 in FIG. 12 and a block of 6 labeled B2 in FIG.
As shown in FIG. 13, the ADRC block of the odd-numbered segment constituted by the odd-numbered pixels of the 4-pixel block is a 64-pixel block indicated by B2 in FIG. 12 and a 64 indicated by B2 in FIG. A of an even segment constituted by even pixels of a block of pixels
It is arranged to the right of the DRC block in the figure.

An ADR of an even segment composed of a block of 64 pixels indicated by C1 in FIG. 12 and an even pixel of the block of 64 pixels indicated by C1 in FIG.
As shown in FIG. 14, the C block is a segment 2 composed of a block of 64 pixels indicated by B1 in FIG. 12 and an even pixel of the block of 64 pixels indicated by B1 in FIG. It is arranged below the ADRC block in the figure. A block of 64 pixels labeled C1 in FIG. 12 and a block of 6 labeled C1 in FIG.
The ADRC block of the odd segment constituted by the odd pixels of the block of 4 pixels is, as shown in FIG. 14, a block of 64 pixels indicated as B1 in FIG. 12 and 64 indicated as B1 in FIG. A of segment 1 composed of odd pixels of a block of pixels
It is arranged below the DRC block in the figure (to the right of the ADRC block of segment 2).

The ADR of the even segment constituted by the even pixel of the block of 64 pixels indicated by C2 in FIG. 12 and the block of 64 pixels indicated by C2 of FIG.
As shown in FIG. 14, the C block is a segment 2 composed of a 64-pixel block indicated by C1 in FIG. 12 and an odd-numbered pixel of the 64-pixel block indicated by C1 in FIG. It is arranged on the right side of the ADRC block in the figure. A block of 64 pixels labeled C2 in FIG. 12 and a block of 6 labeled C2 in FIG.
As shown in FIG. 14, the ADRC block of the odd-numbered segment constituted by the odd-numbered pixels of the 4-pixel block is a 64-pixel block indicated by C2 in FIG. 12 and 64 indicated by C2 in FIG. A of an even segment constituted by even pixels of a block of pixels
It is arranged to the right of the DRC block in the figure.

The above procedure is performed to construct each frame 3
By repeating for each of 3 × 10 blocks (264 × 240 pixels), 66 data in the horizontal direction and 30 data in the vertical direction are arranged from the Y data of the two frames as shown in FIG. A set of ADRC blocks is generated. As shown in FIG. 15B, the ADRC block generated from the Y data is shuffled to form a segment.

By performing the same processing for U data and V data of 88 × 120 pixels, FIG.
As shown in (A), 22 in the horizontal direction and 1 in the vertical direction
A set of five ADRC blocks is generated.
Similarly to the case of the Y data, as shown in FIG.
The ADRC block generated from the U data and the V data is block shuffled to form a segment.

In each ADRC block, pixels located at the same position on the screen of the even-numbered frame and the odd-numbered frame (for example, the pixel indicated by 0-1 and the pixel indicated by 1-1 in FIG. 11). , The absolute value of the difference between the pixel values is taken, and if the maximum value of the absolute values of the differences is less than the threshold Th1, the ADRC block is regarded as a still image.

Of the absolute values of the difference, the maximum value is the threshold value Th2
If (Th1 <Th2), the ADRC block is regarded as a moving image.

In the ADRC block of the still image, the average of the pixel values of the pixels at the same position on the screen of the even-numbered frame and the odd-numbered frame is averaged, and the average of the 32 values is newly added to A.
It is replaced as a DRC block, and the motion flag MF attached to the ADRC block is set to 0.

In the ADRC block of the moving image, 64 pixels collected from the even frame and the odd frame are directly converted to ADR.
As the C block, the motion flag MF attached to the ADRC block is set to 1.

Next, a description will be given of shuffling performed in ADRC block units for each segment. In a segment composed of an ADRC block of Y data numbered 0, if a serial number is assigned to the ADRC block, as shown in FIG. 17A, y0 to y32
The number 9 is assigned to the ADRC block. When the ADRC block is shuffled, the ADRC block of the Y data is arranged as shown in FIG.

Similarly, the A data of U data numbered 0
When a serial number is assigned to the ADRC block in a segment composed of DRC blocks, FIG.
As shown in (C), the numbers of u0 to u54 are ADRs.
Attached to C block. As shown in FIG. 17D, the ADRC blocks of U data are rearranged in the reverse order of the serial numbers.

In a segment composed of an ADRC block of V data numbered 0, its ADR
When serial numbers are assigned to the C blocks, the numbers v0 to v54 are assigned to the ADRC blocks as shown in FIG. The ADRC block of V data is shown in FIG.
As shown in (F), the images are rearranged in the reverse order of the serial numbers.

Next, as shown in FIG. 18, the ADRC block for Y data, the ADRC block for U data, and the ADRC block for V data are shuffled.
After three Y data ADRC blocks, one U
An ADRC block of data is arranged, followed by 3
One Y data ADRC block is arranged, followed by one V data ADRC block.
This arrangement is repeated. For example, segment 0 contains
ADRC block with y0 serial number, y221
ADRC block with a serial number of y112, ADRC block with a serial number of u54, ADRC block with a serial number of y3, and A with a serial number of y224
ADR with DRC block, y115 serial number
The ADRC blocks are arranged in the order of a C block, an ADRC block with a serial number of v54, an ADRC block with a serial number of y6, and the like.

In the segments 1 to 5, the ADRC block is similarly shuffled.

As described above, the pixel data is shuffled for each ADRC block by the block shuffling circuit 102, and the data is stored in the ARC block based on the shuffled ADRC block.
The data is encoded by the DRC encoding circuit 103. By appropriately shuffling the ADRC block, pixels that are missing due to a communication error are dispersed, making it difficult to recognize the position of the missing pixel in the reproduced image, and creating the missing pixel with surrounding pixels. can do.

Next, the configuration of the ADRC encoding circuit 103 will be described with reference to FIG. Block shuffle circuit 1
02, the image shuffled as an ADRC block is a delay circuit 161, a motion feature amount calculation circuit 162, D
The signal is supplied to the R calculation circuit 163 and the delay circuit 166.

The delay circuit 161 delays the two-frame block (an ADRC block composed of 4 × 16 pixels) by a time corresponding to the processing time of the motion characteristic amount calculation circuit 162, and outputs the result to the information amount control circuit 164. The motion feature value calculation circuit 162 calculates a motion feature value which is the maximum value of the absolute value of the inter-frame difference for each ADRC block, and supplies the motion feature value to the information amount control circuit 164. The DR calculation circuit 163
The information amount control circuit 164 calculates, for each ADRC block, a dynamic range DR when the image is regarded as a still image and a dynamic range DR when the image is regarded as a moving image.
To supply.

The information amount control circuit 164 includes a delay circuit 161
2 frame blocks supplied from the motion feature amount supplied from the motion feature amount calculation circuit 162, the DR calculation circuit 16
3 based on the dynamic range DR when regarded as a still image and the dynamic range DR when regarded as a moving image, and the thresholds Th1 and Th2 and the thresholds T1 and T2 supplied from the threshold table 165. , An MF selection threshold including the thresholds Th1 and Th2, and outputs the selected threshold to the static / movement determination circuit 169.

The information amount control circuit 164 includes a two-frame block, a motion feature amount, a dynamic range DR when regarded as a still image, a dynamic range DR when regarded as a moving image, and a threshold value supplied from the threshold table 165. Based on the set of Th1 and Th2 and the thresholds T1 and T2, a Q-bit selection threshold is selected and output to the ADRC encoder 171.

For example, when the motion feature amount is 4, the dynamic range DR when considered as a still image is 14, and the dynamic range DR when considered as a moving image is 15, a threshold T1 with a table index of 0 is set. Is 6 and the threshold is
T2 is 12, the threshold value Th1 is 3, the threshold value Th2 is 3, the threshold value T1 of the table index is 1 is 13, the threshold value T2 is 40,
The case where the threshold Th1 is 5 and the threshold Th2 is 5 will be described.

The information amount control circuit 164 determines that the set is a moving image because the motion feature amount is 4 and the threshold Th2 is larger than 3 for the set of threshold values with the table index of 0. Dynamic range DR for video
Since T2 is greater than 3, q of this ADRC block is 4
The information amount control circuit 164 determines that the code Q is (16 *
4) It is required to be 256 bits from * 4.

Similarly, the information amount control circuit 164 determines that the image is a still image because the motion feature amount is smaller than the threshold value Th1 for the set of threshold values with the table index of 1. The dynamic range DR when considered as a still image is the threshold T1
Since it is larger and smaller than the threshold value T2, q of the ADRC block is 3, and the information amount control circuit 164 determines that the code Q is 96 bits from (8 * 4) * 3.

As described above, in the set of thresholds, the respective thresholds are set so that the information amount decreases as the table index increases. Information amount control circuit 16
4 is that the code Q generated for one buffer is
A table index of 16,104 bits or less and having the largest information amount is selected as a table index TI,
Output to the delay circuit 173.

The information amount control circuit 164 may calculate the number of bits of the code Q corresponding to the table index in parallel, or may sequentially calculate the bit number of the code Q from the larger (or smaller) table index.

An example of the configuration of the information amount control circuit 164 will be described with reference to FIGS. FIG. 20 is a diagram showing a configuration of a former stage of the information amount control circuit 164, and FIG.
12 is a diagram illustrating a configuration of a subsequent stage of the information amount control circuit 164. FIG.

As shown in FIG. 20, the information amount control circuit 16
4 are provided with comparison circuits 181-1 and 181-2 corresponding to the number of thresholds of the dynamic range DR.
, Adders 182-1 and 182-2, and register 1
83-1 and 183-2. A dynamic range DR that can take a value from (0 to 255) is supplied to one input terminal of each of the comparison circuits 181-1 and 181-2.

In FIG. 20, the threshold table 16
5, for example, the threshold value T1 and the threshold value T2 are read and supplied to the other input terminals of the comparison circuits 181-1 and 181-2, respectively. Specifically, the comparison circuit 1
The threshold value T1 is supplied to the other input terminal of 81-1 and the threshold value T2 is supplied to the other input terminal of the comparison circuit 181-2.

Each of comparison circuits 181-1 and 181-2 has a dynamic range DR from input terminal 5.
Is greater than or equal to each threshold, an output of “1” is generated. The comparison outputs of the comparison circuits 181-1 and 181-2 are supplied to one input terminal of one of the adders 182-1 and 182-2 connected to the respective circuits. Adder 182-1
Or the other input terminal of 182-2 is connected to the register 18
It is configured such that its own added output is supplied via 3-1 and 183-2.

Therefore, (T1) is calculated by the adder 182-1.
≤ DR ≤ 255), and the obtained data is stored in the register 183-1. The adder 182-2 calculates the integrated value of the frequency of (T2 ≦ DR ≦ 255), and stores the obtained data in the register 183-
2 is held.

Each of the frequency data stored in each of the registers 183-1 and 183-2 is extracted and supplied to the adder 191 as shown in FIG. Further, the adder 191 has an information amount control circuit 1 in advance.
The frequency data stored in 64 is supplied. In the adder 191, occurrence frequency data of (T1 ≦ DR ≦ 255), occurrence frequency data of (T2 ≦ DR ≦ 255),
The frequency data stored in advance is added to calculate the total amount.

The occurrence value of (T1 ≦ DR ≦ T2-1) is (S3), and the occurrence value of (T2 ≦ DR ≦ 255) is (S4). The occurrence frequency value of (T2 ≦ DR ≦ 255) is (S4 + S3 + S2) (corresponding to the code Q in which q is 4 bits), and the occurrence frequency value of (T1 ≦ DR ≦ 255) is (S3 + S2) (q (Corresponding to a code Q of 3 bits). (S2) (corresponding to the code Q in which q is 2 bits) is generated for all ADRC blocks, and can be stored in the information amount control circuit 164 in advance. When these are added by the adder 191, an output of (S3 + 2 * S3 + 3 * S2) is obtained. This output is the same total amount as when multiplying the allocated bits and performing full addition.

The total amount calculated by the adder 191 is supplied to one input terminal of the comparison circuit 193 via the register 192. A buffer maximum value is supplied to the other input terminal 12 of the comparison circuit 193, and the total amount is compared with the buffer maximum value. On the basis of the comparison output of the comparison circuit 193, a threshold value such that the total amount becomes equal to or less than the buffer maximum value is determined. That is, the threshold table 165 stores a plurality of sets of threshold values so that the total amount of generated data simply increases or decreases in advance, and the threshold values are selected so that the total amount falls within the target.

The information amount control circuit 164 includes a comparison circuit 193
MF selection threshold, Q bit selection threshold,
And the table index TI is selected and output.

Returning to FIG. 19, the delay circuit 166 delays the two-frame block (an ADRC block composed of 4 × 16 pixels) by a time corresponding to the processing time of the delay circuit 161 and the information amount control circuit 164, and Generation circuit 167, delay circuit 168, and static / motion determination circuit 16
9 is output. The stationary block generation circuit 167 calculates the average of the pixel values of the pixels at the same position on the screen of the even-numbered frame and the odd-numbered frame from the two-frame block (the ADRC block including 4 × 16 pixels), and calculates the average of the 32 pixels. Replace the value with a new ADRC block, 1
The data is supplied to the selection circuit 170 as a frame block (an ADRC block composed of 4 × 8 pixels).

The delay circuit 168 delays the two-frame block (an ADRC block composed of 4 × 16 pixels) by a time corresponding to the processing time of the still block generation circuit 167, and supplies it to the selection circuit 170.

The still / moving decision circuit 169 determines the AD based on the two frame blocks (ADRC block composed of 4 × 16 pixels) and the MF selection threshold composed of thresholds Th1 and Th2.
A motion flag MF attached to the RC block is generated and supplied to the selection circuit 170 and the delay circuit 172.

When the motion flag MF is 1, that is, when the ADRC block corresponds to a moving image, the selection circuit 170 supplies the two frame blocks supplied from the delay circuit 168 to the ADRC encoder 171 and supplies the motion flag MF
Is 0, that is, when the ADRC block corresponds to a still image, the one frame block supplied from the still block generation circuit 167 is supplied to the ADRC encoder 171.

The ADRC encoder 171 encodes the two-frame block or one-frame block supplied from the selection circuit 170 on the basis of the Q-bit selection threshold supplied from the information amount control circuit 164, and obtains the dynamic range DR and the minimum value. Output MIN, q, and code Q.

The delay circuit 172 delays and outputs the motion flag MF supplied from the static / movement determination circuit 169 in accordance with the processing time of the selection circuit 170 and the processing time of the ADRC encoder 171.

The delay circuit 173 delays the table index TI supplied from the information amount control circuit 164 in accordance with the delay time of the delay circuit 168, the processing time of the selection circuit 170, and the processing time of the ADRC encoder 171. ,Output.

As described above, the ADRC encoding circuit 103
Is the dynamic range DR for each ADRC block,
A minimum pixel value MIN, a motion flag MF, and a code Q are generated and supplied to the inter-segment shuffle circuit 104 together with the quantization table index TI.

FIG. 22 shows an inter-segment shuffle circuit 10.
FIG. 4 is a diagram for explaining a method of shuffling No. 4; As shown in FIG. 22, there are three ways of shuffling, that is, shuffle 0 and shuffle 2. Shuffle 0 does not shuffle. Shuffle 1 converts the data of segment 0 (segment 1) to segment 4 (segment 5), the data of segment 2 (segment 3) to segment 0 (segment 1), and the data of segment 4 (segment 5) to segment 2 (segment 2). Each segment 3) is shuffled (replaced).

Similarly, shuffle 2 transfers data of segment 0 (segment 1) to segment 2 (segment 3).
Then, the data of segment 2 (segment 3) is shuffled to segment 4 (segment 5), and the data of segment 4 (segment 5) is shuffled to segment 0 (segment 1).

FIG. 23 shows segment 0, segment 3,
FIG. 10 is a diagram for explaining shuffling of a dynamic range DR performed in a segment 5; One segment is composed of five buffers, and one buffer includes 88 ADRC blocks.
Forty DRs are included. D existing in each segment
Numbers 1 to 440 are sequentially assigned to R, and in FIG. 23, for example, the DR of the number 1 of the segment 1 is described as DR1-1.

For the DR of number 1 of each segment,
Shuffle 0 is used (ie, not swapped), shuffle 1 is used for DR number 2 and shuffle 2 is used for DR number 3;
For the DR of No. 4, shuffle is performed in the order of shuffle 0, shuffle 1, and shuffle 2, for example, shuffle 0 is used.

Similarly, shuffling is performed on the code Q. Further, as shown in FIG. 24, for the MIN that is the minimum pixel value, shuffle 1, shuffle 2,
Shuffle is performed in the order of shuffle 0 and shuffle is performed. further,
For the MF that is the motion flag, as shown in FIG. 25, shuffle 2, shuffle 0, and shuffle 1 are performed in this order.

Next, the index TI of the quantization table
Will be described. The TI stored in segment 0 is also stored in segment 3, and the TI stored in segment 3 is also stored in segment 0. That is, segment 0 and segment 3 store both the own TI and the partner's TI.
Similarly, segment 1 and segment 4 and segment 3 and segment 5 respectively store both their own TI and the partner's TI.

As described above, since the dynamic range DR, the minimum pixel value MIN, and the motion flag MF are shuffled, even if a communication error occurs and a packet containing any data is lost, The receiving device 2
When decoding a given ADRC block, missing data can be recovered based on other data that could be received. Further, even if a communication error occurs due to the shuffling of the code Q and a packet including the code Q is lost, the pixels corresponding to the missing code Q are dispersed, and the receiving apparatus 2 It becomes easier to cut out, and when decoding a predetermined ADRC block, it becomes easier to create a missing pixel value based on the decoded pixel value.

As described above, by shuffling data between segments, the communication device 2 can easily recover data and create missing pixels, and can reproduce an image to be reproduced even if a packet is lost in the transmission path. Quality can be maintained.

The data output from the inter-segment shuffle circuit 104 of the encoding circuit 15 is
Is input into a packet. FIG. 26 shows the configuration of the packet. The packet configuration shown in FIG. 26 is composed of 201 bytes, of which DR is A × 8 Mbit, MI is
N is B × 8M bits, MF is C × 8M bits, TI is D
× (8M / D) bits and code Q (VL-Data) are composed of 8 × N bits. Data is stored using a plurality of packets of this size unit.

By configuring the dynamic range DR, the minimum pixel value MIN, the motion flag MF, the table index TI, and the code Q to be a multiple of 8, the shuffling described above is performed. It has a unified architecture and is suitable to be handled in byte units regardless of the bit size of each data.
In the above description and the following description, FIG.
This is a case where the packet configuration shown in FIG.

Next, the process of transmitting an image by the transmission device 1 will be described with reference to the flowchart of FIG. Step S
In 11, the A / D conversion circuit 42 of the thinning unit 13
An input interlaced image signal having a field frequency of 60 Hz is sampled at a predetermined sampling rate, and Y data at a predetermined data rate is sampled.
Output as U data and V data. Step S
At 12, the thinning filter 43 thins out the Y data, U data, and V data by に in the horizontal and vertical directions, respectively, and converts the data into progressive data having a frame frequency of 30 Hz.

In step S13, UV vertical 1/2
The circuit 45 thins out U data and V data in the vertical direction by half. In step S14, the format conversion circuit 14 converts the Y data, U data, and V data into a format that can be supplied to the encoding circuit 15.

In step S15, the encoding circuit 15
, Shuffles, encodes, and further shuffles the Y data, U data, and V data supplied from the format conversion circuit 14 and outputs the result to the packetization circuit 17. In step S16, the packetizing circuit 17
The data supplied from the encoding circuit 15 is packetized and output to the transmission circuit 19. In step S19, the transmitting circuit 19 transmits the packet supplied from the packetizing circuit 17 by a predetermined method, and the process ends.

As described above, the transmitting apparatus 1 compresses the input image signal and outputs it as a packet.

Next, the receiving device 2 will be described. FIG.
FIG. 8 is a block diagram illustrating a configuration of the receiving device 2. The packet received by the receiving circuit 301 is transmitted to the packet disassembling circuit 3.
02. When a loss occurs in a received packet, the reception circuit 301 sets a loss flag for each bit of data of the lost packet. The packet disassembly circuit 302 temporarily stores the supplied packet in the external memory 3.
03, the packet temporarily stored in the external memory 303 is disassembled, the data included in the packet is extracted, and supplied to the decoding circuit 304 together with the missing flag.
A missing flag may be attached to the code Q in units of 8 bits, and a missing flag may be attached to each of the dynamic range DR, the minimum pixel value MIN, and the motion flag MF.

The decryption circuit 304 is provided for the packet disassembly circuit 30.
2 is temporarily stored in the external memory 305, and the AD temporarily stored in the external memory 305 is stored in the external memory 305.
The data encoded by the RC method is decoded and supplied to the format conversion circuit 306. The decoding circuit 304 outputs a dynamic range DR or a minimum value MIN necessary for decoding.
If data such as is missing, the missing data is recovered and decoded, and when decoding is successful, the corresponding missing flag is reset.

The format conversion circuit 306 temporarily stores the data supplied from the decoding circuit 304 in the external memory 30.
7, the data temporarily stored in the external memory 307 is subjected to a process reverse to the process of the format conversion circuit 14, and is supplied to the missing pixel creation circuit 308.

The missing pixel creation circuit 308 converts the data of the missing pixels during transmission by the class classification adaptive processing based on the data supplied from the format conversion circuit 306.
The data is generated from the pixel values of the pixels located around the missing pixel, that is, spatially or temporally, and the data is supplied to the format conversion circuit 310. The initialization circuit 309 supplies a coefficient set and the like to the missing pixel creation circuit 308 when the receiving device 2 is activated.

The format conversion circuit 310 temporarily stores the data supplied from the missing pixel creation circuit 308 in the external memory 311, and converts the data temporarily stored in the external memory 311 into the quadruple-density resolution creation circuit 312. Convert to progressive 3: 0.5: 0.5 image data with a frame frequency of 30 Hz that can be processed,
The converted data is supplied to the quad resolution resolution creating circuit 312.

The quadruple-density resolution creation circuit 312 uses the image data supplied from the format conversion circuit 310 to
Image data having a density four times that of the supplied image is created and supplied to the interpolation unit 314. The initialization circuit 313 includes:
The coefficient set and the like are supplied to the quadruple-density resolution creation circuit 312.

The interpolation section 314 is provided for the format conversion circuit 3
31, an external memory 332, a UV vertical doubling circuit 333, an external memory 334, a D / A conversion circuit 335, and a post-filter 336.

The format conversion circuit 331 receives the data supplied from the quadruple-density resolution creation circuit 312 and temporarily stores it in the external memory 332.
The data temporarily stored in the UV vertical doubling circuit 333
Is converted into progressive 3: 0.5: 0.5 image data having a frame frequency of 30 Hz, which can be processed, and outputs the converted data to the UV vertical doubling circuit 333.

The UV vertical doubling circuit 333 temporarily stores the data supplied from the format conversion circuit 331 in the external memory 334, and converts the U data and V data of the data temporarily stored in the external memory 334 into The data is interpolated twice in the vertical direction, and the Y data, U data, and V data are supplied to the D / A conversion circuit 335.

The D / A conversion circuit 335 outputs the Y data, U data, and V data supplied from the UV vertical doubling circuit 333.
The data is converted into an analog Y signal, U signal, and V signal, and output to the post-filter 336. The post-filter 336 is a filter (so-called sinX / X filter) that passes only valid components of the analog Y signal, U signal, and V signal output from the D / A conversion circuit 335. The Y, U, and V signals that have passed through the post-filter 336 are
5 or output as it is as an output of the receiving device 2.

The NTSC encoder 315 generates and outputs a component signal (Y / C signal) and a composite video signal based on the input Y signal, U signal, and V signal.

The PLL circuit 316 generates a reference signal as a reference for processing of each circuit, and supplies the reference signal to the control circuit 317. The control circuit 317 includes a packet disassembly circuit 302, a decoding circuit 304, and a format conversion circuit 30.
6. Missing pixel creation circuit 308, format conversion circuit 3
The generated control signal is supplied to the 10, 4 times dense resolution creation circuit 312, the interpolation unit 314, and the NTSC encoder 315 to control the operation of the entire reception device 2.

With reference to FIG. 29, the process of expanding image data received by receiving apparatus 2 when a signal is output in the interlace mode will be described. The image data decoded by the decoding circuit 304 is transmitted to the format conversion circuit 30.
6, Y data having 264 pixels horizontally and 240 pixels vertically per frame in a progressive system having a frame frequency of 30 Hz, and 88
It is converted into U data and V data of pixels × 120 pixels × 2.

The Y data, U data, and V constituting one frame output from format conversion circuit 306
After the missing pixel is created by the missing pixel creation circuit 308, the data is converted to 60 Hz by the quadruple-density resolution creation circuit 312.
Are converted into an interlaced first field and a second field having a field frequency of At this time, the Y data is data including 528 pixels in the horizontal direction and 240 pixels in the vertical direction per one field. The U data and the V data are data including 176 pixels in the horizontal direction and 120 pixels in the vertical direction per one field.

The Y data, U data, and V data output from the quadruple-density resolution creation circuit 312 are converted into 528 pixels in the horizontal direction and 480 pixels in the vertical direction per frame by the format conversion circuit 331.
The data is converted into data including pixels, and the U data and the V data are each divided by 176 in the horizontal direction per one frame.
The data is converted into data including pixels and 240 pixels in the vertical direction.

Further, the U data and V data are converted by the UV vertical doubling circuit 333 into data including 176 pixels in the horizontal direction and 480 pixels in the vertical direction per one frame. The D / A conversion circuit 335 performs digital-to-analog conversion, converts Y data to an analog Y signal, and converts U data and V data to an analog U signal and an analog V signal, respectively.

As described above, the interpolation section 314 causes the thinning section 1
By performing the process corresponding to No. 3, an original image having a sufficient image quality can be reproduced from data transmitted at a bit rate reduced to a practical value.

As described above, the receiving apparatus 2 decompresses the data of the received image to obtain the interlaced Y signal,
Output as U signal and V signal.

Next, the decoding circuit 304 will be described.
FIG. 30 is a block diagram showing a configuration of the decoding circuit 304. The control signal supplied from the control circuit 317 is input to the timing signal generation circuit 351. The timing signal generation circuit 351 generates a timing signal based on the control signal, and generates an inter-segment shuffle circuit 352, a q-bit MF recovery circuit 353,
DRMIN recovery circuit 354, ADRC decoding circuit 35
5, to the block deshuffle circuit 356 and the recovery error determination circuit 357.

Data supplied from the packet disassembly circuit 302 is input to the inter-segment shuffle circuit 352. The inter-segment shuffling circuit 352 performs the reverse operation of the inter-segment shuffling circuit 104 of the transmitting apparatus 1 shown in FIG. 9 and returns the shuffled data to the original order. When there is no missing in the deshuffled data, the inter-segment deshuffle circuit 352 converts the deshuffled data to A
This is supplied to the DRC decoding circuit 355. When there is a lack in the deshuffled data, the inter-segment deshuffle circuit 3
52 supplies the shuffled data to the q-bit MF recovery circuit 353 and the DRMIN recovery circuit 354. When there is an error in the data of a plurality of blocks extending between segments, the inter-segment shuffle circuit 352 outputs a continuous error signal to the block shuffle circuit 356.

The q-bit MF recovery circuit 353 utilizes the fact that the correlation between adjacent pixels remains in the compressed code Q, and based on the correlation between the pixel values of the pixels of three consecutive ADRC blocks, calculates the correlation. Is maximized, a combination of q and the value of the motion flag MF is obtained, and the DRMIN recovery circuit 3
Output to 54. The q-bit MF recovery circuit 353
Emi for determining whether the motion flag MF is correct
The n1 and the emin2 (the minimum value of the adjacent pixel difference and the second smallest value) are transmitted to the recovery error determination circuit 357.

FIG. 31 shows a q-bit MF recovery circuit 35.
FIG. 3 is a block diagram showing a configuration of No. 3; The deshuffled data supplied from the segment deshuffle circuit 352 is supplied to a q-bit MF extraction circuit 371.

The q-bit MF extracting circuit 371 extracts q and the motion flag MF from the supplied data, and calculates three block evaluation value calculating circuits 372-1 to 372-372.
6 respectively. If q and the motion flag MF are lost due to an error, the q-bit MF extraction circuit 371
Is a combination of the possible combinations of q and the motion flag MF in the three block evaluation value calculation circuits 372-1 to 372-1.
2-6.

For example, when q is set to “2” and “
The motion flag MF in which “0” is set is supplied to the three-block evaluation value calculation circuit 372-1, and q in which “3” is set.
The motion flag MF in which “0” is set is supplied to the three-block evaluation value calculation circuit 372-2, and the q in which “4” is set and the motion flag MF in which “0” is set are:
It is supplied to a three-block evaluation value calculation circuit 372-3. Further, for example, the q in which “2” is set and the motion flag MF in which “1” is set are the three-block evaluation value calculation circuit 3.
Q and "3" which are supplied to 72-4 and "3" are set.
The motion flag MF in which “1” is set is supplied to the three-block evaluation value calculation circuit 372-5, and q in which “4” is set.
The motion flag MF in which “1” is set is supplied to the three-block evaluation value calculation circuit 372-6.

The three-block evaluation value calculation circuits 372-1 to 372-6 calculate the difference between the pixel values of adjacent pixels from the pixel value decoded from the cut-out code Q based on q and the motion flag MF. Evaluation value based on (LE: Linear
Error), and supplies an evaluation value to the selector 373 together with q and the motion flag MF. Hereinafter, when it is not necessary to individually distinguish the three block evaluation value calculation circuits 372-1 to 372-6, the three block evaluation value calculation circuit 3
No. 72.

The evaluation value is calculated by equation (1). The smaller the evaluation value, the higher the pixel correlation.

[0171]

(Equation 1) Q _{i, j} indicates a non-missing code Q included in the ADRC block to be recovered. Q ′ _{i, j} is Q
FIG. 14 shows a code Q included in another ADRC block including the pixels of the 64-pixel block illustrated in FIGS. 12 and 13 including the pixels configuring the ADRC block including _{i and j} . For example, when Q _{i, j} is included in the ADRC block of the even segment with the number “0” in FIG.
_{i and j} are included in the ADRC block of the odd-numbered segment with the number “3”. When Q _{i, j} is included in the ADRC block of the even segment with the number “4” in FIG.
_{i and j} are included in the ADRC block of the odd-numbered segment with the number “1”. q _{i, j} indicates q from which Q _{i, j} is cut out.
q ′ _{i, j} indicates q that cuts out Q ′ _{i, j} . Nj is ADR
Indicates the number of codes Q not missing in the C block. j
Is a number for identifying three ADRC blocks for which evaluation values are calculated.

The offset j is used to absorb the difference in the dynamic range, q, between the three ADRC blocks, and to calculate a more accurate evaluation value. Equations (2), (3), and ( It is calculated in 4).

Offset j = (Xj × 24 ^-qj- Yj × ^24-q'j ) / 128 (2)

[0174]

(Equation 2)

[0175]

(Equation 3)

From the evaluation values supplied from each of the three-block evaluation value calculation circuits 372-1 to 372-6, the selector 373 determines an evaluation value having the minimum value and an evaluation value having the next largest value after the minimum value. Select and emi respectively
The signals are supplied to the recovery error determination circuit 357 as n1 and emin2.

The selector 373 outputs the q and the motion flag M output by the one that outputs the evaluation value having the minimum value among the three block evaluation value calculation circuits 372-1 to 372-6.
F is selected and supplied to the DRMIN recovery circuit 354 as an output of the q-bit MF recovery circuit 353.

The DRMIN recovery circuit 354 calculates the correct value of q and the correct value of the motion flag MF supplied from the q-bit MF recovery circuit 353 and the corresponding ADRC shown in FIG.
The correct value of the dynamic range DR for each ADRC block and the minimum value MIN of the pixel value for each ADRC block are obtained based on the blocks and output to the ADRC decoding circuit 355.

Referring to FIGS. 32 to 34, DRMIN
The recovery process of the dynamic range DR or the minimum pixel value MIN of the recovery circuit 354 will be described. The pixel value L is calculated from the code Q by the formula (5) according to the ADRC method.
Is decrypted by the processing of.

L = DR / (2 ^Q ) * q + MIN (5)

As shown in FIG. 32, the relationship between q and the pixel value L is represented by a straight line having a slope of DR / (2 ^Q ) and a contact piece having a minimum pixel value MIN.

From the set of (q, L) in the ADRC block, DR / (2 ^Q ) in equation (5) and the minimum pixel value M
IN is determined statistically. The regression line of (q, L) shown in FIG. 33 is assumed to be equal to the line of Expression (5).

At this time, if b is known, a
It is calculated by equation (6).

A = (B-nb) / A (6)

If a is known, b is calculated by equation (7).

B = (B-Aa) / n (7)

Here, n is the number of points of (q, L), A is Σx, and B is Σy.

The dynamic range DR is calculated by equation (8).

DR = (ΣL'-N * MIN) / (Σq) * 2 ^Q = (((DR '/ 2 ^Q' ) * Y) + N * MIN'-N * MIN) / X * 2 ^Q = (N * (MIN'-MIN) * 2Q '+ DR' * Y) * 2 ^Q / (X * 2 ^{Q '} ) (8)

As shown in FIG. 34A, assuming that the number of non-missing blocks adjacent to the block for which the dynamic range DR or the minimum value MIN of the pixel value is to be recovered is ni, N is Σni It is. In FIG. 34, a hatched square indicates a dynamic range DR.
Alternatively, it indicates a block in which the minimum value MIN of the pixel value is to be recovered. In FIG. 34, squares without oblique lines indicate blocks with no missing portions. When q of the block for which the dynamic range DR or the minimum value MIN of the pixel value is to be recovered is missing, as shown in FIG.
ni is 0.

DR 'indicates the dynamic range of a non-missing block adjacent to a block to be recovered. MIN ′ indicates the minimum pixel value of a block that is adjacent to the block to be recovered and that is not missing. Q ′ indicates the code Q of a non-missing block adjacent to the block to be recovered.

X is represented by equation (9).

[0193]

(Equation 4)

Y is represented by equation (10).

[0195]

(Equation 5)

The minimum value MIN of the pixel value is calculated by equation (11).

MIN = (ΣL'-Σq * (DR / 2 ^Q )) = ((DR '/ 2 ^Q' ) * Y + N * MIN '-(DR / 2 ^Q ) * X) / N = (N * MIN '* 2 ^{Q' + Q} + DR '* Y * 2 ^Q -DR * X * 2 ^Q' ) / (N * 2 ^{Q '+ Q} ) (11)

The ADRC decoding circuit 355 decodes the code Q based on the data supplied from the inter-segment deshuffling circuit 352 when there is no missing in the deshuffled data, and obtains Y data and U data obtained as a result of decoding. , And V data to the block shuffle circuit 356.

The ADRC decoding circuit 355 outputs the DRMIN recovery circuit 354 when q or the motion flag MF is missing and the q-bit MF recovery circuit 353 can obtain the correct value of q and the correct value of the motion flag MF. The correct value of q supplied via the
The code Q supplied from the DRMIN recovery circuit 354 is decoded based on the correct value of F, the dynamic range DR, and the minimum value MIN of the pixel value for each block, and the Y data, U data, and V data
", Corresponding to Y data, U data, and V data.
The data is supplied to the block shuffle circuit 356 together with the error flag in which “0” is set.

ADRC decoding circuit 355 corresponds to code Q when q or motion flag MF is missing and q-bit MF recovery circuit 353 cannot find the correct value of q and the correct value of motion flag MF. Y
Error flag for data, U data and V data
1 ”is set and supplied to the block shuffle circuit 356.

The ADRC decoding circuit 355 determines that the dynamic range DR or the minimum value MIN of the pixel value is missing and the DRMIN recovery circuit 354 determines the correct value of the dynamic range DR and the correct value of the minimum value MIN of the pixel value. Is obtained, the DRMIN recovery circuit 3 based on the q supplied through the DRMIN recovery circuit 354, the motion flag MF, the correct value of the dynamic range DR, and the correct value of the minimum pixel value MIN.
54, and decodes the Y data, U data, and V data obtained as a result of the decoding together with an error flag set to “0” corresponding to the Y data, U data, and V data. , Block deshuffle circuit 3
To 56.

The ADRC decoding circuit 355 determines that the dynamic range DR or the minimum value MIN of the pixel value is missing, and the DRMIN recovery circuit 354 determines that the dynamic range DR has the correct value and the minimum value MIN of the pixel value.
When the correct value cannot be obtained, the error flag of the Y data, U data and V data corresponding to the code Q is set to "1", and the block deshuffle circuit 35 is set.
6

A recovery error determination flag Fd is based on emin1 and emin2 supplied from the q-bit MF recovery circuit 353.
Is output to the block deshuffle circuit 356, and the Y of the corresponding pixel of the previous frame stored in the external memory 305 is output.
Stop updating data, U data, and V data.

The block shuffle circuit 356 has the ADR
When the Y data, the U data, and the V data are supplied together with the error flag in which “0” is set from the C decoding circuit 355, the error flag, and the Y data, the U data, and the V data are stored. Block deshuffle circuit 35
6 stores an error flag when Y data, U data, and V data are supplied together with an error flag in which “1” is set from the ADRC decoding circuit 355, and stores Y data, U data, and V data. do not do. Further, even when a continuous error signal is supplied from the recovery error determination circuit 357 or the inter-segment deshuffle circuit 352, the block deshuffle circuit 356 outputs the external memory 305.
Stops updating of the Y data, U data, and V data of the corresponding pixel of the previous frame stored by.

Block deshuffle circuit 356 restores Y data, U data, and V data to the original sequence and the original sequence by the reverse operation of block shuffle circuit 102 of transmitting apparatus 1 shown in FIG. Y data, U data,
And V data are output.

As described above, even if the data q required for decoding, the motion flag MF, the dynamic range DR, or the minimum value MIN of the pixel value is lost, the decoding circuit 304 outputs the q-bit MF recovery circuit 353 and DRMIN
Since the recovery circuit 354 recovers the lost data, decoding can be performed.

If there is an error in the recovery, the pixel value stored in the external memory
Since the pixel value before the frame is output, when there is no motion,
It is possible to output a pixel value without a sense of discomfort.

Next, the processing timing of the decoding circuit 304 will be described with reference to the timing charts of FIGS. FIG. 35 shows an inter-segment deshuffle circuit 35.
4 shows an example of processing timing of the decoding circuit 304 in a case where there is no missing in the data output from the decoding circuit 304. The inter-segment deshuffle circuit 352 performs a deshuffle process, and outputs the deshuffled data to the ADRC decoding circuit 355 after a predetermined time has elapsed (in the example of FIG. 35, after 362 clocks from the start of input). The ADRC decoding circuit 355 decodes and outputs data supplied from the inter-segment shuffling circuit 352.

FIG. 36 shows an inter-segment deshuffle circuit 3.
An example of processing timing of the decoding circuit 304 in a case where there is a missing in the data output from the decoder 52 will be described. The inter-segment deshuffle circuit 352 performs deshuffle processing and, when there is no data loss, earlier than the timing of supplying data to the ADRC decoding circuit 355 (in the example of FIG. 36,
After 123 clocks from the input start), the descrambled data is output to the q-bit MF recovery circuit 353. q
The bit MF recovery circuit 353 supplies q and the motion flag MF to the DRMIN recovery circuit 354 at a predetermined timing.

The DRMIN recovery circuit 354 provides an inter-segment shuffle circuit 352 when there is no data loss.
Supplies the dynamic range and the minimum value of the pixel to the ADRC decoding circuit 355 at the same timing as when the data is supplied to the ADRC decoding circuit 355.

Therefore, even if the decoding circuit 304 recovers the q, the motion flag MF, the dynamic range DR, and the minimum value MIN of the pixel, the decoding circuit 304 has the same timing as that when the data output from the inter-segment shuffling circuit 352 has no missing data. Thus, the data can be decoded and output.

As described above, the decoding circuit 304 executes the decoding because the q-bit MF recovery circuit 353 and the DRMIN recovery circuit 354 recover the lost data even if the predetermined data necessary for decoding is lost. can do. When a signal for selecting a mode in which no error processing is performed (user selectable mode) is input to the decoding circuit 304, the q-bit MF recovery circuit 3
53 and the DRMIN recovery circuit 354 do not execute recovery.

FIGS. 37 (A) and (B) and FIG. 38
The data output timing of the block shuffle circuit 356 will be described with reference to FIGS.

In the methods shown in FIGS. 37 (A) and (B), 2
Two memories for storing a frame image are provided. FIG.
As shown in FIG. 37A, when data is written in one memory, data is read out from the other memory and output. When the processing for two frames of images is completed, as shown in FIG. , Writes data to the other memory, reads data from one memory, and outputs it.

On the other hand, the external memory 305 is provided with two banks (bank 1 and bank 2 in the figure) for storing images of two frames, respectively, and the block deshuffle circuit 356 operates as shown in FIG. As shown in FIG.
The image data corresponding to the DRC is transmitted to the ADRC decoding circuit 355.
38, the data is written to the bank 1 and FIG.
As shown in (B), when image data is not supplied from the ADRC decoding circuit 355 (for example, during the period from the start of input to 364 clocks shown in FIG. 36), by specifying an address, a block from the bank 2 can be obtained. Read the shuffled data. In the next two frames, the block shuffle circuit 356 uses three ADRCs.
Is supplied to the bank 2 when the image data corresponding to is supplied from the ADRC decoding circuit 355, and when the image data is not supplied from the ADRC decoding circuit 355, the block-shuffled data is read from the bank 1.

In this way, the image data can be block-shuffled and output to the block shuffle circuit 356 by one external memory 305.

FIG. 39 shows a recovery error determination circuit 357.
FIG. 3 is a block diagram showing the configuration of FIG. Evaluation value emi having the minimum value supplied from q-bit MF recovery circuit 353
n1 and the evaluation value emin having the second smallest value
2 is supplied to the difference calculation circuit 431. The difference calculation circuit 431 calculates the difference between the input emin1 and emin2 and outputs the difference to the delay circuit 432. Delay circuit 432
Delays the data supplied from the difference calculation circuit 431 for a predetermined time, and outputs the data to the adder 435 and the delay circuit 433.

The delay circuit 433 delays the data supplied from the delay circuit 432 for a predetermined time and outputs the data to the adder 435 and the delay circuit 434.

The delay circuit 434 delays the data supplied from the delay circuit 433 for a predetermined time and outputs the data to the adder 435 and the determination circuit 437.

[0220] The adder 435 adds the data supplied from the delay circuit 432, the data supplied from the delay circuit 433, and the data supplied from the delay circuit 434, and supplies the result to the determination circuit 436.

The determination circuit 436 determines whether or not the data supplied from the adder 435 is equal to or less than a predetermined threshold Te1, and determines whether the data supplied from the adder 435 is equal to the threshold T1.
If it is determined that it is equal to or less than e1, "1" is output to the AND circuit 438, and if it is determined that the data supplied from the adder 435 exceeds the threshold Te1, "0" is ANDed.
Output to the circuit 438.

The determination circuit 437 determines whether the data supplied from the delay circuit 434 is equal to or less than a predetermined threshold Te2, and determines that the data supplied from the delay circuit 434 is equal to or less than the threshold Te2. In this case, "1" is AND
When the data supplied from the delay circuit 434 exceeds the threshold Te2, “0” is output to the AND circuit 438.

When the data supplied from determination circuit 436 is “1” and the data supplied from determination circuit 437 is “1”, AND circuit 438 outputs “1” to hold circuit 439. The AND circuit 438 includes the determination circuit 4
When the data supplied from 36 is “0” or the data supplied from the determination circuit 437 is “0”, “0” is output to the hold circuit 439.

The hold circuit 439 holds the input data until a reset signal is input, and
Output to 0. The delay circuit 440 includes a hold circuit 439
Is delayed for a predetermined time and supplied to the hold circuit 439, and the recovery determination flag Fd
Is output to the block deshuffle circuit 356.

As described above, the recovery error determination circuit 35
7 is an evaluation value emi for three ADRC blocks.
The integrated value of the difference between n1 and the evaluation value emin2 is equal to or smaller than the threshold Te1 and the evaluation value e for the ADRC block
When the difference between min1 and the evaluation value emin2 is equal to or smaller than the threshold Te2, that is, the evaluation value emin1 and the evaluation value emin
2 and the recovered q or motion flag M
When it is highly probable that F is incorrect, 1 is set to the recovery determination flag Fd.

Next, a process in which the data decoded by the ADRC decoding circuit 355 is temporarily stored in the block shuffle circuit 356 and read out will be described. The inter-segment deshuffle circuit 352 sets “1” to an error flag corresponding to the missing data when the data supplied from the packet disassembly circuit 302 has a loss, and sets an error corresponding to the data when there is no loss. The flag is set to “0”, and the code Q and the error flag for each ADRC block are output to the ADRC decoding circuit 355. The ADRC decoding circuit 355 decodes the code Q and outputs Y data, U data, V data, and an error flag to the block shuffle circuit 356.

When the corresponding error flag is “0”, the block deshuffle circuit 356 outputs the Y data, the U data,
And V data and an error flag are stored in the external memory 30.
5, when the corresponding error flag is "1", the error flag is stored in the external memory 305, and the Y data, U data, and V data are not stored in the external memory 305.

The external memory 305 has two banks, and one bank stores two frames of Y data, U data, and V data, and an error flag.
The bank of the external memory 305 is switched every two frames. Therefore, when the corresponding error flag is “1”, the external memory 305 stores Y data, U data, and V data four frames before.

FIG. 40 shows a recovery error determination circuit 357.
FIG. 17 is a diagram for explaining the processing of the block shuffle circuit 356 when the recovery determination flag Fd in which “1” is set is output. evaluation values emin1 and 2 having the minimum value supplied from the q-bit MF recovery circuit 353 and 2
The difference between the evaluation value emin2 having the second smallest value is equal to or less than a predetermined value, and emin1 and emin of three consecutive ADRC blocks including the ADRC block.
When the sum of the differences of min2 is equal to or smaller than a predetermined value, the q-bit MF recovery circuit 353 determines that the recovery of q has failed, and therefore determines that the data extraction has also failed. At this time, the recovery error determination circuit 357 outputs “
A recovery determination flag Fd in which "1" is set is output.

The recovery error judgment circuit 357 sets "1"
Output the recovery determination flag Fd in which
The block deshuffle circuit 356 regards that the extraction of the data from the subsequent buffer (88 ADRC blocks) has failed, sets “1” to the error flag supplied thereafter, and sets the error flag to the outside. Memory 3
05. In the case of the user selectable escape mode, data is stored in the external memory 305.

FIG. 41 shows an inter-segment shuffle circuit 3.
FIG. 53 is a diagram for describing processing of the block shuffle circuit 356 when a continuous error signal is output. The inter-segment shuffling circuit 353 includes a corresponding even ADRC.
When detecting both the block and the odd ADRC block, a continuous error signal is output. q-bit MF
Since the recovery circuit 353 uses the correlation between the pixel values of adjacent pixels in the recovery processing of q and the motion flag MF, the corresponding even ADRC block and odd AD
When both RC blocks are missing, q bit MF
The recovery circuit 353 cannot execute recovery processing. When the continuous error signal is supplied, the block deshuffle circuit 356 transmits the subsequent buffer (88
It is considered that the data extraction from the ADRC blocks has failed, and the error flag supplied thereafter is set to "
1 "is set, the error flag is stored in the external memory 305, and the decoded data supplied thereafter is not stored in the external memory 305.

FIG. 42 is a diagram for explaining the processing of the block shuffle circuit 356 in the case of the user selectable gray mode. In the case of the gray mode, the block shuffle circuit 356 sets the gray (to make the corresponding data only 128 instead of making all Y, U, V 128) to the data whose corresponding error flag is “1”. Set and output.

FIGS. 43 and 44 are diagrams for explaining processing corresponding to a user-selectable recovery-off input (a signal input when a user operates a predetermined switch (not shown)) of the block deshuffle circuit 356. is there. "
When a recovery off input with 1 ”is supplied,
The block deshuffle circuit 356 includes the external memory 305
As shown in FIG. 43, the q-bit MF recovery circuit 3
When the Y data, U data, and V data are decoded by the ADRC decoding circuit 355 without passing through 53, the decoded Y data, U data, and V data are stored. As shown in FIG. 44, the q-bit MF recovery circuit 3
When the q and the motion flag are recovered at 53, the block deshuffle circuit 356 does not store the decoded Y data, U data, and V data in the external memory 305, and deletes all corresponding error flags. "1"
Set to. By changing the recovery off input, q
The effect of the recovery of the bit MF recovery circuit 353 and the like can be easily confirmed.

FIG. 45 shows a block deshuffle circuit 356.
FIG. 7 is a diagram illustrating a process corresponding to an escape signal (a signal input when a user operates a predetermined switch (not shown)). When the escape signal is supplied, as shown in FIG. 45, even if the recovery determination flag Fd is set to “1”, the block deshuffle circuit 356 forcibly sets the error flag supplied thereafter to “1”. The error flag is stored in the memory without setting "", and the decoded Y data, U data, and V data supplied thereafter are stored in the memory. By changing the escape signal, the effect of the concealing process of the block shuffle circuit 356 can be easily confirmed.

FIGS. 46 and 47 are flow charts for explaining the process of outputting the data decoded by the block shuffle circuit 356. In step S31, the inter-segment shuffling circuit 352 receives the error flag generated by the receiving circuit 301, and
5 The ADRC decoding circuit 355 outputs the error flag supplied from the inter-segment deshuffle circuit 352 to the block deshuffle circuit 356, and the block deshuffle circuit 356 receives the error flag.

In step S32, the block shuffle circuit 356 selects the first error flag corresponding to the buffer (predetermined 88 ADRC blocks) supplied from the ADRC decoding circuit 355. Step S
In 33, the block shuffle circuit 356 determines whether or not the continuous error signal supplied from the inter-segment shuffle circuit 352 is “0”, and when it is determined that the continuous error signal is “0”. , Step S34
Then, it is determined whether or not the recovery determination flag Fd supplied from the recovery error determination circuit 357 is “0”. If it is determined in step S34 that the recovery determination flag Fd is “0”, step S35
The block deshuffle circuit 356 causes the external memory 305 to store an error flag.

In step S33, when it is determined that the continuous error signal is "1", and in step S3
If it is determined in step 4 that the recovery determination flag Fd is "1", the data extraction has failed, and the process proceeds to step S47, where the block deshuffle circuit 356 is executed.
Determines whether or not the escape signal is off. If it is determined that the escape signal is off, step S
Going to 48, the error flag selected in step S32 is set to "1", and proceeding to step S35, the block shuffle circuit 356 stores the error flag in the external memory 305.

If it is determined in step S47 that the escape signal is on, the error flag is not set, so that the procedure skips step S48, proceeds to step S35, and proceeds to step S35.
56 causes the external memory 305 to store an error flag.

In step S36, the block shuffle circuit 356 determines whether or not all error flags corresponding to the buffer have been processed. If it is determined that all error flags have been processed, the process proceeds to step S37. , The block shuffle circuit 356 outputs the Y data, the U data, and the V data decoded from the ADRC decoding circuit 355.
Receive data.

If it is determined in step S36 that all the error flags in the buffer have not been processed yet, the process proceeds to step S49, where the block deshuffle circuit 3
56 selects the next error flag, returns to step S33, and repeats the process of setting the error flag.

In step S38, the block shuffle circuit 356 selects the first decoded data of the buffer supplied from the ADRC decoding circuit 355.

In step S39, the block shuffle circuit 356 outputs the inter-segment deshuffle circuit 352.
It is determined whether or not the continuous error signal supplied from is determined to be “0”. If the continuous error signal is determined to be “0”, the process proceeds to step S40, where the selected decoded data is It is determined whether the corresponding error flag is “0”. If it is determined that the error flag corresponding to the selected decoded data is “0”, the process proceeds to step S41, where the block deshuffle circuit 356 is executed.
Causes the external memory 305 to store the selected decoded Y data, U data, and V data. The procedure proceeds to step S42.

If it is determined in step S39 that the continuous error signal is "1",
In 40, if it is determined that the error flag corresponding to the selected decoded data is "1", the data is missing, so that step S41 is skipped and the process proceeds to step S42.

In step S42, the block shuffle circuit 356 determines whether or not all the decoded data in the buffer has been processed. If it is determined that all of the decoded data in the buffer has been processed, S43
Proceed to.

In the step S42, the buffer (88
If it is determined that all the decoded data of the ADRC blocks have not been processed yet, the process proceeds to step S50, the block deshuffle circuit 356 selects the next data, returns to step S39, and decodes the next data. The process of storing data is repeated.

At step S43, the block shuffle circuit 356 selects the first data stored in the external memory 305. In step S44, the block shuffle circuit 356 sets the error flag to “1”.
Then, it is determined whether or not the mode is the gray mode. If it is determined that the error flag is “0” or the mode is not the gray mode, the process proceeds to step S 45, and the selected data is output to the format conversion circuit 306. I do.

If it is determined in step S44 that the error flag is "1" and the mode is the gray mode, the flow advances to step S51 to execute the block deshuffle circuit 3
56 outputs the gray data to the format conversion circuit 306, and proceeds to step S46.

At step S46, the block shuffle circuit 356 determines whether all the data stored in the bank of the external memory 305 have been processed, and processes all the data stored in the external memory 305. If it is determined that the processing has been performed, the processing ends.

In step S46, the external memory 30
If it is determined that all the data stored in the external memory 305 have not been processed, the process proceeds to step S52, where the block shuffle circuit 356 selects the next data stored in the bank of the external memory 305, and proceeds to step S52. Returning to S44,
Repeat the process of outputting data.

As described above, the block shuffle circuit 35
No. 6 can output data with the least sense of discomfort even if the code Q is missing.

Next, a process of outputting data of the block deshuffle circuit 356 when a user-selectable recovery off switch (not shown) is "0" will be described with reference to the flowchart of FIG. In step S71, the inter-segment shuffle circuit 352
01 receives the error flag generated and supplies it to the ADRC decoding circuit 355. The ADRC decoding circuit 355 outputs the error flag supplied from the inter-segment deshuffle circuit 352 to the block deshuffle circuit 356, and the block deshuffle circuit 356 receives the error flag.

In step S72, the block shuffle circuit 356 determines that q and the motion flag are q bits M
In the F recovery circuit 353 and the like, it is determined whether or not the recovery has been performed. If it is determined that the q and the motion flag MF and the like have not been recovered, the process proceeds to step S73 to store an error flag in the external memory 305.

If it is determined in step S72 that q and the motion flag have been recovered, the flow advances to step S78, where the block shuffle circuit 356 sets all the received error flags to "1".
Go to 73.

At step S74, the block shuffle circuit 356 receives the decoded Y data, U data and V data from the ADRC decoding circuit 355.

In step S75, the block shuffle circuit 356 determines that q and the motion flag are q bits M
In the F recovery circuit 353 and the like, it is determined whether or not the recovery has been performed. If it is determined that the q and the motion flag MF have not been recovered, the process proceeds to step S76, where the Y data, U data, V
Store the data.

If it is determined in step S75 that q, the motion flag MF, and the like have been recovered, step S76 is skipped, and the Y data, U data, and V
The process proceeds to step S77 without storing the data.

At step S77, the block shuffle circuit 356 outputs the Y data, U data, V data and error flag stored in the external memory 305 to the format conversion circuit 306, and ends the processing.

As described above, when the recovery off input is "1", the block shuffle circuit 356 does not output the recovered data.

As described above, even if predetermined data necessary for decoding is lost, decoding circuit 304 executes decoding because q-bit MF recovery circuit 353 and DRMIN recovery circuit 354 recover lost data. can do. Further, even if decoding cannot be performed, the decoding circuit 304 can output data with the least discomfort. Also, by setting the recovery off input to “1”,
You can easily check the effect of recovery.

Next, the missing pixel creating circuit 308 will be described. When reproducing a progressive image, the missing pixel creation circuit 308 performs a classification adaptive process,
As shown in FIG. 49A, the pixel values of peripheral pixels in the same frame (frame t) of a pixel to be created, and one of the same positions in the horizontal and vertical directions as those pixels A pixel value of a missing pixel is created based on a pixel value of a pixel in a previous frame (frame t-1). Further, the missing pixel creating circuit 308 performs the class classification adaptive processing, as shown in FIG. 49B, to determine the pixel values of the peripheral pixels in the same field (field t) of the pixel to be created, The field immediately before the same position in the horizontal and vertical directions as the pixel (field t-1), and the two fields before the same position in the horizontal and vertical directions as the pixels around the pixel to be created. Based on the pixel value of the pixel in the field (field t-2),
Create pixel values for missing pixels.

FIG. 50 is a block diagram showing a configuration of the missing pixel creating circuit 308. The pixel value, which is the data input to the missing pixel creating circuit 308, and the error flag indicating the missing pixel are supplied to the preprocessing circuit 501 and the tap construction circuit 502-1.

The preprocessing circuit 501 generates the value of the missing pixel by a linear interpolation filter based on the input pixel value and the error flag indicating the missing pixel, and assigns the value to the missing pixel. The setting is supplied to the tap construction circuits 502-2 to 502-5. Note that the pixel value is also referred to as tap data.

The processing of the pre-processing circuit 501 will be described with reference to the flowchart of FIG. In step S131, the preprocessing circuit 501 determines whether or not the target pixel is missing based on the error flag. If it is determined that the target pixel is not missing, the process proceeds to step S13.
Proceeding to 2, the pixel value of the target pixel is set to the target pixel, and the process ends.

If it is determined in step S131 that the target pixel is missing, the process proceeds to step S133.
The pre-processing circuit 501 determines whether any of the pixels adjacent to the target pixel in the horizontal direction is missing based on the error flag. If it is determined in step S133 that none of the pixels adjacent to the target pixel in the horizontal direction is missing, the process proceeds to step S134, where the pre-processing circuit 501 determines whether the target pixel is adjacent to the target pixel in the horizontal direction. The average value of the pixel values of the two pixels is set as the pixel value of the target pixel, and the process ends.

If it is determined in step S133 that one of the pixels adjacent to the target pixel in the horizontal direction is missing, the process proceeds to step S135, where the preprocessing circuit 501 It is determined whether any horizontally adjacent pixels are missing. If it is determined in step S135 that any of the pixels adjacent to the target pixel in the horizontal direction is not missing, the process proceeds to step S136, where the preprocessing circuit 501 is executed.
Sets the pixel value of a non-missing pixel horizontally adjacent to the target pixel to the pixel value of the target pixel, and the process ends.

If it is determined in step S135 that none of the pixels adjacent to the target pixel in the horizontal direction is missing, the process proceeds to step S137, where the pre-processing circuit 501 sets It is determined whether or not any of the pixels adjacent to the target pixel in the vertical direction is missing. If it is determined in step S137 that none of the pixels vertically adjacent to the target pixel is missing, the process proceeds to step S138, where the pre-processing circuit 501 determines whether the target pixel is adjacent to the target pixel in the vertical direction. The average value of the pixel values of the two pixels is set as the pixel value of the target pixel, and the process ends.

If it is determined in step S137 that one of the pixels adjacent to the target pixel in the vertical direction is missing, the process proceeds to step S139, where the pre-processing circuit 501 determines whether or not the target pixel is defective based on the error flag. It is determined whether or not all the pixels adjacent to the target pixel are missing. If it is determined in step S139 that any of the pixels adjacent to the target pixel is not missing, the process proceeds to step S40, where the preprocessing circuit 501
The pixel value of the non-missing pixel adjacent to the target pixel is set as the pixel value of the target pixel, and the process ends.

If it is determined in step S139 that all pixels adjacent to the target pixel are missing, the process proceeds to step S41, where the pre-processing circuit 501
The pixel value of the pixel of the past frame at the same position as the target pixel is set to the pixel value of the target pixel, and the process ends.

As described above, the preprocessing circuit 501 linearly interpolates the pixel value of the target pixel from the pixel values of the peripheral pixels. By the interpolation processing by the preprocessing circuit 501, the range of taps that can be used in subsequent processing can be expanded.

Tap construction circuits 502-1 to 502-5
Respectively transmit the tap data and the like at a predetermined timing with the motion class generation circuit 503 and the DR class generation circuit 5.
04, the space class generation circuit 505, and the missing class generation circuit 506.

The motion class generation circuit 503 generates a motion class code and a static flag based on the parameters supplied from the initialization circuit 309, the error flag supplied from the tap construction circuit 502-1 and the selected tap. And tap construction circuits 502-2 through 50-50
2-5 and output to the class synthesis circuit 507. The motion class code has 2-bit information indicating the amount of motion,
The static motion flag indicates the presence / absence of motion with one bit.

FIG. 52A shows a motion class generation circuit 50.
3 is a diagram illustrating an example of a 3 × 3 pixel centered on a pixel to be created, which is used for calculating a time activity by 3; FIG. In FIG. 52A, “error” indicates a missing pixel. FIG. 52B is a diagram illustrating an example of 3 × 3 pixels of the immediately preceding frame corresponding to the pixels illustrated in FIG. FIG. 52 (A) and FIG. 52 (B)
L1 to L3 shown in FIG. 3 indicate lines, and the same numbers of the lines indicate the same positions in the vertical direction. H shown in FIGS. 52 (A) and 52 (B)
1 to H3 respectively indicate the horizontal position of the pixel,
The same number indicates the same position in the horizontal direction.

In the cases shown in FIGS. 52A and 52B, the time activity is calculated by equation (12).

Time activity = | (q2) − (p2) | + | (q3) − (p3) | + | (q4) − (p4) | + | (q6) − (p6) | + | (q7) − (P7) | + | (q9) − (p9) | (12) In Expression (12), () represents a pixel value of a pixel, and |
| Represents a function for obtaining an absolute value.

The motion class generation circuit 503 calculates the spatial activity by adding 1 to the difference between the maximum value and the minimum value of the 3 × 3 pixels centering on the pixel to be created.

FIG. 53 is a diagram showing an example of 3 × 3 pixels centered on a missing pixel to be created, which is used for calculating a spatial activity by the motion class generation circuit 503. At this time, the spatial activity is expressed by the formula (1)
It is calculated in 3).

Spatial activity = Max (qi) −Min (qi) +1 (13) In equation (13), Max (qi) is q1 to q9
Indicates the maximum pixel value of Min (qi) indicates the minimum value of the pixel values of q1 to q9.

The motion class generation circuit 503 selects thresholds having different values based on the spatial activity. The motion class generation circuit 503 includes a threshold for motion determination,
And a motion class code based on the time activity.

FIG. 54 is a diagram for explaining the threshold value for motion determination. A different value is used as the threshold value for the motion determination depending on the value of the spatial activity. As the spatial activity increases, a larger threshold value is used. This takes into account that when the spatial activity is large, the motion has at least a large value for the temporal activity.

In the processing for standard density data, a motion class code is set using a fixed threshold value regardless of spatial activity.

Next, the process of setting the motion class code of the motion class generation circuit 503 will be described with reference to the flowchart of FIG. In step S151, the motion class generation circuit 503 determines whether or not the time activity is equal to or smaller than the threshold value 1. When it is determined that the time activity is equal to or smaller than the threshold value 1, the process proceeds to step S152, and the motion class code is set. The value is set to 0, and the process ends.

If it is determined in step S151 that the time activity exceeds the threshold value 1, step S
Proceeding to 153, the motion class generation circuit 503 determines whether or not the time activity is equal to or less than the threshold 2, and when it is determined that the time activity is equal to or less than the threshold 2,
Proceeding to step S154, the motion class code is set to 1, and the process ends.

[0283] If it is determined in step S153 that the time activity exceeds threshold value 2, the process proceeds to step S153.
Proceeding to 155, the motion class generation circuit 503 determines whether or not the time activity is less than or equal to the threshold 3, and if it is determined that the time activity is less than or equal to the threshold 3, the process proceeds to step S156, where the motion class code is And the process ends.

If it is determined in step S155 that the time activity exceeds the threshold value 3, step S
Proceeding to 157, the motion class generation circuit 503 sets the motion class code to 3, and the process ends.

As described above, the motion class generation circuit 503
Sets a motion class code based on a threshold and time activity.

The motion class generation circuit 503 sets a motion flag again based on the motion class codes of a plurality of pixels. For example, as shown in FIG. 56, the motion class generation circuit 503 sets the motion class code of the target pixel based on the motion class codes of the pixels around the pixel to be created.

The process of setting a motion class code by the motion class generation circuit 503 will be described with reference to the flowchart in FIG. In step S171, the motion class generation circuit 503 determines that the number of motion class codes for which the value of 3 is set among the motion class codes of the seven pixels is
It is determined whether or not the number is larger than the threshold value 3. If it is determined that the number of motion class codes for which the value of 3 is set is larger than the threshold value 3, the process proceeds to step S172, and 3 is set as the motion class code. The process ends.

If it is determined in step S171 that the number of motion class codes for which a value of 3 is set is equal to or smaller than the threshold value 3, the process proceeds to step S173, where the motion class generation circuit 503 sets the value of 3 to It is determined whether or not a value obtained by adding the number of motion class codes that have been set and the number of motion class codes that have a value of 2 is greater than a threshold value 2, and a motion class code that has a value of 3 being set. Number of and 2
If it is determined that the value obtained by adding the number of motion class codes to which the value of “1” is set is larger than the threshold value 2, step S1
Proceeding to 74, the motion class code is set to 2, and the process ends.

When it is determined in step S173 that the value obtained by adding the number of motion class codes having the value of 3 and the number of motion class codes having the value of 2 is equal to or smaller than the threshold value 2. Proceeding to step S175, the motion class generation circuit 503 sets the number of motion class codes set to a value of 3, the number of motion class codes set to a value of 2, and the value of 1. It is determined whether the value obtained by adding the number of motion class codes is greater than a threshold value 1 or not, and the number of motion class codes for which a value of 3 is set,
If it is determined that the value obtained by adding the number of motion class codes to which the value of 2 is set and the number of motion class codes to which the value of 1 is added is larger than the threshold value 1, step S
Proceeding to 176, the motion class code is set to 1 and the process ends.

In step S175, the number of motion class codes set to a value of 3, the number of motion class codes set to a value of 2, and the number of motion class codes set to a value of 1 are calculated. If it is determined that the added value is equal to or smaller than the threshold value 1, the process proceeds to step S177, where the motion class generation circuit 503 sets 0 to the motion class code, and the process ends.

As described above, the motion class generation circuit 503
Sets a final motion class code based on the motion class codes of a plurality of pixels and a threshold value stored in advance.

As described above, the motion class generation circuit 503
Sets and outputs a motion class code from the pixel values of a plurality of pixels. The motion class generation circuit 503 sets and outputs a static flag based on the motion class code. For example, when the motion class code is 0 or 1, the static flag is set to 0, and when the motion class code is 2 or 3, the static flag is set to 1.

The tap construction circuit 502-2 calculates a prediction tap variable selection range including prediction taps of all classes based on the motion class code and the still / moving flag supplied from the motion class generation circuit 503 and the position of the missing pixel. , And the taps in the selected prediction tap variable selection range are supplied to the variable tap selection circuit 508.

The tap construction circuit 502-3 selects a class tap based on the motion class code and the static flag supplied from the motion class generation circuit 503, and selects the selected class tap together with the selected class tap. Is supplied to the DR class generation circuit 504 indicating whether or not is missing. The DR class generation circuit 504 includes:
Based on the class tap supplied from the tap construction circuit 502-3 and the error flag, a dynamic range is detected only from the class taps not missing, and a DR class code corresponding to the detected dynamic range is generated. Output to the class synthesis circuit 507.

The tap construction circuit 502-4 selects a class tap on the basis of the motion class code and the static flag supplied from the motion class generation circuit 503, and selects the selected class tap together with the selected class tap. Is supplied to the space class generation circuit 505 indicating whether or not is missing. The space class generation circuit 505 includes:
Based on the class taps supplied from the tap construction circuit 502-4 and the error flag, a space class code is generated only from the class taps that are not missing, and output to the class synthesis circuit 507. The space class generation circuit 5
In step 05, a space class code may be generated from all class taps.

Based on the motion class code and the still / moving flag supplied from the motion class generating circuit 503, the tap construction circuit 502-5 generates an error flag corresponding to the missing pixel and a surrounding area of the missing pixel. Is supplied to the missing class generation circuit 506. The missing class generation circuit 506 includes the tap construction circuit 50.
Based on the error flags corresponding to the missing pixels and the error flags corresponding to the pixels around the missing pixels supplied from 2-5, a missing class code is generated, and a class combining circuit 507 is generated. Output to

The class synthesizing circuit 507 integrates the class codes based on the motion class code, the static / dynamic flag, the DR class code, the space class code, and the missing class code to form a final class code, and Output to the holding class code selection circuit 509.

The coefficient holding class code selection circuit 509
A coefficient set supplied from the initialization circuit 309,
And a variable tap selection circuit 508 based on the prediction code and the class code supplied from the class synthesis circuit 507.
Then, a variable tap is selected, and a prediction coefficient sequence is output to the estimation prediction calculation circuit 510.

A prediction tap data sequence selected by the variable tap selection circuit 508 will be described with reference to FIGS.

FIG. 58 shows that when the motion class code is 0,
Tap construction circuit 502-2 is a variable tap selection circuit 508
FIG. 7 is a diagram showing an example of a prediction tap variable selection range output to the control unit and a prediction tap data string selected by the variable tap selection circuit 508. In FIG. 58, taps surrounded by solid lines indicate prediction tap variable selection ranges output by tap construction circuit 502-2, and taps surrounded by dotted lines indicate examples of prediction tap data strings selected by variable tap selection circuit 508. Show. The prediction tap data sequence selected by the variable tap selection circuit 508 includes taps included in the prediction tap variable selection range output by the tap construction circuit 502-2.

[0301] For example, when the motion class code is 0, the tap construction circuit 502-2 selects an adjacent tap. For example, when the motion class code is 0, all the variable tap selection circuits 508 select adjacent taps.

FIG. 59 shows that when the motion class code is 1,
FIG. 9 is a diagram illustrating a prediction tap variable selection range output by a tap construction circuit 502-2 and a prediction tap data sequence selected by a variable tap selection circuit 508. In FIG. 59, the taps surrounded by solid lines indicate the prediction tap variable selection range output by the tap construction circuit 502-2, and the taps surrounded by the dotted lines indicate prediction tap data strings selected by the variable tap selection circuit 508. The prediction tap data sequence selected by the variable tap selection circuit 508 includes taps included in the prediction tap variable selection range output by the tap construction circuit 502-2.

For example, when the motion class code is 1, the tap construction circuit 502-2 selects a horizontally adjacent tap and a tap that is not adjacent to the horizontally adjacent tap. For example, when the motion class code is 0,
The variable tap selection circuit 508 selects five adjacent taps and five horizontally adjacent taps that are separated from the five adjacent taps.

FIG. 60 shows a prediction tap variable selection range output by tap construction circuit 502-2 and a prediction tap data string selected by variable tap selection circuit 508 when a missing tap exists and the motion class code is 0. FIG.
The black circles in the figure indicate missing taps. In FIG. 60, taps surrounded by solid lines indicate prediction tap variable selection ranges output by tap construction circuit 502-2, and taps surrounded by dotted lines indicate prediction tap data strings selected by variable tap selection circuit 508. The prediction tap data sequence selected by the variable tap selection circuit 508 includes taps included in the prediction tap variable selection range output by the tap construction circuit 502-2.

For example, when a missing tap exists and the motion class code is 0, the tap construction circuit 502-2 selects an adjacent tap. For example, when a missing tap exists and the motion class code is 0, the variable tap selection circuit 508 determines the same number of adjacent taps (for example, a tap next to the missing tap or a missing tap as there are no missing taps). Tap in the same row as above).

FIG. 61 shows a prediction tap variable selection range output by the tap construction circuit 502-2 and a prediction tap data sequence selected by the variable tap selection circuit 508 when a missing tap exists and the motion class code is 1. FIG.
The black circles in the figure indicate missing taps. In FIG. 60, taps surrounded by solid lines indicate prediction tap variable selection ranges output by tap construction circuit 502-2, and taps surrounded by dotted lines indicate prediction tap data strings selected by variable tap selection circuit 508. The prediction tap data sequence selected by the variable tap selection circuit 508 includes taps included in the prediction tap variable selection range output by the tap construction circuit 502-2.

For example, when a missing tap exists and the motion class code is 1, the tap construction circuit 502-2 selects a horizontally adjacent tap and a tap that is not adjacent to the horizontally adjacent tap. I do. For example, when a missing tap exists and the motion class code is 0, the variable tap selection circuit 508 outputs the same number of horizontally adjacent taps (for example, taps next to the missing tap) as there are no missing taps. Or taps in the same row as the missing taps) and the same number of horizontally adjacent taps that are separated from the adjacent taps and that are the same as when there are no missing taps.

Based on the output mode set from the initialization circuit 309, the estimation prediction calculation circuit 510 calculates the prediction tap data string supplied from the variable tap selection circuit 508 and the prediction tap data string supplied from the coefficient holding class code selection circuit 509. Based on the coefficient sequence, the pixel value of the missing pixel is calculated using an estimation expression such as a linear estimation expression.

Thus, the missing pixel creation circuit 308
The missing pixel value is calculated based on the pixel values around the missing pixel by class classification based on the dynamic range, the movement, the lack, the change in the pixel value, and the like.

The estimation / prediction calculation circuit 510 increases the image quality of the input image (increases in gradation (increase in the number of bits of Y data, U data, and V data), removal of noise, and quantization distortion. (Including removal of distortion in the time direction), creation of quadruple-density resolution, etc.).

Next, the quadruple-density resolution creating circuit 312 will be described. When an interlaced image is input, the quadruple-density resolution creating circuit 312 applies the classification adaptive processing as shown in FIG. 62A and FIG. The created pixel is arranged at a position shifted by 1/4 in the vertical direction with respect to the position of and between the two fields before and after.

When a progressive image is input, the quadruple-density resolution creating circuit 312 applies the classification adaptive processing as shown in FIG. 62 (C) and FIG. 62 (D), The created pixel is arranged at the same position in the vertical direction with respect to the position of the pixel of the frame and between the two frames before and after.

The quadruple-density resolution creating circuit 312 uses the classification adaptive processing as shown in FIGS. 62 (C) and 62 (D) when outputting an image of the progressive system, as shown in FIG. 62. Then, two pixels are created between the two input frames based on the pixel values of a predetermined number of pixels in the two frames.

FIG. 63 is a block diagram showing the structure of the quadruple-density resolution creating circuit 312. Format conversion circuit 3
The Y data, U data, and V data supplied from 10 are the tap selection circuit 601, the tap selection circuit 602,
And a tap selection circuit 603. The tap selection circuit 601 selects a pixel used to calculate a pixel value of a pixel created to increase the density by four times, according to the tap position information stored in the register 609.
The pixel selected by the tap selection circuit 601 is supplied to the estimation prediction calculation circuit 607.

The tap selection circuit 602 includes a register 610
In the pixel values of the pixels in the vicinity of the pixel to be created, a pixel to be used for space class classification is selected according to the tap position information stored in. The pixel selected by the tap selection circuit 602 is supplied to the space class detection circuit 604. The space class detection circuit 604 detects a space class. The detected space class is supplied to the class synthesis circuit 606.

The tap selection circuit 603 includes a register 611
According to the position information of the tap stored in, a pixel used for class classification corresponding to the motion is selected based on pixels in the vicinity of the pixel to be created. The pixel selected by the tap selection circuit 603 is the motion class detection circuit 6
05. The motion class detection circuit 605 detects a motion class. The detected motion class is supplied to the class synthesis circuit 606 and the register 610. The tap position information of the register 610 is switched according to the motion class supplied from the motion class detection circuit 605.

[0317] The class synthesis circuit 606 integrates the space class and the motion class to form a final class code.

The class code is supplied as an address to the coefficient memory 612, and a coefficient set corresponding to the class code is read from the coefficient memory 612. The coefficient set is supplied to the estimation prediction operation circuit 607. The estimation prediction operation circuit 607 calculates the pixel value of the pixel to be created using a linear estimation formula based on the pixel value of the pixel supplied from the tap selection circuit 601 and the coefficient set supplied from the coefficient memory 612. . Estimation prediction operation circuit 607
Output from line data L1 and line data L
2 is supplied to the line-sequential conversion circuit 613.

[0319] The coefficient set is Y data including 528 pixels in the horizontal direction and 240 pixels in the vertical direction per field as a teacher signal, and 176 pixels in the horizontal direction and 1 pixel in the vertical direction per field.
Using U data and V data including 20 pixels, a coefficient set for each class is generated in advance and stored in the initialization circuit 313.

The line-sequential conversion circuit 613 has a line memory, and converts the line data L1 and L2 output from the estimation / prediction calculation circuit 607 into line-sequential data based on the output timing data stored in the register 608. Output. The line-sequential conversion circuit 613 outputs image data having a density four times that of the supplied image.

Register 608, register 609, register 610, register 611, and coefficient memory 612
Stores the coefficient set and the like supplied from the initialization circuit 313 when the initialization signal is supplied to the initialization circuit 313.

FIG. 64 shows pixels (large circles in the figure) input to the quadruple-density resolution creation circuit 312 when the quadruple-density resolution creation circuit 312 generates and outputs quadruple-density resolution pixels in an interlaced manner. Position), and 4
FIG. 11 is a diagram illustrating the position of a pixel (indicated by a small circle in the figure) output from the double-density resolution creating circuit 312. The frame input to the quad-density resolution creating circuit 312 has a frequency of 30 Hz, and the field output by the quad-density resolution creating circuit 312 has a frequency of 60 Hz. Pixels of the first field to be output are output shifted from the input pixels by one line of the screen line output in the vertical direction, and the number of pixels per line is doubled. Has become. The output pixels of the second field are shifted downward from the input pixels by one line of the screen line output in the vertical direction, and the number of pixels per line is doubled. Has become.

FIG. 65 shows that the quadruple-density resolution creating circuit 312 generates pixels of quadruple-density resolution in a progressive manner,
The position of the pixel (indicated by a large circle in the figure) input to the quadruple-density resolution creation circuit 312 when outputting, and the pixel (indicated by a small circle in the figure) output by the quadruple-density resolution creation circuit 312 FIG. The frame input to the quad-density resolution creating circuit 312 has a frequency of 30 Hz, and the frame output from the quad-density resolution creating circuit 312 has a frequency of 60 Hz. The pixels of the output frame are output on the same line as the input pixels, and 1
The number of pixels per line has doubled.

As described above, the quadruple-density resolution creation circuit 31
2 creates and outputs an image having twice the number of pixels per line at twice the frame frequency and twice the field frequency based on the input image.

Next, the UV vertical doubling circuit 333 will be described. FIG. 66 is a block diagram showing a configuration of the UV vertical doubling circuit 333. The UV vertical double circuit 333 has 4
The double-density resolution creation circuit 312 receives U data and V data whose number of pixels has been increased. The input data is delayed one line at a time by the line FIFOs 701-1 to 701-6, and sequentially output to the subsequent stage.

The multiplication circuit 702-1 multiplies the input data by a coefficient and outputs the result to the addition circuit 703.

The multiplication circuits 702-2 to 702-7 are provided by line FIFOs 701-1 to 701-6, respectively.
The input data is multiplied by a coefficient and output to the adding circuit 703.

The addition circuit 703 adds the data supplied from the multiplication circuits 702-1 to 702-7 and supplies the result to the latch circuit 704. The latch circuit 704 latches input data when a clock signal is input.

The operation of the UV vertical doubling circuit 333 will be described with reference to FIG. Line FIFO 701-1 to 70
1-6 outputs data of pixels (pixels indicated by circles in FIG. 67) at the same position in the horizontal direction of the screen on six lines continuous with the input line. Multiplication circuit 702-1
702 to 702-7 multiply the input data by respective coefficients and output the result to the addition circuit 703. Adder circuit 703
Add the output data and output the result to the latch circuit 704. A clock signal is input to the latch circuit 704 at a timing when the number of lines is doubled (a timing indicated by a black triangle in FIG. 67). Therefore, the UV vertical doubling circuit 333 doubles the number of input U data and V data lines and outputs the result.

Next, the receiving process of the receiving apparatus 2 will be described with reference to the flowchart of FIG. Step S201
, The packet disassembly circuit 302
Decomposes the received packet and supplies it to the decoding circuit 304. In step S202, the decoding circuit 304 converts the data input from the packet
Decoding is performed using the C method. When data is lost during transmission, in decoding by the decoding circuit 304, q, the motion flag MF, the dynamic range DR, or the minimum value MIN of the pixel value is recovered, and the Y data, U data, and V data are decoded. .

[0331] In step S203, the format conversion circuit 306 converts the Y data, U data, and V data supplied from the decoding circuit 304 into a predetermined format that can be processed by the missing pixel creation circuit 308. In step S204, the missing pixel creating circuit 308
Pixels that cannot be decoded by the decoding circuit 304 among the missing pixels are created by class adaptation processing.

In step S205, the format conversion circuit 310 converts the Y data, U data, and V data supplied from the missing pixel creation circuit 308 into a predetermined format that can be processed by the quad density resolution creation circuit 312. In step S206, the quadruple-density resolution creation circuit 312 creates pixels by class adaptation processing based on the input Y data, U data, and V data, quadruples the pixel density, and interpolates the pixels. Output to

In step S207, the format conversion circuit 331 changes the data format of the Y data, U data, and V data supplied from the quad-density resolution creation circuit 312 so that a predetermined interpolation process can be performed. .
In step S208, the UV vertical double circuit 333
Is to interpolate the pixels of U data and V data in the vertical direction so that the number of lines is doubled. Step S20
At 9, the D / A conversion circuit 335 performs digital-to-analog conversion at a predetermined clock rate and outputs an analog Y signal, an analog U signal, and an analog V signal, and the process ends.

As described above, the receiving apparatus 2 creates a missing pixel when a pixel is missing, and removes noise or removes a pixel when the pixel is not missing. , Can create and output high quality pixels.

Next, generation of a prediction coefficient used in the missing pixel creating circuit 308 will be described.

FIG. 69 is a block diagram for explaining a learning device for generating a prediction coefficient used in the missing pixel creating circuit 308. The image input to the learning device is input to a noise adding circuit 801, a selection circuit 802, and a normal equation operation circuit 812. The error flag is output from the selection circuit 802, the preprocessing circuit 803, and the tap construction circuit 80.
4-1 to 804-5. The error flag is
In accordance with the content of the learning of the learning device, it is set so as to appropriately indicate the lack.

[0337] The noise addition circuit 801 adds noise to the pixels of the input image, and supplies the pixels with the noise added to the selection circuit 802. The selection circuit 802 selects one of a pixel input to the learning device and a pixel to which noise supplied from the noise addition circuit 801 is added based on the error flag, and converts the selected pixel into a pre-processing circuit. 803 and the tap construction circuit 804-1. For example, when indicating that the error flag is missing, the selection circuit 802 selects a pixel input to the learning device, and supplies the selected pixel to the preprocessing circuit 803 and the tap construction circuit 804-1. When the selection circuit 802 indicates that the error flag is not missing, the selection circuit 802 selects a pixel to which noise is added, and selects the selected pixel with the pre-processing circuit 8.
03 and the tap construction circuit 804-1.

The preprocessing circuit 803 generates a missing pixel value by a linear interpolation filter based on the input pixel value and the error flag indicating the missing pixel.
The value is set for the pixel that is determined to be missing and supplied to the tap construction circuits 804-2 to 804-5. The preprocessing circuit 803 performs the same processing as the preprocessing circuit 501, and a detailed description of the processing will be omitted.

[0339] Tap construction circuits 804-1 to 804-5
Respectively transmit the tap data and the like at a predetermined timing with the motion class generation circuit 805 and the DR class generation circuit 8.
06, the space class generation circuit 807, and the missing class generation circuit 808.

A motion class generation circuit 805 generates a motion class code and a static flag based on the parameters supplied from the initialization circuit 814, the error flag supplied from the tap construction circuit 804-1 and the selected tap. And tap construction circuits 804-2 to 804-2
4-5 and output to the class synthesis circuit 809. Since the motion class generation circuit 805 performs the same processing as the motion class generation circuit 503, detailed description of the processing is omitted.

The tap construction circuit 804-2 calculates a prediction tap variable selection range including prediction taps of all classes based on the motion class code and the static flag supplied from the motion class generation circuit 805 and the position of the missing pixel. , And the taps in the selected prediction tap variable selection range are supplied to the variable tap selection circuit 810.

The tap construction circuit 804-3 selects a class tap based on the motion class code and the static flag supplied from the motion class generation circuit 805, and selects the selected class tap together with the selected class tap. Is supplied to the DR class generation circuit 806 indicating whether or not is missing. The DR class generation circuit 806
Based on the class tap supplied from the tap construction circuit 804-3 and the error flag, a dynamic range is detected only from the class taps not missing, and a DR class code corresponding to the detected dynamic range is generated. Output to the class synthesis circuit 809.

The tap construction circuit 804-4 selects a class tap based on the motion class code and the static flag supplied from the motion class generation circuit 805, and selects the selected class tap together with the selected class tap. Is supplied to the space class generation circuit 807 indicating whether or not is missing. The space class generation circuit 807
Based on the class taps supplied from the tap construction circuit 804-4 and the error flag, a space class code is generated only from the class taps that are not missing and output to the class synthesis circuit 809.

The tap construction circuit 804-5 determines the error flag corresponding to the missing pixel and the vicinity of the missing pixel based on the motion class code and the static flag supplied from the motion class generation circuit 805. Are supplied to the missing class generation circuit 808. The missing class generation circuit 808 includes the tap construction circuit 80.
Based on the error flags corresponding to the missing pixels and the error flags corresponding to the pixels around the missing pixels supplied from 4-5, a missing class code is generated, and a class combining circuit 809 is generated. Output to

The class synthesizing circuit 809 integrates the class codes on the basis of the motion class code, the static / dynamic flag, the DR class code, the space class code, and the missing class code to form a final class code. Output to the code selection circuit 811.

[0346] The class code selection circuit 811 is based on the prediction structure supplied from the initialization circuit 814 and the class code supplied from the class synthesis circuit 809.
The variable tap selection circuit 810 selects a variable tap, and outputs a class code to the normal equation operation circuit 812.

The normal equation operation circuit 812 determines whether the image as the teacher data and the predicted tap data sequence as the learning data are input to the variable tap selection circuit 8 based on the output mode set from the initialization circuit 814.
When supplied from 10, a prediction coefficient W that minimizes the error is calculated by using the values and by the least square method.

Here, the prediction coefficient W calculated by the normal equation operation circuit 812 will be briefly described.

For example, the prediction value E [y] of the pixel value y of the input original image which is the teacher data is now converted to the pixel values x ₁ , x ₂ ,.
And a predetermined linear prediction model defined by a linear combination of predetermined prediction coefficients w ₁ , w ₂ ,... In this case, the predicted value E [y] can be expressed by the following equation.

E [y] = w ₁ x ₁ + w ₂ x ₂ +... (14)

Therefore, for generalization, a matrix W composed of a set of prediction coefficients w, a matrix X composed of a set of learning data, and a matrix Y ′ composed of a set of predicted values E [y],

(Equation 6) Defines the following observation equation.

XW = Y '(15)

Then, it is considered that a least square method is applied to this observation equation to obtain a predicted value E [y] close to the pixel value y of the original image. In this case, a matrix Y consisting of a set of pixel values y of the original image and a matrix E consisting of a set of residuals e of predicted values E [y] for the pixel values y of the original image are given by:

[0354]

(Equation 7) From equation (15), the following residual equation is established.

XW = Y + E (16)

In this case, the prediction coefficient w _i for obtaining a prediction value E [y] close to the pixel value y of the original image is represented by a square error

(Equation 8) Can be obtained by minimizing.

Therefore, when the above-mentioned square error is differentiated by the prediction coefficient w _i , the result becomes 0, that is, the prediction coefficient w _i that satisfies the following equation is a prediction value E [y that is close to the pixel value y of the original image. ]
Is the optimum value.

[0358]

(Equation 9) ... (17)

Then, first, the equation (16) is calculated by using the prediction coefficient w
By differentiating with _i , the following equation is established.

[0360]

(Equation 10) ... (18)

From equations (17) and (18), equation (1)
9) is obtained.

[0362]

[Equation 11] ... (19)

Further, considering the relationship among learning data x, prediction coefficient w, teacher data y, and residual e in the residual equation of equation (16), the following normal equation is obtained from equation (19). be able to.

[0364]

(Equation 12) ... (20)

The normal equation of equation (20) can be established by the same number as the number of prediction coefficients w to be obtained.
By solving the equation (20), the optimum prediction coefficient w can be obtained. In solving equation (20), for example, a sweeping-out method (Gauss-Jordan elimination method) or the like can be applied.

[0366] That is, for example, now, the pixel value of the prediction tap data is the learning _{data, x 1, x 2, x} 3, and ..., prediction coefficients W to be obtained _{w 1, w 2, w 3} , ..., the linear linear combination of
To determine the pixel value y of a pixel, the prediction coefficients w ₁ , w ₂ ,
It is necessary that w ₃ ,... satisfy the following expression.

Y = w ₁ x ₁ + w ₂ x ₂ + w ₃ x ₃ +...

Thus, the normal equation operation circuit 812
From a prediction tap of the same class and a pixel of the corresponding teacher data, a prediction value w ₁ x ₁ + w ₂ x ₂ + w ₃ for a true value y
The prediction coefficient w ₁ , which minimizes the square error of x ₃ +.
w _2, w _3, ··· are obtained by solving the normal equation shown in the above Expression (20).

Accordingly, by performing this processing for each class, a prediction coefficient W is generated for each class.

The prediction coefficient W for each class obtained by the normal equation operation circuit 812 is supplied to the coefficient memory 813 together with the class code. As a result, in the coefficient memory 813, the prediction coefficient W from the normal equation operation circuit 812 is stored at the address corresponding to the class indicated by the class code.

As described above, the learning device can generate the coefficients used by the missing pixel creating circuit 308.

Next, the process of generating the prediction coefficient W by the learning device will be described with reference to the flowchart in FIG. In step S301, the missing class generation circuit 808 of the learning device determines whether or not the target pixel is missing based on the error flag. If it is determined that the target pixel is missing, the process proceeds to step S302. , The class synthesis circuit 809 generates the motion class code supplied from the motion class generation circuit 805, the DR class code supplied from the DR class generation circuit 806, the space class code supplied from the space class generation circuit 807, and the missing class generation. Based on the missing class code supplied from the circuit 808, a class code corresponding to the missing pixel of interest is generated, and the process proceeds to step S304.

If it is determined in step S301 that the pixel of interest is not missing, the process proceeds to step S303, where the class synthesis circuit 809 sets the motion class generation circuit 80
5 based on the motion class code supplied from the DR class generation circuit 806, the space class code supplied from the space class generation circuit 807, and the missing class code supplied from the missing class generation circuit 808. Then, a class code corresponding to the normal pixel of interest is generated, and the process proceeds to step S304.

[0374] In step S304, the variable tap selection circuit 810 selects a prediction tap corresponding to the class indicated by the class code.

[0375] In step S305, the normal equation calculation circuit 812 generates a normal equation corresponding to the class code. In step S306, the normal equation calculation circuit 812 sets the input original pixels that are the teacher data and the prediction taps that are the learning data to the generated normal equation, and solves the prediction coefficient by solving the normal equation.
Calculate W.

In step S307, the normal equation calculation circuit 812 sets the prediction coefficient
W is stored in the coefficient memory 813, and the process ends.

The prediction coefficient W stored in the coefficient memory 813
Is the initialization circuit 30 of the missing pixel creation circuit 308
9 is loaded into the coefficient holding class code selection circuit 509 when the receiving apparatus 2 starts up, for example.

As described above, based on the prediction coefficient W generated by the learning device, the receiving device 2
Create missing pixels, when no pixels are missing, noise removed or higher resolution, etc.
High quality pixels can be created and output.

Although the image transmission system has been described as communicating an image, it is not limited to an image but may transmit data such as audio.

The above-described series of processes can be executed by hardware, but can also be executed by software. When a series of processing is executed by software, a program constituting the software can execute various functions by installing a computer built into dedicated hardware or installing various programs. It is installed from a recording medium into a possible general-purpose personal computer or the like.

FIG. 71 is a diagram illustrating an example of a recording medium and a computer. CPU (Central Processing Uni
t) 901 indicates various application programs and OS
(Operating System) is actually executed. ROM (Read-on
ly Memory) 902 generally stores basically fixed data of programs used by the CPU 901 and parameters for calculation. RAM (Random-Access Memory)
Reference numeral 903 stores a program used in the execution of the CPU 901 and parameters that change as appropriate in the execution. These are interconnected by a host bus 904 including a CPU bus and the like.

The host bus 904 is connected to a PCI (Peripheral Component Interconnect / Interf
ace) bus or the like.

The keyboard 908 is operated by the user when inputting various commands to the CPU 901. The mouse 909 is operated by the user when instructing or selecting a point on the screen of the display 910. The display 910 is a liquid crystal display device or a CRT (Cathode Ray
Tube) and displays various information as text or images. An HDD (Hard Disk Drive) 911 drives a hard disk and records or reproduces a program or information executed by the CPU 901 on the hard disk.

The drive 912 includes the mounted magnetic disk 941, optical disk 942, and magneto-optical disk 94.
3 or data or a program recorded in the semiconductor memory 944, and reads the data or the program into the interface 907, the external bus 906,
The data is supplied to the bridge 905 and the RAM 903 connected via the host bus 904.

The keyboard 908 to the drive 9
12 is connected to the interface 907, and the interface 907 is connected to the external bus 906 and the bridge 9.
05 and the CPU 901 via the host bus 904.

As shown in FIG. 71, the recording medium is a magnetic disk 941 (including a floppy disk) storing the program and an optical disk 942 which are distributed separately from the computer to provide the program to the user.
(CD-ROM (Compact Disc-Read Only Memory), DVD (Dig
ital Versatile Disc), magneto-optical disc 943
(Including MD (Mini-Disc)) or semiconductor memory 9
ROM 9 storing a program, which is provided to a user in a state in which it is not only configured by package media such as
02 and the HDD 911.

[0387] In this specification, the step of describing a program stored in a recording medium is not limited to processing performed in a chronological order in the described order. This also includes processing executed in parallel or individually.

In this specification, the system is
It represents the entire device composed of a plurality of devices.

[0389]

According to the information processing apparatus according to the first aspect, the information processing method according to the eleventh aspect, and the recording medium according to the twenty-first aspect, based on a plurality of data around the target data, The data of interest is classified into one of a plurality of preset classes, conversion information corresponding to the classified class is selected, and the data of interest is converted into higher-quality data based on the conversion information. In the above, the attention data is classified into different classes depending on whether or not the attention data is missing, so that higher quality data can be generated regardless of whether or not the attention data is missing. Become like

[0390] According to the learning apparatus of the twenty-second aspect and the learning method of the twenty-third aspect, the attention data is assigned to one of a plurality of classes set in advance based on the plurality of data around the attention data. Classified and according to the class,
Conversion information for converting the data of interest into higher-quality data is generated, and in the class classification, the data of interest is classified into different classes depending on whether or not the data of interest is missing. By using the conversion information, higher-quality data can be generated regardless of whether or not the target data is missing.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating an image transmission system for transmitting an image.

FIG. 2 is a block diagram illustrating a configuration of an embodiment of a transmission device 1 according to the present invention.

FIG. 3 is a diagram illustrating a process of compressing an image signal input to the transmission device 1.

FIG. 4 is a block diagram illustrating a configuration of a thinning filter 43.

FIG. 5 is a diagram illustrating an output of a latch circuit 81.

FIG. 6 is a diagram illustrating an output of a latch circuit 84.

FIG. 7 is a block diagram showing a configuration of a UV vertical half circuit 45.

FIG. 8 is a diagram illustrating an output of a latch circuit 94.

FIG. 9 is a block diagram showing a configuration of an encoding circuit 15;

FIG. 10 is a diagram illustrating a unit of processing. .

FIG. 11 is a diagram illustrating an ADRC block.

FIG. 12 is a diagram illustrating a relationship between pixels in a frame and an ADRC block.

FIG. 13 is a diagram illustrating a relationship between pixels in a frame and an ADRC block.

FIG. 14 is a diagram illustrating a relationship between pixels in a frame and an ADRC block.

FIG. 15 is a diagram illustrating an ADRC block and a segment of Y data.

FIG. 16 is a diagram illustrating ADRC blocks and segments of U data or V data.

FIG. 17 is a diagram illustrating shuffling performed in ADRC block units.

FIG. 18 is a diagram illustrating segments.

19 is a diagram illustrating a configuration of an ADRC encoding circuit 103. FIG.

FIG. 20 is a diagram illustrating a configuration of an information amount control circuit 164.

FIG. 21 is a diagram illustrating a configuration of an information amount control circuit 164.

FIG. 22 is a diagram for explaining how to shuffle an inter-segment shuffle circuit 104;

FIG. 23 is a diagram for explaining how to shuffle the inter-segment shuffle circuit 104;

FIG. 24 is a diagram illustrating how to shuffle an inter-segment shuffle circuit 104;

FIG. 25 is a diagram illustrating how to shuffle an inter-segment shuffle circuit 104;

FIG. 26 is a diagram illustrating a configuration of a packet.

FIG. 27 is a diagram illustrating a process of transmitting an image by the transmission device 1.

FIG. 28 is a block diagram illustrating a configuration of a receiving device 2.

FIG. 29 is a diagram illustrating a process of expanding an image.

30 is a block diagram of a decoding circuit 304. FIG.

FIG. 31 is a block diagram showing a configuration of a q-bit MF recovery circuit 353.

FIG. 32 is a diagram illustrating a process of recovering a dynamic range DR or a minimum pixel value MIN.

FIG. 33 is a diagram illustrating a process of recovering the dynamic range DR or the minimum value MIN of the pixel value.

FIG. 34 is a diagram illustrating a process of recovering the dynamic range DR or the minimum value MIN of the pixel value.

FIG. 35 is a diagram illustrating an example of processing timing of the decoding circuit 304 when data output from the inter-segment shuffling circuit 352 has no loss.

FIG. 36 is a diagram illustrating an example of processing timing of the decoding circuit when data output from the inter-segment shuffling circuit is missing;

FIG. 37 is a diagram showing a method of storing data.

FIG. 38 is a diagram for explaining the timing of outputting data of the block shuffle circuit 356.

FIG. 39 is a block diagram showing a configuration of a recovery error determination circuit 357.

FIG. 40 is a diagram illustrating a case where the recovery error determination circuit 357 outputs a recovery determination flag Fd in which “1” is set.
FIG. 9 is a diagram for explaining processing of a block shuffle circuit 356.

FIG. 41 is a diagram illustrating processing of the block shuffle circuit 356 when the inter-segment shuffle circuit 353 outputs a continuous error signal.

FIG. 42 is a diagram illustrating processing of the block shuffle circuit 356 in the gray mode.

FIG. 43 is a diagram illustrating a process corresponding to a recovery off input of the block deshuffle circuit 356.

FIG. 44 is a diagram illustrating a process corresponding to a recovery off input of the block deshuffle circuit 356.

FIG. 45 is a diagram illustrating processing corresponding to an escape signal of the block shuffle circuit 356.

FIG. 46 is a flowchart illustrating a process of outputting data decoded by the block shuffle circuit 356.

FIG. 47 is a flowchart illustrating a process of outputting data decoded by the block shuffle circuit 356.

FIG. 48 is a flowchart illustrating a process of outputting data of the block shuffle circuit 356 when the recovery off input is “1”.

FIG. 49 is a diagram illustrating pixels used in processing of a missing pixel creation circuit 308.

FIG. 50 is a block diagram showing a configuration of a missing pixel creating circuit 308.

FIG. 51 is a flowchart illustrating processing of a preprocessing circuit 803.

FIG. 52 is a diagram illustrating an example of a pixel used for calculating a time activity.

FIG. 53 is a diagram illustrating an example of pixels used for calculating a spatial activity.

FIG. 54 is a diagram illustrating a threshold for motion determination.

FIG. 55 is a flowchart illustrating a process of setting a motion class code of the static / motion determining circuit 521.

FIG. 56 is a diagram illustrating an example of pixels used for majority determination of a motion class code.

FIG. 57 is a flowchart illustrating a process of setting a motion class code of the motion class generation circuit 805.

FIG. 58 is a diagram illustrating a predicted tap data sequence selected by a variable tap selection circuit 810.

FIG. 59 is a diagram illustrating a predicted tap data sequence selected by a variable tap selection circuit 810.

FIG. 60 is a diagram illustrating a predicted tap data sequence selected by a variable tap selection circuit 810.

FIG. 61 is a diagram illustrating a predicted tap data sequence selected by a variable tap selection circuit 810.

FIG. 62 is a view for explaining the positions of pixels used in the processing of the quad-density resolution creating circuit 312;

FIG. 63 is a block diagram showing a configuration of a quad resolution resolution creating circuit 312.

FIG. 64 shows the positions of pixels input to the quadruple-density resolution creation circuit 312 and the quadruple-density resolution creation circuit 31 when the quadruple-density resolution creation circuit 312 outputs in an interlaced manner.
FIG. 2 is a diagram showing the position of a pixel output from the pixel 2;

FIG. 65 illustrates the positions of pixels input to the quadruple-density resolution creating circuit 312 and the quadruple-density resolution creating circuit 31 when the quadruple-density resolution creating circuit 312 outputs in a progressive manner.
FIG. 2 is a diagram showing the position of a pixel output from the pixel 2;

FIG. 66 is a block diagram showing a configuration of a UV vertical doubling circuit 333.

FIG. 67 is a diagram illustrating an output of the latch circuit 704.

FIG. 68 is a flowchart illustrating a reception process of the reception device 2.

FIG. 69 is a block diagram illustrating a learning device.

FIG. 70 is a flowchart illustrating a coefficient generation process.

FIG. 71 is a diagram illustrating an example of a recording medium and a computer.

[Explanation of symbols]

1 transmitter, 2 receiver, 13 decimation unit,
15 encoding circuit, 17 packetizing circuit, 42 A
/ D conversion circuit, 43 thinning filter, 45 UV vertical half circuit, 102 block shuffle circuit, 1
03 ADRC encoding circuit, 104 inter-segment shuffle circuit, 164 information amount control circuit, 171 A
DRC encoder, 302 packet disassembly circuit, 3
04 decoding circuit, 305 external memory, 308 missing pixel creation circuit, 3124 times dense resolution creation circuit, 33
3 UV vertical doubling circuit, 335 D / A conversion circuit,
352 inter-segment deshuffle circuit, 353 q-bit MF recovery circuit, 354 DRMIN recovery circuit, 355 ADRC decoding circuit, 356 block deshuffle circuit, 357 recovery error determination circuit, 501 preprocessing circuit, 502-1 to 502-5
Tap construction circuit, 503 Motion class generation circuit,
504 DR class generation circuit, 505 space class generation circuit, 506 missing class generation circuit, 507 class synthesis circuit, 508 variable tap selection circuit, 50
9 Coefficient holding class code selection circuit, 510 estimation prediction operation circuit, 801 noise addition circuit, 802 selection circuit, 803 preprocessing circuit, 804-1 to 80
4-5 tap construction circuit, 805 motion class generation circuit, 806 DR class generation circuit, 807 space class generation circuit, 808 missing class generation circuit, 809
Class synthesis circuit, 810 variable tap selection circuit,
811 class code selection circuit, 812 normal equation operation circuit, 813 coefficient memory, 901 CPU,
902 ROM, 903 RAM, 911 HDD, 941
Magnetic disk, 942 optical disk, 943 magneto-optical disk, 944 Semiconductor memory

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Tsutomu Watanabe 6-7-35 Kita-Shinagawa, Shinagawa-ku, Tokyo Inside Sony Corporation (72) Inventor Hideki Otsuka 6-35, Kita-Shinagawa, Shinagawa-ku, Tokyo Inside Sony Corporation (72) Inventor Yasuaki Takahashi 6-7-35 Kita-Shinagawa, Shinagawa-ku, Tokyo Sony Corporation (72) Inventor Seiji Wada 6-35, Kita-Shinagawa, Shinagawa-ku, Tokyo Soni (Inc.) (72) Inventor Takahiro Nagano 6-35, Kita-Shinagawa, Shinagawa-ku, Tokyo Sony Corporation (72) Inventor Koji Ota, 6-35-35 Kita-Shinagawa, Shinagawa-ku, Tokyo Sony Corporation Inside the company (72) Inventor Juichi Shiraki 6-35, Kita-Shinagawa, Shinagawa-ku, Tokyo F-term in Sony Corporation (reference) 5C021 PA36 RB08 SA25 YA06 YC00 5C059 MA28 NN23 RF09 SS20 TA68 TC41 UA05 UA 11 UA39

Claims (23)

[Claims] 1. Classification means for classifying the data of interest into one of a plurality of preset classes based on a plurality of data around the data of interest, and selecting conversion information corresponding to the classified class. And a conversion unit for converting the data of interest into higher-quality data based on the conversion information. The class classification unit determines whether the data of interest is missing by determining whether the data of interest is missing or not. An information processing apparatus characterized in that the information is classified into different classes. 2. The conversion information is information for generating, for a missing class in which the data of interest is missing, the data of interest based on the plurality of surrounding data, wherein the data of interest is 2. The information processing apparatus according to claim 1, wherein for a non-missing class that is not missing, the information is information for converting the target data into higher-quality data. 3. The information according to claim 2, wherein the conversion information is information for converting the target data into data with reduced noise for the non-missing class. Processing equipment. 4. The information processing apparatus according to claim 1, wherein the conversion information is information obtained by learning in advance. 5. The conversion information may be linear or non-linear,
2. The information processing apparatus according to claim 1, wherein the prediction coefficient is a prediction coefficient used for a first-order or multi-order estimation expression. 6. The method according to claim 1, wherein the class classification unit classifies the data of interest into one of a plurality of preset classes based on a class tap that is a plurality of data around the data of interest. The information processing apparatus according to claim 1. 7. The information processing apparatus according to claim 1, wherein the conversion unit converts the target data into higher-quality data based on prediction taps corresponding to the classified classes. . 8. The classifying means classifies the data of interest into one of a plurality of preset classes based on a plurality of data spatially surrounding the data of interest. The information processing apparatus according to claim 1. 9. The classifying means classifies the data of interest into one of a plurality of preset classes based on a plurality of data temporally surrounding the data of interest. The information processing apparatus according to claim 1. 10. The information processing apparatus according to claim 1, wherein the data of interest is image data. 11. A class classification step of classifying the data of interest into one of a plurality of preset classes based on a plurality of data around the data of interest, and selecting conversion information corresponding to the classified class. And a conversion step of converting the data of interest into higher quality data based on the conversion information. In the class classification step, the data of interest is determined by whether or not the data of interest is missing. Are classified into different classes. 12. The conversion information is information for generating, for a missing class in which the data of interest is missing, the data of interest based on the plurality of surrounding data. 12. The information processing method according to claim 11, wherein for a non-missing class which is not missing, the information is information for converting the attention data into higher quality data. 13. The information according to claim 12, wherein the conversion information is information for converting the target data into data with reduced noise for the non-missing class. Processing method. 14. The conversion information according to claim 11, wherein the conversion information is information obtained by learning in advance.
An information processing method according to claim 1. 15. The information processing method according to claim 11, wherein the conversion information is a prediction coefficient used for a linear or non-linear, or first- or multi-order estimation formula. 16. The class classification step, wherein the data of interest is classified into one of a plurality of preset classes based on a class tap which is a plurality of data around the data of interest. Claim 1
2. The information processing method according to 1. 17. The information processing according to claim 11, wherein, in the conversion step, the data of interest is converted into higher-quality data based on prediction taps corresponding to the classified classes. Method. 18. The class classification step, wherein the data of interest is classified into one of a plurality of preset classes based on a plurality of data spatially surrounding the data of interest. The information processing method according to claim 11, wherein 19. The class classification step, wherein the data of interest is classified into one of a plurality of preset classes based on a plurality of data temporally surrounding the data of interest. The information processing method according to claim 11, wherein 20. The information processing method according to claim 11, wherein said attention data is image data. 21. A class classification step of classifying the data of interest into one of a plurality of preset classes based on a plurality of data around the data of interest, and selecting conversion information corresponding to the classified class. And a conversion step of converting the data of interest into higher quality data based on the conversion information. In the class classification step, the data of interest is determined by whether or not the data of interest is missing. Are classified into different classes, and a computer readable program is recorded on the recording medium. 22. Class classifying means for classifying the data of interest into one of a plurality of preset classes based on a plurality of data around the data of interest, and classifying the data of interest corresponding to the class. Conversion information generating means for generating conversion information for converting to higher quality data, wherein the class classification means classifies the data of interest into different classes depending on whether the data of interest is missing or not. A learning device characterized by the following. 23. A class classification step of classifying the data of interest into one of a plurality of classes set in advance based on a plurality of data around the data of interest; A conversion information generating step of generating conversion information for converting the data into higher quality data. In the class classification step, the data of interest is classified into different classes depending on whether the data of interest is missing or not. A learning method characterized in that: