JP2003304533A - Image signal transmission apparatus and method, and image signal receiving apparatus and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image signal transmission apparatus for sub-sampling data to suppress an amount of the data to be transmitted thereby bringing efficiency to the data transmission. <P>SOLUTION: Information s of a target pixel and object pixels and a vector v are fed to a vector conversion section 4 and the vector v is fed from a memory 3 to a local decoder 31 on the basis of scan information s of a result from searching the object pixels from an object searching section 9. The vector conversion section 4 calculates a difference vector between the target pixel and the object vectors. The local decoder 31 receives the vector v from the memory 3, the vector v and the information s from an object evaluation section 5, and the information s from a scan memory 8, performs local decoding together with a sub sample while referring to a difference vector table 7, and supplies a difference e to the object evaluation section 5. The object evaluation section 5 evaluates the object pixel on the basis of the received difference e and determines pixels to be transmitted among the object pixels on the basis of the result of evaluation. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image signal transmitting apparatus and method for transmitting image data used in, for example, a digital television, a digital video recording / reproducing apparatus, etc., and an image signal receiving apparatus and method.

[0002]

2. Description of the Related Art Conventionally, NTSC (National Television)
System Committee) method and PAL (Phase Alternat)
Ion by Line) method, a raster scan is used that scans one line at a time from the top left of the screen. In this raster scan, since the scanning order is determined, it is not necessary to transmit information regarding the scan, for example, address information. All bits can be sent as pixel information.

At present, a method for transmitting an image as efficiently as possible is being considered. As one of the methods, a transmission method is known in which the amount of information is reduced by utilizing the correlation of images. That is, the closer the pixel is to an image, the higher the correlation, and the efficiency can be improved by removing the redundancy. Specifically, it is possible to considerably improve efficiency by sending difference information of neighboring pixels.

At this time, in order to positively utilize the correlation between neighboring pixels of an image, so-called random scanning, which gives a degree of freedom in scanning, is used. However, unlike the raster scan, the random scan needs to transmit the address information for each pixel together with the pixel information. As described above, the random scan transmits with the addition of the address information, but since the neighboring pixel correlation is used, it is more efficient than the raster scan without transmitting the address information as a whole.

[0005]

However, if the address is simply transmitted as it is, the amount of information is too large. Therefore, in the case of random scan, this problem is solved by transmitting the relative position from the pixel of interest as the address information together with the pixel information.

However, it is desired to further reduce the amount of data to be transmitted and further improve the efficiency of transmission.

Therefore, an object of the present invention is to provide an image signal transmitting apparatus and method, and an image signal receiving apparatus and method capable of suppressing the amount of data to be transmitted by performing sub-sampling and further improving the efficiency of transmission. To provide.

[0008]

The present invention is an image signal transmission apparatus for transmitting a pixel at an arbitrary position by adding address information, wherein a pixel of interest is selected in an input image signal and the pixel of interest is selected. A plurality of pixels that are located over one or more pixels as candidate pixels, and a difference between the pixel value of the selected target pixel and the pixel value of the selected plurality of candidate pixels, respectively. An image signal transmission device, comprising: a transmission pixel determining unit that determines a pixel to be generated and to transmit based on the difference.

The present invention is an image signal transmission apparatus for transmitting a pixel at an arbitrary position by adding address information, and in an input image signal in which non-transmission pixels are thinned out by subsampling the image signal. A candidate selecting means for selecting a pixel and selecting a plurality of pixels located in a predetermined direction as a candidate pixel, a pixel value of the selected target pixel, and a difference between the pixel values of the selected plurality of candidate pixels, respectively. An image signal transmission device, comprising: a transmission pixel determining unit that determines a pixel to be generated and to transmit based on the difference.

The present invention is an image signal transmission method for transmitting a pixel at an arbitrary position by adding address information, wherein a target pixel is selected in an input image signal and one or more pixels are selected for the target pixel. A plurality of pixels that are located in an interlaced position are selected as candidate pixels, the pixel values of the selected target pixel and the pixel values of the selected plurality of candidate pixels are respectively generated, and pixels are transmitted based on the differences. The image signal transmission method is characterized in that

The present invention is an image signal transmission method for transmitting a pixel at an arbitrary position by adding address information, wherein an input image signal in which non-transmission pixels are thinned out by subsampling the image signal is noticed. Pixels are selected, a plurality of pixels located in a predetermined direction are selected as candidate pixels, and a difference between the pixel value of the selected target pixel and the pixel values of the selected plurality of candidate pixels is generated, and the difference is calculated. The image signal transmission method is characterized in that the pixels to be transmitted are determined based on the above.

According to the present invention, in an image signal receiving apparatus for receiving transmission data in which address information is added to a pixel at an arbitrary position, a receiving means for receiving the transmission data and a pixel value is generated from the received transmission data. An image comprising pixel value generation means, output position determination means for determining an output position from received transmission data, and pixel prediction means for predicting pixel values not transmitted from the received transmission data It is a signal receiving device.

According to the present invention, in an image signal receiving method for receiving transmission data in which address information is added to a pixel at an arbitrary position, the transmission data is received, a pixel value is generated from the received transmission data, and the pixel value is received. The image signal receiving method is characterized in that the output position is determined from the transmitted data and the pixel value not transmitted is predicted from the received transmitted data.

A pixel at a position where one or more pixels adjacent to the target pixel are skipped over is set as a candidate pixel, and a pixel having the smallest difference between the candidate pixel and the target pixel is selected as a transmission pixel, and one or more further jumps are performed. The transmission efficiency can be improved by not transmitting the pixels.

[0015]

BEST MODE FOR CARRYING OUT THE INVENTION For easy understanding of the present invention,
The previously proposed image signal transmission device will be described below with reference to the drawings. It should be noted that components having the same function throughout the drawings are designated by the same reference numerals to avoid duplication of description. FIG. 1 shows a block diagram of the encoder side, and FIG.
Shows a flowchart of the encoding process. Figure 3
FIG. 4 shows a block diagram on the decoder side, and FIG. 4 shows a flowchart of decoding processing.

In FIG. 1, reference numerals 1a, 1b and 1c are used.
Is an input terminal to which, for example, a (4: 4: 4) color component signal (YUV) is input. The input component signal is supplied to the vectorization unit 2. The vectorization unit 2 vectorizes the input component signal as one point in the YUV space. The output of the vectorization unit 2 is written in the frame memory 3 having a size of one screen (this memory 3 is called a vector memory).

Based on the scan information s as a result of the candidate pixel search from the candidate search unit indicated by reference numeral 9, the vector memory 3 sends the scan information s of the target pixel and each candidate pixel and the vector v to the vector conversion unit 4. To be The vector conversion unit 4 calculates the difference vector from the target pixel to each candidate vector, and sends the difference vector to the candidate evaluation unit 5.

The local decoder 6 performs local decoding while referring to the difference vector table 7, supplies the decoded value to the candidate evaluation section 5, and the candidate evaluation section 5 determines a transmission pixel from the candidate pixels. The multiplexing unit 10 multiplexes the scan information s viewed from the pixel of interest of the transmission pixel and the index information i of the representative vector, and the transmission data is taken out to the output terminal 11. Further, the scan information s is supplied from the candidate evaluation unit 5 to the scan memory 8. The output of the scan memory 8 is supplied to the candidate search unit 9, and the output of the candidate search unit 9 is given to the vector memory 3.

Referring to the flow chart shown in FIG.
The flow of the encoding process will be described. In step S1, the pixel to be transmitted first is determined. For example, select the pixel in the center of the image. However, the pixel to be transmitted first may be at any position. In step S2, the pixel is quantized and transmitted. The transmitted pixel is set as the pixel of interest (step S
3).

In step S4, candidate pixels are acquired from the scan memory 8 according to the scanning method.
In step S5, it is determined whether the number of candidate pixels reaches a prescribed number. 4 if the scanning method is zigzag
The number of candidate pixels is a specified number. If it is determined in step S5 that the number of candidate pixels does not reach the specified number, candidate pixels are acquired in raster scan order until the number reaches the specified number (step S6).

When the number of candidate pixels reaches the prescribed number, in step S7, the pixel to be transmitted is locally decoded according to the selected table to determine the transmission pixel. And step S
In 8, the scan direction and the index of the difference vector are transmitted. Steps S4, S6 and S7 are processes that require access to the memory. Step S9
Now, pay attention to the transmitted pixels. And step S1
At 0, it is determined whether all pixels have been transmitted. If all pixels have been transmitted, the encoding process ends. If all the pixels have not been transmitted, the process returns to step S4 to transmit the next pixel.

The decoder will be described with reference to FIG. Reference numeral 21 is an input terminal. The input data is supplied to the dividing unit 22, and the input data is divided into scan information s and index information i. Output position determination unit 23
Receives the scan information s, determines the output position while referring to the scan memory 24, and outputs the output position data a. The vector reconstructing unit 25 reconstructs a vector from the index information i and outputs a vector v.

The vector v is supplied to the componentization section 27, and the vector v is converted into the component signal YUV. The component signal is supplied to the frame memory 28. Output position data a is also supplied to the frame memory 28. Then, the pixel value at the position of the output position data a is output from the frame memory 28 to the output terminal 29.

Referring to the flow chart shown in FIG. 4,
The flow of decoding processing will be described. In step S21, the signal from the encoder is received as the signal of the first pixel. In step S22, the received pixel data is reconstructed. In step S23, the reconstructed pixel data is output to the specified position. Steps S22 and S23
Is a process that requires access to the memory.

In step S24, the signal from the encoder is received. In step S25, a position to be output in all directions is searched for on the scan memory. In the direction in which the output position is not determined, the output position is searched in the raster scan order (step S26). Step S2
At 7, the output position is determined. In step S28, the pixel data is reconstructed from the index information i. In step S29, the reconstructed pixel data is output to the frame memory. In step S30, it is determined whether all pixels have been output. If all the pixels have been output, the decoding process ends. If not, the process returns to step S24 to receive the next pixel.

In the conventional raster scan, by fixing the scan, only the pixel information is sent without sending the address information. On the other hand, in the above-mentioned proposed transmission method, the relative position from the pixel of interest is transmitted as a scan signal, but at this time, how to jump over the already transmitted pixel becomes a problem.

Interlacing of already transmitted pixels will be described with reference to FIG. In FIG. 5, the target pixel is indicated by a double circle, and the candidate pixel of the transmission pixel in the search is indicated by a light shadow. White circles indicate untransmitted pixels and black circles indicate already transmitted pixels. As a search method for searching for pixels to be transmitted,
Pixels located in four directions of up, down, left, and right with respect to the pixel of interest indicated by a double circle are targets of search.

In FIG. 5A, since all the pixels other than the target pixel are untransmitted pixels, as shown in FIG. 5B, four pixels which are adjacent to the target pixel in the four directions of up, down, left, and right are candidate pixels for transmitting pixels. It is said that However, as shown in FIG. 5C, when the pixel of interest is surrounded by the already-transmitted pixels, the position where the already-transmitted pixels exist is specified only by the relative relationship of up, down, left and right. In such a case, the untransmitted pixels are skipped over the already-transmitted pixels and the untransmitted pixels are set as the candidate pixels of the transmitted pixels. That is, as shown in FIG. 5D, since the two pixels located to the right of the pixel of interest are already-transmitted pixels,
The third pixel to the right of the pixel of interest, skipping the two pixels, is taken as the candidate pixel for the transmission pixel. Further, since one pixel in the downward direction of the target pixel is the already-transmitted pixel, the second pixel in the downward direction from the target pixel by skipping the one pixel is set as the candidate pixel of the transmission pixel.

For this processing, the encoder and the decoder have memories for one frame (scan memories 8 and 24).
By setting a flag for each of the transmitted pixels and setting a flag for the transmitted pixel, the transmitted pixel and the untransmitted pixel are distinguished. It should be noted that the information indicating whether the pixel has been sent (sent) or not (is not sent) can be shared by the encoder and the decoder, and it is not necessary to send this information.

Next, the scan pattern will be described. In the conventional raster scan, as described above, the scan rule is set such that the lines are sent one by one from the upper left of the screen. Therefore, there is no need to transmit information regarding scanning, that is, address information for each pixel.

On the other hand, in the case of random scan, address information must be sent for every pixel. However, in the case of an image having a standard resolution per pixel, a data amount of 10 bits is required only for the address information in both the x direction and the y direction. Since the address information is information that must be sent accurately, the amount of data cannot be reduced by quantization or the like. As a result, the amount of address information data is
The data is quite heavy.

Therefore, in the proposed transmission method, the transmission is made more efficient by giving the scanning a degree of freedom.
“Give freedom” does not mean to be completely fixed,
It means not to be totally free. The relative position with respect to the pixel of interest is transmitted based on a predetermined scan pattern.

There can be various scan patterns. FIG. 6 shows examples of some scan patterns.
FIG. 6A shows a conventional raster scan shown as a reference. In the case of raster scan, since the scan direction is fixed, it is not necessary to transmit address information.

FIG. 6B shows four pixels on the left, right, top and bottom with respect to the target pixel.
Any of the pixels located in the direction is transmitted. This FIG. 6B
Hereinafter, the scan shown in is referred to as a "chess scan". In the case of this chess scan, 2-bit address information is required for one pixel. The broken line in the upper left indicates that the raster scan is switched to when no candidate pixel can be acquired.

The scan pattern shown in FIG.
“It is referred to as“ zigzag scan ”) is a search for candidate pixels by a zigzag scan in four regions divided around a transmission pixel. Therefore, pixels can be transmitted to the end without the scan being stuck. 2-bit address information is required for one pixel.

In FIG. 6D, any of the pixels located in a total of 8 directions, that is, vertically, horizontally and diagonally with respect to the pixel of interest, is transmitted. The scan shown in FIG. 6D is hereinafter referred to as "8-direction chess scan". In the case of this chess scan in 8 directions, 3-bit address information is required for one pixel. Since the scan pattern of the chess scan in 8 directions selects the transmission pixel from many candidate pixels,
Although the neighboring pixel correlation can be utilized more, there is a problem that the address information increases by 1 bit as compared with other scanning methods, and the data that can be assigned to pixels decreases by 1 bit.

Color space difference vector coding in the above-described transmission method will be described. As shown in FIG. 7, the target pixel data and the transmission pixel data, which are component signals, are treated as vectors in the color space, and the difference vector is transmitted. This method can be considerably more efficient than generating the difference separately for each component of the component signal. Figure 8A
Further, as shown in FIG. 8B, the set of YUV component color space vectors has a stronger correlation than the set of RGB component color space vectors, and thus the set is small. Further, the color space difference vector is formed in the vicinity of (0,0) by forming the color space difference vector.

As shown in FIG. 9, a vector representing a set of color space difference vectors is obtained in advance, and a difference vector table is created. In the table, each difference vector is distinguished by an index. The difference vector table is shared by the encoder side and the decoder side, and the index is transmitted. If zigzag is used as a scanning method in a frame of 8 bits for one pixel, a 6-bit (64) representative vector other than the address can be held. The vectorization unit 2 converts the component signal into a vector v in the color space.

Furthermore, FIG. 1 shows the processing relating to the candidate selection method and the local decoding in the above-mentioned transmission method.
This will be described with reference to 0. This processing is performed by the vector conversion unit 4, the local decoder 6 and the candidate evaluation unit 5 in FIG.
Done by Further, in FIG. 2, this is done in step S7. In FIG. 10, a double circle pixel is a target pixel, a circle is a candidate pixel, and the numbers in the circle are serial numbers given to the candidate pixels. Each candidate pixel is not necessarily adjacent to the pixel of interest and is obtained from a certain direction in the scan pattern. As shown in FIG. 10A, it is assumed that there are, for example, four candidate pixels for one target pixel.

First, the target pixel is applied to the difference vector table which is shared by the encoding side and the decoding side with respect to the locally decoded value. As an example, if there are 64 (2 ⁶ ) difference vectors of 6 bits, as shown in FIG. 10B, if the difference vectors are added to the local decode value of the pixel of interest, 64 color vectors are obtained. Created. These are currently available transmission pixel values.

Next, as shown in FIG. 10C, the transmissible pixel values that can be created from each candidate and the target pixel are compared, and the transmissible pixel value with the strongest correlation is determined for each candidate based on the comparison result. To do. The most correlatable pixel value that can be transmitted is the optimum approximation value of each candidate. Local decoder 6
Is supplied with the vector of the pixel of interest, and the local decoder 6 locally decodes the pixel of interest and a process of generating a transmission pixel value that can be created as shown in FIG.
A process of calculating a difference between each transmission pixel value and each candidate pixel value and a process of generating an optimum approximate value of each candidate are performed.

Finally, as shown in FIG. 10D, the difference value between each candidate and the optimum transmissible pixel is obtained. The candidate having the smallest calculated difference value is determined as the candidate having the smallest transmission error among the four candidates, and the transmission pixel is determined. The direction of the determined transmission pixel viewed from the pixel of interest and the index of the difference vector are multiplexed and transmitted. As shown in FIG. 10D, the candidate evaluation unit 5 obtains the error of each candidate pixel after transmission and performs the process of determining the transmission pixel.

The present invention improves the efficiency of the previously proposed transmission method. That is, the transmission efficiency is further improved.

In the previously proposed chess scan,
All pixels of the image are transmitted. On the other hand, in the chess scan to which the present invention is applied, the amount of data is too large when all pixels are transmitted, so thinning out pixels,
That is, the amount of image data is deleted by sub-sampling. The subsample will be described with reference to FIG. In FIG. 11, the pixels to be transmitted or the pixels to be transmitted are indicated by solid lines, and the thinned pixels or the pixels predicted by linear interpolation from surrounding pixels are indicated by broken lines.

As shown in FIG. 11A, the normal pixels are filled without any gap. Pixels filled without this gap are thinned out pixel by pixel. At this time, as shown in FIG. 11B, it is common that pixels are thinned out in a staggered pattern, that is, in a so-called five-eye grid pattern in adjacent lines as if thinning out every pixel even when viewed in the vertical direction. Target. This Figure 1
When receiving and reconstructing the subsampled image shown in 1B, the maximum surrounding 4 pixels can be used to predict the decimated pixels, as shown in FIG. 11C.

There are various methods for restoring the thinned pixels on the receiving side, but here, the simplest linear interpolation is used. Here, the linear interpolation will be briefly described with reference to FIG. As shown in FIG. 12A, there are consecutive pixels a0, a1, a2, .... In addition, a number indicating the value of the pixel is shown in the vicinity of the solid line and the broken line circle indicating the pixel. At this time, the pixel a1 thinned out by the sub-sample can be predicted by linear interpolation as shown in Expression (1). a1 = (a0 + a2) / 2 (1)

Generally, in a natural image, neighboring pixels have a strong correlation. Therefore, the pixel values are often continuous as shown in FIG. 12B. Therefore, by averaging the neighboring pixels as in Expression (1), the value of the pixel located at the center can be predicted. However, this correlation between neighboring pixels is a statistical property, and when viewed locally, there is a portion where this property does not hold at all. Therefore, the failure may be noticeable in some places, or the image may be blurry. Note that FIG. 12B shows changes in the color difference signal v of the consecutive pixels a0, a1, and a2.

When sub-sampling is performed by raster scanning, pixels are thinned out as shown in FIG. 11B. However, as described above, the neighborhood pixel correlation in the image is not always locally established. Further, a place where the neighboring pixel correlation is not established is often near the edge, and is visually conspicuous even if the number is small.

As described above, when sub-sampling is performed by the raster scan, the scan is fixed,
Pixels are similarly thinned out regardless of the local characteristics of the image, so that the image is greatly deteriorated.

The previously proposed chess scan and the chess scan to which the present invention is applied will be described with reference to FIGS. 13 and 14. This FIG. 13 and FIG.
Reference numeral 4 indicates a target pixel by a double circle, a candidate pixel which is a candidate for a transmission pixel in the search is indicated by a solid line, and a pixel thinned out by sub-samples is indicated by a broken line. It should be noted that a light shadow is given to the selected pixel. Further, in each drawing, an example of the pixel value is shown near the pixel.

First, in the previously proposed chess scan shown in FIG. 13A, the direction in which the difference between the target pixel and the candidate pixels above, below, left and right of the target pixel is minimized is searched for. As a result, in this example, as shown in FIG. 13B,
The right candidate pixel with the smallest difference is selected.

Next, in the chess scan to which the present invention is applied, which is shown in FIG. 14A, the direction in which the difference between the pixel of interest and the candidate pixel skipping one pixel on the left, right, top and bottom adjacent to the pixel of interest is the smallest. Is searched. By skipping one pixel, sub-sampling is performed along with the search in the scan direction. In this embodiment, the number of pixels to be skipped is fixed to one. If the number of pixels to be skipped is not fixed, pixels other than the pixel of interest transmit only relative positions, and the address is lost on the receiving side. The skipped pixel is treated in the same manner as the already transmitted pixel.

An example of the scan order in the embodiment of the present invention will be described. A candidate pixel having the smallest difference between the target pixel and the candidate pixel, the average of the target pixel and the candidate pixel, and the difference between the thinned pixel located between the target pixel and the candidate pixel is transmitted. In the case of FIG. 14, the difference between the pixel of interest and the candidate pixel that skips one pixel above the pixel of interest is “| 50-10 | + | 10− (10
+50) / 2 | = 60 ”. The difference between the pixel of interest and the candidate pixel skipping one pixel on the right adjacent to the pixel of interest is "| 50-30 | + | 45- (30 + 50) / 2.
| = 25 ”. The difference between the pixel of interest and the candidate pixel that has skipped one pixel below the pixel of interest is “| 5
0-110 | + | 75- (110 + 50) / 2 | = 6
5 ”. The difference between the pixel of interest and the candidate pixel that skips one pixel on the left adjacent to the pixel of interest is “| 50-5
0 | + | 30- (50 + 50) / 2 | = 20 ".
Therefore, in one embodiment of the present invention, as shown in FIG. 14B, the left candidate pixel having the smallest difference is selected.

In this embodiment, the total difference between the difference between the actually transmitted pixels and the true value of the subsampled and restored pixels is minimized. Candidate pixels are selected as transmission pixels. That is, in this embodiment, since the pixel of interest, the candidate pixel, and the candidate pixel having the smallest difference between the thinned pixels are selected, the neighboring pixel correlation is further utilized.

The number of interlaced pixels between the target pixel and the candidate pixel, that is, the number of thinned pixels can be two or more. When the number of pixels to be thinned out becomes large, prediction is performed by weighted linear interpolation. As a matter of course, the more pixels are thinned out, the greater the deterioration becomes. Actually, such an operation is performed while locally decoding. This is because the best image quality is obtained by evaluating the pixels actually reproduced after the transmission.

Here, as a result of predicting the scan order from the target pixel to the seventh transmission pixel, "up → left → right → right →
“Down → left → down”, and the operation will be described with reference to FIG. As an example, it is a chess scan in 8 directions shown in FIG. 6D, and sub-sampling is performed according to the rule that one pixel in the transmission direction is skipped on the line of the scan pattern. The numbers shown in the vicinity of the pixels in FIG. 15 indicate the order of pixels to be transmitted. Further, in FIG. 15, the target pixel is indicated by a double circle, the thinned pixel is indicated by a black circle, and the transmission pixel is shaded.

FIG. 16 shows an example of the scan order in another embodiment of the present invention. In another embodiment, the embodiment of the present invention is applied to a signal in which pixels are thinned out in advance in an input image signal in a five-eye grid pattern (see FIG. 11). In this other embodiment, according to the above-described scan order “up → left → right → right → down → left → down”, the process shown in FIG.
Pixels are transmitted in the order shown in FIG.

Also in this other embodiment, the pixels to be subsampled are treated the same as the already transmitted pixels, and the remaining pixels are transmitted.

Further, in the case of this other embodiment, since the pixels are thinned out in advance, the true value of the thinned pixels cannot be known. Therefore, in the other embodiment, the candidate pixel having the smallest difference between the target pixel and the candidate pixel is transmitted.
In the case of FIG. 14 described above, the difference between the target pixel and the upper candidate pixel adjacent to the target pixel is “| 50-10 | = 4.
It becomes "0". The difference between the pixel of interest and the candidate pixel on the right adjacent to the pixel of interest is “| 50−30 | = 20”.
The difference between the target pixel and the lower candidate pixel adjacent to the target pixel is “| 50-110 | = 60”. The difference between the pixel of interest and the candidate pixel on the left adjacent to the pixel of interest is “| 50-50 | = 0”. Therefore, in this other embodiment, the left candidate pixel with the smallest difference is selected.

As described above, even if the scanning order is the same, the order of the pixels to be transmitted changes if the time point at which the sub-sampling is performed is different. Then, comparing one embodiment with other embodiments, one embodiment can adaptively perform sub-sampling according to the characteristics of the image.

This one embodiment and the other embodiments can also be applied to the zigzag scan shown in FIG. 6C. Even if there are already-transmitted pixels or pixels to be sub-sampled on the line searched by the zigzag scan, it is assumed that the transmitted pixels are searched for by the scan pattern shown in FIG. 6C. In addition, when the operation of zigzag scanning, that is, the transmission order is set to "up → left → right → right → bottom → left → down", it is difficult to see the direction by zigzag scanning, but "up" is seen from the pixel of interest. To search the upper left area, "Right" searches the upper right area from the pixel of interest, "Down" searches the lower right area from the pixel of interest, and "Left" to ,
A region in the lower left direction when viewed from the pixel of interest is searched.

FIG. 1 shows an embodiment of the configuration of an encoder to which an example of the scan order according to the embodiment of the present invention is applied.
This will be described with reference to FIG. The vector memory 3 based on the scan information s as a result of the candidate pixel search from the candidate search unit 9.
From the target pixel and the scan information s of each candidate pixel are sent to the vector conversion unit 4, and the vector v is further sent to the vector conversion unit 4 and the local decoder 31. The vector conversion unit 4 calculates the difference vector from the target pixel to each candidate vector, and sends the difference vector to the candidate evaluation unit 5.

In the local decoder 31, the vector v is supplied from the vector memory 3, the vector v and the scan information s are supplied from the candidate evaluation unit 5, and the scan information s is supplied from the scan memory 8, and the subsample is included while referring to the difference vector table 7. Local decoding is performed by the local decoding, and the difference value e of the local decoding is supplied to the candidate evaluation unit 5. This difference value e is the difference when transmitting the selected candidate pixel and the difference between the true value and the restored value of the pixel that is subsampled and restored later. Is a locally decoded vector obtained by referring to.

In the candidate evaluation section 5, the local decoder 31
The candidate pixel is evaluated from the difference value e supplied from the above, and the transmission pixel is determined from the candidate pixels based on the evaluation result. In this candidate evaluation unit 5, the scan information s of the transmission pixel viewed from the determined target pixel is supplied to the multiplexing unit 10 and the scan memory 8, and the index information i of the representative vector thereof is supplied to the multiplexing unit 10. .

The scan memory 8 is a frame memory for storing the transmitted information of the scan as a flag. The scan information s is supplied from the scan memory 8 to the local decoder 31.

In the multiplexer 10, the scan information s of the transmission pixel viewed from the pixel of interest and the index information i of the representative vector are multiplexed, and the transmission data is taken out to the output terminal 11. Further, the scan information s is supplied from the candidate evaluation unit 5 to the scan memory 8. The output of the scan memory 8 is supplied to the candidate search unit 9, and the output of the candidate search unit 9 is given to the vector memory 3.

Here, a detailed block diagram of an example of the local decoder 31 is shown in FIG. The vector v from the candidate evaluation section 5 is supplied to the difference calculation circuit 46 and the linear interpolation circuit 50 via the terminal 41. The scan information s from the candidate evaluation section 5 is supplied to the difference calculation circuit 46 and the thinned pixel search circuit 48 via the terminal 42. The scan information s from the scan memory 8 is supplied to the thinned pixel search circuit 48 via the terminal 43. The difference calculation circuit 49 for calculating the difference between the vector v from the vector memory 3 and the true value via the terminal 44.
Is supplied to.

The difference calculation circuit 46 calculates the difference between the target pixel and the candidate pixels above, below, left and right of the target pixel based on the vector v from the candidate evaluation section 5 and the scan information s. Specifically, in the difference calculation circuit 46, the candidate evaluation unit 5
The vector v from is supplied to the local decoder 47. Then, the difference calculation circuit 46 selects the optimum approximate value of the candidate pixel from each difference value e supplied from the local decoder 47. That is, the candidate having the smallest difference value between the candidate pixel and each difference value e is selected, and the difference value e of the selected candidate is supplied to the difference calculation circuit 51.

In the local decoder 47, the vector v is locally decoded by referring to the difference vector table 7 via the terminal 45. That is, as shown in FIG. 10C, the difference calculation circuit 4 calculates the result of comparing the values of transmissible pixels that can be created from each candidate and the target pixel.
6 is supplied.

The thinned pixel search circuit 48 searches for pixels to be thinned out when a candidate pixel is selected based on the scan information s from the candidate evaluation section 5 and the scan information s from the scan memory 8. The thinned pixels to be searched are supplied to the difference calculation circuit 49 with the true value as the scan information s.

In the difference calculation circuit 49 from the true value, the vector v from the vector memory 3 and the thinned pixel search circuit 4
The difference between the thinned pixel value predicted from the scan information s from 8 and the true value is calculated. Specifically, in the difference calculation circuit 49 from the true value, the vector v from the vector memory 3
And scan information s from the thinned pixel search circuit 48
Is supplied to a linear interpolation circuit 50 which is connected to the difference calculation circuit 49 from the true value. Then, the difference calculation circuit 49 with the true value calculates the difference between the vector v supplied from the linear interpolation circuit 50 and the true value vector v of the thinned pixels. The calculated difference value e is supplied to the difference calculation circuit 51.

The linear interpolation circuit 50 linearly interpolates the thinned pixels from the supplied vector v and scan information s. At this time, the thinned pixels are predicted by the linear interpolation shown in FIG. 12 and Expression (1) described above. The predicted value of the thinned pixel interpolated by the linear interpolation is supplied to the difference calculation circuit 49 with the true value as the vector v.

In the difference calculation circuit 51, the difference calculation circuit 46
And the difference value e from the difference calculation circuit 49 between the true value and the true value are added. The addition result is supplied as the difference value e to the candidate evaluation unit 5 via the terminal 52.

The procedure of the encoding process will be described with reference to the flowchart shown in FIG. Step S41
At, the first transmitted pixel is determined. For example, the pixel in the middle of the image is selected. However, the pixel to be transmitted first may be at any position. In step S42, the pixel is vectorized and transmitted. Step S4
In 3, the transmitted pixel is set as the pixel of interest. Step S4
In 4, the candidate pixels are acquired from the scan memory 8 according to the scanning method.

In step S45, it is determined whether or not the number of candidate pixels has reached the specified number determined for each scan pattern. In the chess scan to which the present invention is applied, the prescribed number of candidate pixels is "4". Then, if it is determined in this step S45 that the number of candidate pixels does not reach the prescribed number, the control moves to step S46, and if it is determined that the number has reached the prescribed number, step S4
Control is transferred to 7. In step S46, untransmitted pixels are acquired as the number of candidate pixels in the raster scan order until the specified number is reached.

In step S47, the pixel of interest and each candidate pixel are locally decoded. Step S48
Then, pixels to be thinned out from the target pixel and the locally decoded candidate pixel are predicted by linear interpolation. The candidate pixels selected by the raster scan can be treated in the same manner because they can be thinned out in the raster scan order. In step S49, the candidate pixel having the smallest sum of the difference between the true value of the thinned pixel and the value predicted by the linear interpolation and the difference between the target pixel and the candidate pixel is selected as the scanning direction.

In step S50, the target pixel and the selected candidate pixel are transmitted. In step S51, the transmitted candidate pixel is set as the target pixel. In step S52,
It is determined whether all pixels have been transmitted. If it is determined that all pixels have been transmitted, this flowchart ends, and if it is determined that there are still untransmitted pixels, step S
Control returns to 44.

A decoder according to an embodiment of the present invention will be described with reference to FIG. Input data, which is encoded data, is supplied from the input terminal 21 to the dividing unit 22, and the input data is divided into scan information s and index information i. The output position determination unit 23 receives the scan information s, determines the output position with reference to the scan memory 24, and outputs the output position data a. The vector reconstructing unit 25 reconstructs the vector v from the index information i and outputs the vector v. In this case, the vector reconstructing unit 25 reconstructs the vector v using the selected difference vector table 26.

The vector v from the vector reconstruction unit 25
Is supplied to the componentization unit 27, the delay circuit 61, and the thinned pixel prediction circuit 62. In the delay circuit 61, the supplied vector v is delayed by one pixel and is supplied to the thinned pixel prediction circuit 62.

The thinned pixel prediction circuit 62 predicts thinned pixels from the vector v of transmission pixels from the vector reconstruction circuit 25 and the vector v of transmission pixels delayed by one pixel from the delay circuit 61. At this time, by the linear interpolation circuit 63 connected to the thinned pixel prediction circuit 62,
Linear interpolation of thinned pixels is performed. The predicted thinned-out pixels are converted into a vector v by the componentization unit 2
7 is supplied.

In the componentization section 27, the vector v
Are converted into component signals YUV. The converted component signal YUV is supplied to the frame memory 28. The output position data a is stored in the frame memory 28.
Is also being supplied. Then, the pixel value at the position of the output position data a is output from the frame memory 28 to the output terminal 29.

The flow of the decoding process will be described with reference to the flowchart shown in FIG. In step S61, the signal from the encoder is received as the signal of the first pixel. In step S62, the received pixel data is reconstructed. In step S63, the reconstructed pixel data is output to the specified position.

In step S64, the signal from the encoder is received. In step S65, the positions to be output in all directions are searched on the scan memory. In step S66, the output position is searched for in the raster scan order in the direction in which the output position is not determined. Step S
At 67, the output position is determined. In step S68, the pixel data is reconstructed using the difference vector table. In step S69, the reconstructed pixel data is output to the frame memory.

In step S70, the pixels thinned out between the pixel of interest and the output pixel are predicted by linear interpolation. In step S71, the thinned pixels are output. In step S72, it is determined whether all pixels have been output. If all the pixels have been output, the decoding process ends. Otherwise, control returns to step S64 to receive the next pixel.

In this embodiment, sub-sampling is performed for each pixel, but more pixels can be thinned out by using weighted linear interpolation or the like. However, the image is deteriorated by thinning out many pixels.

In this embodiment, the present invention is applied to eight-direction chess scan, but the present invention may be applied to four-direction chess scan and may be applied to zigzag scan. .

The present invention is not limited to the above-described embodiment of the present invention and the like, and various modifications and applications are possible without departing from the gist of the present invention.

[0088]

According to the present invention, when a sub-sample is applied to a system that employs a random scan as a scanning method, it is possible to make a sub-sample utilizing the characteristics of the random scan. It can be more efficient than introducing sub-samples in a fixed scan.

According to the present invention, the input image signal can be applied to a signal subsampled in advance, and the transmission efficiency can be further improved.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an encoder in a transmission system previously proposed by the applicant of the present application.

FIG. 2 is a flowchart showing a flow of an encoding process in the transmission system previously proposed by the applicant of the present application.

FIG. 3 is a block diagram showing a configuration of a decoder in the transmission system previously proposed by the applicant of the present application.

FIG. 4 is a flowchart showing a flow of a decoding process in the transmission system previously proposed by the applicant of the present application.

FIG. 5 is a schematic diagram for explaining a method of skipping already-transmitted pixels in the transmission method previously proposed by the applicant of the present application.

FIG. 6 is a schematic diagram for explaining a method of searching for pixels to be transmitted in the transmission method previously proposed by the applicant of the present application.

FIG. 7 is a schematic diagram for explaining a color space difference vector in the transmission method previously proposed by the applicant of the present application.

FIG. 8 is a schematic diagram for explaining a color space difference vector in the transmission method previously proposed by the applicant of the present application.

FIG. 9 is a schematic diagram for explaining a color space difference vector in the transmission system previously proposed by the applicant of the present application.

FIG. 10 is a schematic diagram used to describe a candidate selection method and local decoding according to the embodiment of the present invention.

FIG. 11 is a schematic diagram used to explain a subsample.

FIG. 12 is a schematic diagram used for explaining linear interpolation.

FIG. 13 is a schematic diagram for explaining a transmission method previously proposed by the applicant of the present application.

FIG. 14 is a schematic diagram for explaining an embodiment of a transmission system of the present invention.

FIG. 15 is a schematic diagram for explaining an example of a scan order of an embodiment in the transmission system of the present invention.

FIG. 16 is a schematic diagram for explaining another example of the scan order of the embodiment in the transmission system of the present invention.

FIG. 17 is a block diagram showing the configuration of an embodiment of an encoder in the transmission system of the present invention.

FIG. 18 is a block diagram showing the configuration of an embodiment of a local decoder in the transmission system of the present invention.

FIG. 19 is a flowchart showing the flow of an embodiment of an encoding process in the transmission system of the present invention.

FIG. 20 is a block diagram showing the configuration of an embodiment of a decoder in the transmission system of the present invention.

FIG. 21 is a flowchart showing a flow of an embodiment of a decoding process in the transmission system of the present invention.

[Explanation of symbols]

2 ... Vectorization unit, 3 ... Vector memory, 4 ...
..Vector conversion unit, 5 ... Candidate evaluation unit, 7 ... Difference vector table, 8 ... Scan memory, 9 ...
・ Candidate search unit, 10 ... Multiplexing unit, 31 ... Local decoder

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Ken Hori             6-735 Kita-Shinagawa, Shinagawa-ku, Tokyo Soni             -Inside the corporation (72) Inventor Toru Miyake             6-735 Kita-Shinagawa, Shinagawa-ku, Tokyo Soni             -Inside the corporation F term (reference) 5C059 LB04 LB15 PP16 RB01 RC00                       SS02 TA00 TA71 TB01 TC02                       TC06 TC34 TC42 TD05 UA34                 5C063 AB03 AB05 AC01 AC10 BA08                       CA01 DA01 DA07 DA13                 5J064 AA02 BB01 BB03 BC25 BD02

Claims (8)

[Claims] 1. An image signal transmission device for transmitting a pixel at an arbitrary position by adding address information, wherein a target pixel is selected in an input image signal, and one or more pixels are selected for the target pixel. Candidate selection means for selecting a plurality of interlaced pixels as candidate pixels, and a difference between the pixel value of the selected pixel of interest and the pixel value of the selected plurality of candidate pixels, and the difference And a transmission pixel determining means for determining a pixel to be transmitted based on the above. 2. The transmission pixel determining means obtains a first difference between the pixel value of the selected target pixel and the pixel values of the plurality of selected candidate pixels, respectively, and skips one or more of the differences. A second difference between the pixel value and the pixel value obtained by predicting the one or more skipped pixel values is obtained, and the candidate pixel having the smallest sum of the first and second differences is determined as the pixel to be transmitted. The image signal transmission device according to claim 1, wherein 3. An image signal transmission device for transmitting a pixel at an arbitrary position by adding address information, wherein an image signal is sub-sampled so that a pixel of interest is detected in an input image signal in which non-transmission pixels are thinned out. Selection, a candidate selection means for selecting a plurality of pixels located in a predetermined direction as candidate pixels, a pixel value of the selected target pixel, and a difference between the pixel values of the plurality of selected candidate pixels, respectively. An image signal transmission device, comprising: a transmission pixel determining unit that determines a pixel to be generated and to transmit based on the difference. 4. An image signal transmission method for transmitting a pixel at an arbitrary position by adding address information, wherein a target pixel is selected in an input image signal, and one or more pixels are selected for the target pixel. A plurality of interlaced pixels are selected as candidate pixels, a difference between the pixel value of the selected target pixel and the pixel values of the selected plurality of candidate pixels is generated, and transmission is performed based on the difference. An image signal transmission method, characterized in that a pixel to be processed is determined. 5. The pixel value of the selected pixel of interest and the selected 1 when determining the pixel to be transmitted.
To a first difference between the pixel values of a plurality of candidate pixels, and a second difference between the pixel value at which the one or more skipped pixel values are predicted and the pixel value at which the one or more skipped pixel values are predicted, respectively. 5. The image signal transmission method according to claim 4, wherein the candidate pixel that is acquired and has the smallest sum of the first and second differences is determined as a pixel to be transmitted. 6. An image signal transmission method for transmitting a pixel at an arbitrary position by adding address information, wherein an input image signal in which non-transmission pixels are thinned out by subsampling the image signal Selection, a plurality of pixels located in a predetermined direction are selected as candidate pixels, and a difference between the pixel value of the selected target pixel and the pixel values of the selected plurality of candidate pixels is generated, and An image signal transmission method characterized in that a pixel to be transmitted is determined based on a difference. 7. An image signal receiving apparatus for receiving transmission data in which address information is added to a pixel at an arbitrary position, receiving means for receiving the transmission data, and a pixel for generating a pixel value from the received transmission data. It has a value generation means, an output position determination means for determining an output position from the received transmission data, and a pixel prediction means for predicting a pixel value not transmitted from the received transmission data. Image signal receiving device. 8. An image signal receiving method for receiving transmission data in which address information is added to a pixel at an arbitrary position, wherein the transmission data is received, a pixel value is generated from the received transmission data, and the pixel value is received. An image signal receiving method, characterized in that the output position is determined from the transmitted data and the pixel value not transmitted is predicted from the received transmitted data.