JP4182676B2 - Image signal transmitting apparatus and method, and image signal receiving apparatus and method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a transmission apparatus and method, and a reception apparatus and method related to transmission of image data used in, for example, a digital television and a digital video recording / reproducing apparatus.
[0002]
[Prior art]
When transmitting digitized image data for television broadcasting or the like, encoding processing is performed with reference to pixels located around the transmitted pixel. Such processing makes it possible to perform efficient compression coding by utilizing the property that a general image that does not include an edge or the like often has strong autocorrelation in a neighboring region. As an order of pixels to be transmitted, a raster scan in which images are sequentially transmitted from the upper left to the lower right has been used. In such a transmission method, for example, 8-bit data is transmitted per pixel including the luminance signal Y and the color difference signals U and V. In this case, it is not necessary to transmit data indicating the pixel address.
[0003]
In such a transmission method, when a pixel value adjacent in the horizontal direction changes abruptly, an effect caused by compressing an image obtained by decoding transmitted data with reference to neighboring pixels appears. For example, when an image of one “bar” in the vertical direction is transmitted using the encoding method as described above, the “bar” is shaded due to the influence of neighboring pixels referred to at the time of encoding. It becomes an image.
[0004]
In order to solve such a problem or reduce the degree thereof, the applicant of the present application has previously proposed a transmission method in which the scan order is not fixed. This transmission method determines the pixel having the highest degree of correlation with the pixel of interest from among the untransmitted pixels located in the four directions (up, down, left, and right) with respect to the pixel of interest, or in the eight directions including the diagonal direction in these four directions And calculating a difference value between the pixel determined as described above and the target pixel, and scan information indicating a direction of the pixel determined to have the highest degree of correlation with the target pixel together with the calculated difference value with respect to the target pixel. It is intended to be transmitted. In the transmission method described above, a difference value is transmitted as pixel information, a pixel having the smallest difference value is selected as a pixel to be transmitted next, and the difference value and scan order data are transmitted. On the receiving side, the output position of the pixel is determined from the scan information, and the decoded pixel value is output to the determined position. According to such a transmission method, it is possible to perform image transmission much more efficiently than the conventional NTSC system.
[0005]
[Problems to be solved by the invention]
In the transmission method described above, it is necessary to transmit scan information in addition to pixel information, and the scan information only indicates the scan order. Therefore, scan information that needs to be transmitted in addition to image data has not been effectively used.
[0006]
Accordingly, an object of the present invention is to provide an image signal transmission apparatus and method, and an image signal receiving apparatus capable of transmitting embedded information in addition to one image information as information to be transmitted because the scan information has embedded information. And to provide a method.
[0007]
[Means for Solving the Problems]
In order to solve the above-described problem, the invention of claim 1 is an image signal transmission device that transmits a pixel together with address information indicating a position of the pixel to be transmitted when transmitting a pixel at an arbitrary position, Input component signal as a vector in color space Vectorizing means to vectorize and vectorized Component signal Memory to store the image Pre-set position Selected as a target pixel, candidate selection means for selecting a plurality of adjacent pixels not yet transmitted from the target pixel as candidate pixels, a vector of the selected target pixels, and a plurality of selected candidate pixels A transmission pixel determining means for acquiring a difference from a vector and determining a transmission pixel as a candidate pixel having the smallest acquired difference; and a selected target pixel Centered on Determined transmission pixel Direction Scan information generating means for generating scan information indicating the address, and the scan information generated by transmitting address information together with the quantized value of the selected pixel of interest Part of In Consists of several bits Transmission means for embedding embedding information and transmitting a vector of transmission pixels determined together with the scan information; In the candidate selection means, the pixel that was determined and transmitted as the transmission pixel in the transmission pixel determination means immediately before is newly selected as the target pixel from the memory, and has not yet been transmitted from the newly selected target pixel. A plurality of adjacent pixels are newly selected as candidate pixels, and the transmission pixel determination unit repeats the process of newly determining a transmission pixel from the newly selected pixel of interest and the newly selected plurality of candidate pixels. An image signal transmission device.
The invention of claim 5 is an image signal transmission method for transmitting a pixel together with address information indicating a position of the pixel to be transmitted when transmitting a pixel at an arbitrary position, Input component signal as a vector in color space Vectorizing step to vectorize, vectorized Component signal Storing in a memory, and Pre-set position Are selected as a target pixel, a candidate selection step is performed to select a plurality of adjacent pixels not yet transmitted from the target pixel as candidate pixels, a vector of the selected target pixels, and a plurality of selected candidate pixels. A transmission pixel determination step of acquiring a difference from the vector and determining a transmission pixel as a candidate pixel having the smallest acquired difference, and the selected target pixel Centered on Determined transmission pixel Direction A scan information generation step for generating scan information indicating the address, and the scan information generated by transmitting address information together with the quantized value of the selected pixel of interest Part of In Consists of several bits A transmission step of embedding embedded information and transmitting a vector of transmission pixels determined together with the scan information; In the candidate selection step, a pixel that has been determined as a transmission pixel and transmitted immediately before is newly selected as a target pixel from the memory, and a plurality of adjacent pixels that are not yet transmitted from the newly selected target pixel Is newly selected as a candidate pixel, and in the transmission pixel determination step, a process of newly determining a transmission pixel from the newly selected pixel of interest and a plurality of newly selected candidate pixels is repeated. This is an image signal transmission method.
[0008]
The invention of claim 6 Input component signal as a vector in color space Vectorized and vectorized Component signal Stored in the memory Pre-set position Are selected as the target pixel, a plurality of adjacent pixels not yet transmitted from the target pixel are selected as candidate pixels, and the difference between the selected target pixel vector and the selected plurality of candidate pixel vectors is selected. The candidate pixel with the smallest obtained difference is determined as the transmission pixel, and the selected pixel of interest is selected. Centered on Determined transmission pixel Direction The scan information generated is generated, the address information is transmitted together with the quantized value of the selected pixel of interest, and the generated scan information Part of In Consists of several bits An image signal receiving apparatus for embedding embedded information and receiving transmission data including a transmission pixel vector determined together with the scan information, receiving means for receiving the transmission data, and transmission pixels included in the received transmission data A pixel value decoding unit that decodes a pixel value from the vector, an output position determination unit that determines an output position of the decoded pixel value from scan information included in the received transmission data, and converts the scan information into embedded information An image signal receiving apparatus having embedded information restoring means.
The invention of claim 7 Input component signal as a vector in color space Vectorized and vectorized Component signal Stored in the memory Pre-set position Are selected as the target pixel, a plurality of adjacent pixels not yet transmitted from the target pixel are selected as candidate pixels, and the difference between the selected target pixel vector and the selected plurality of candidate pixel vectors is selected. The candidate pixel with the smallest obtained difference is determined as the transmission pixel, and the selected pixel of interest is selected. Centered on Determined transmission pixel Direction The scan information generated is generated, the address information is transmitted together with the quantized value of the selected pixel of interest, and the generated scan information Part of In Consists of several bits An image signal receiving method for embedding embedded information and receiving transmission data including a transmission pixel vector determined together with the scan information, the receiving step for receiving the transmission data, and the transmission pixels included in the received transmission data A pixel value decoding step for decoding a pixel value from the vector, an output position determining step for determining an output position of the decoded pixel value from scan information included in the received transmission data, and converting the scan information into embedded information An image signal receiving method including an embedded information restoring step.
[0009]
In the present invention, the embedded information is converted into scan information. The transmission pixel information determined according to the scan information and the scan information are transmitted. On the receiving side, the transmitted image can be restored by outputting the received pixel information according to the scan information. Also, the embedded information can be restored from the scan information. Thus, the scan information has not only the scan order but also embedded information, and the scan information can be used effectively.
[0010]
DETAILED DESCRIPTION OF THE INVENTION
Generally, in order to transmit a digital image signal as efficiently as possible, it is preferable to use the pixel neighborhood correlation of an image. If the pixels are similar to each other, the efficiency can be considerably improved by sending difference information. Since the raster scan is used in the NTSC system, the scan method is fixed. On the other hand, in the scan proposed previously, the scan has a degree of freedom in order to positively use the image neighborhood correlation. In order to scan freely, it is necessary to transmit address information indicating the position of the pixel in the image together with the pixel information. Even if the address information is added, the pixel neighborhood correlation is utilized, so that overall efficiency is improved. Even so, if the address is transmitted as it is, there is too much information on the address. Thus, the previously proposed transmission method solves this problem by transmitting the relative position from the pixel of interest.
[0011]
In order to facilitate understanding of the present invention, the previously proposed image signal transmission apparatus will be described below with reference to the drawings. FIG. 1 shows a block diagram on the encoder side, and FIG. 2 shows a flowchart of the encoding process. FIG. 3 shows a block diagram on the decoder side, and FIG. 4 shows a flowchart of the decoding process.
[0012]
In FIG. 1, reference numerals 1a, 1b, and 1c are input terminals to which, for example, (4: 4: 4) color component signals (YUV) are input. An input component signal is supplied to the vectorization unit 2. In the vectorization unit 2, the input component signal is vectorized as a point in the YUV space. The output of the vectorization unit 2 is written into a frame memory 3 having a size for one screen (this memory 3 is called a vector memory).
[0013]
Scan signal as a result of the candidate search from the candidate search unit indicated by reference numeral 9 (which indicates the direction of the next transmission pixel as viewed from the transmitted target pixel, and is also referred to as scan direction and scan order data as appropriate hereinafter). Pixel of interest from vector memory 3 based on Vector v When , Scan information s for each candidate pixel and , For each candidate pixel The vector v is sent to the vector conversion unit 4. In the vector conversion unit 4, a difference vector from the target pixel to each candidate vector is calculated, and the difference vector is sent to the candidate evaluation unit 5.
[0014]
The local decoder 6 performs local decoding while referring to the difference vector table 7, supplies the decoded value to the candidate evaluation unit 5, and the candidate evaluation unit 5 determines a transmission pixel from the candidates. In the multiplexing unit 10, the scan signal s viewed from the target pixel of the transmission pixel and the index i of the representative vector are multiplexed, and transmission data is extracted to the output terminal 11. Further, the scan signal s is supplied from the candidate evaluation unit 5 to the scan memory 8. The output of the scan memory 8 is supplied to the candidate search unit 9, and the output of the candidate search unit 9 is given to the vector memory 3.
[0015]
The flow of the encoding process will be described with reference to the flowchart shown in FIG. In step S1, a pixel to be transmitted first is determined. For example, select the pixel in the middle of the image. However, the first pixel to be transmitted may be at any position. In step S2, the pixel is quantized and transmitted. The transmitted pixel is set as a target pixel (step S3).
[0016]
In step S4, candidates are acquired from candidate positions obtained from the scan memory 8. In step S5, it is determined whether the number of candidates reaches a specified number. If the scanning method is a zigzag, the number of four candidates is the specified number. If it is determined in step S5 that the number of candidates does not reach the specified number, candidates are acquired in the raster scan order until the specified number is reached (step S6).
[0017]
If the number of candidates reaches the specified number, in step S7, the transmission pixel is determined by local decoding using the selected table. In step S8, the scan signal and the index of the difference vector are transmitted. Steps S4, S6, and S7 are processes that require access to the memory. In step S9, attention is paid to the transmitted pixel. In step S10, it is determined whether all the pixels have been transmitted. If all the pixels are transmitted, the encoding process ends. If all the pixels have not been transmitted, the process returns to step S4 to transmit the next pixel.
[0018]
The decoder will be described with reference to FIG. Reference numeral 21 is an input terminal. The input data is supplied to the dividing unit 22, and the input data is divided into the scan signal s and the index information i. The output position determination unit 23 receives the scan signal s, determines the output position with reference to the scan memory 24, and outputs the output position data a. The vector reconstruction unit 25 reconstructs a vector from the index information i and outputs a vector v.
[0019]
The vector v is supplied to the componentization unit 27, and the vector v is converted into a component signal YUV. Component signals are supplied to the frame memory 28. The frame memory 28 is also supplied with output position data a. Then, the pixel value at the output position a is output from the frame memory 28 to the output terminal 29.
[0020]
The flow of the decoding process will be described with reference to the flowchart shown in FIG. In step S21, the signal from the encoder is received as the first pixel signal. In step S22, the received pixel data is reconstructed. In step S23, the reconstructed pixel data is output to a specified position. Steps S22 and S23 are processes that require access to the memory.
[0021]
In step S24, a signal from the encoder is received. In step S25, the position to be output in all directions is searched on the scan memory. In the direction where the output position is not determined, the output position is searched in the raster scan order (step S26). In step S27, the output position is determined. In step S28, pixel data is reconstructed from the index information. In step S29, the reconstructed pixel data is output to the frame memory. In step S30, it is determined whether all the pixels have been output. If all the pixels have been output, the decoding process ends. Otherwise, the process returns to step S24, and the next pixel is received.
[0022]
In a conventional raster scan, only pixel information is sent without sending address information by fixing the scan. On the other hand, in the proposed transmission method described above, the relative position from the target pixel is transmitted as a scan signal, but at this time, there is a problem of how to jump over the already transmitted pixel.
[0023]
With reference to FIG. 5, jumping over already transmitted pixels will be described. In FIG. 5, the target pixel is indicated by a double circle, and a pixel that is a transmission pixel candidate in the search is indicated by a light shadow. White circles indicate untransmitted pixels, and black circles indicate already transmitted pixels. As a search method for searching for a pixel to be transmitted, pixels located in four directions (up, down, left, and right) with respect to a target pixel indicated by a double circle are searched.
[0024]
In FIG. 5A, since all the pixels other than the target pixel are untransmitted pixels, as shown in FIG. 5B, four pixels adjacent to the target pixel in four directions are set as transmission pixel candidates. However, as shown in FIG. 5C, when the target pixel is surrounded by the transmitted pixels, the position where the transmitted pixels exist is specified only by the relative relationship of up, down, left and right. In such a case, the already transmitted pixels are skipped, and the untransmitted pixels are set as transmission pixel candidates. That is, as shown in FIG. 5D, since the two pixels located in the right direction of the target pixel are already transmitted pixels, the third pixel in the right direction from the target pixel is determined as a transmission pixel candidate by skipping the two pixels. . In addition, since one pixel in the lower direction of the target pixel is a pixel that has already been transmitted, the second pixel in the lower direction from the target pixel is determined to be a candidate for the transmission pixel.
[0025]
For this processing, the encoder and the decoder each have a memory for one frame (scan memories 8 and 24), and the transmitted pixels are flagged to distinguish the transmitted pixels from the untransmitted pixels. It should be noted that information on whether a pixel has been sent (whether it has been sent) or not sent (whether it has been sent) can be shared by the encoder and decoder, and there is no need to send this information.
[0026]
Next, the scan pattern will be described. In the conventional raster scan, a scan rule is set such that the images are sent one by one in order from the upper left of the screen. Therefore, there is no need to transmit any information regarding scanning. All bits can be sent as pixel data. However, since any image is scanned in the same way, it does not take advantage of the neighborhood correlation of images.
[0027]
On the other hand, when scanning at random, the address information must be sent. However, in the case of an image having a standard resolution per pixel only by address information, a data amount of 10 bits is required in both the x and y directions. Since the address information is information that must be sent accurately, the amount of data cannot be reduced by quantization or the like. As a result, the data amount of the address information is considerably heavy although efficiency is improved by using the neighborhood correlation.
[0028]
Therefore, in the proposed transmission method, transmission is made more efficient by giving freedom to scanning. “Give a degree of freedom” means that it is not completely fixed, but on the other hand it is not free at all. Based on a predetermined scan pattern, the relative position to the target pixel is transmitted.
[0029]
There can be various scan patterns. FIG. 6 shows examples of several scan patterns. FIG. 6A shows a conventional raster scan shown for reference. In the case of raster scanning, since the scanning direction is fixed, it is not necessary to transmit address information. In FIG. 6B, any of pixels located in four directions, up, down, left, and right with respect to the target pixel is transmitted. In this case, 2-bit address information is required for one pixel. The broken line in the upper left indicates that switching to raster scanning is performed when no candidate can be acquired.
[0030]
The scan pattern (zigzag) shown in FIG. 6C is for searching for candidates by zigzag scanning with respect to four regions divided around the transmission pixel. Therefore, it is possible to transmit the pixels to the end without causing the scan to get stuck. 2-bit address information is required for one pixel. In FIG. 6D, any of the pixels located in a total of eight directions, up and down, left and right, and oblique to the target pixel is transmitted. In this case, 3-bit address information is required for one pixel. The scan pattern in FIG. 6D selects transmission pixels from many candidates, so that it is possible to make better use of the neighborhood correlation, but the address information is increased by one bit compared to other scanning methods, and the pixels are increased accordingly. There is a problem that data that can be allocated is reduced by 1 bit.
[0031]
In the above transmission method, a candidate is selected based on a certain scan pattern, the difference vector in the difference vector table is applied to the candidate, and the index of the difference vector with the least error is transmitted. When the number of bits allocated to each pixel is determined, the number of candidates that can be acquired at one time and the number of difference vectors in the difference vector table are inversely proportional. An example in which 8 bits are assigned to one pixel is shown below.
[0032]
As in the scan pattern shown in FIG. 6D described above, when 8 candidates are acquired at a time, 3 bits are required to identify the candidates, so 5 bits can be used for the difference vector. The number of difference vectors is 32. When acquiring 4 candidates at a time, as in the scan pattern shown in FIG. 6B or 6C, 2 bits are required to identify the candidates, so 6 bits are used for the difference vector. And the number of difference vectors is 64. Furthermore, when two candidates are acquired at a time only in the horizontal direction, one bit is required for specifying the candidate, so 7 bits can be used for the difference vector, and the number of difference vectors Becomes 128. Further, when the scan pattern is fixed as in the conventional raster scan, all 8 bits can be assigned to the difference vector, so the number of difference vectors is 256.
[0033]
In this way, increasing the number of candidate acquisitions reduces the number of difference vectors, and increasing the number of difference vectors reduces the number of candidate acquisitions. Generally, when many candidates can be acquired, there are many options, so the number of difference vectors may be small, and vice versa. As a relationship between the number of candidate acquisitions and the number of difference vectors, a well-balanced one is considered to change depending on the image transmission status and the tendency of the image itself.
[0034]
The color space difference vector coding in the above transmission method will be described. As shown in FIG. 7, the pixel-of-interest data and transmission pixel data, which are component signals, are treated as vectors in the color space, and the difference vector is transmitted. This method can be considerably more efficient compared to generating the differences separately for each component of the component signal. As shown in FIG. 8A and FIG. 8B, the set of color space vectors of YUV components has a stronger correlation than the color space vector of RGB components, so the set is small. Further, the color space difference vector is formed in the vicinity of (0, 0) by forming the color space difference vector.
[0035]
As shown in FIG. 9, a vector representative of a set of color space difference vectors is obtained in advance, and a difference vector table is created. In the table, each difference vector is distinguished by an index. The difference vector table is shared between the encoder side and the decoder side, and an index is transmitted. If a zigzag is used as a scanning method with an 8-bit frame for one pixel, it can have 6-bit (64) representative vectors other than addresses. The vectorization unit 2 converts the component signal into a vector v in the color space.
[0036]
Furthermore, the candidate selection method and local decoding processing in the transmission method described above will be described with reference to FIG. This processing is performed by the vector conversion unit 4, the local decoder 6 and the candidate evaluation unit 5 in FIG. Moreover, in FIG. 2, it is made in step S7. In FIG. 10, a double circle pixel is a target pixel, a circle is a candidate pixel, and a number in a circle is a serial number assigned to the candidate pixel. Each candidate pixel is not necessarily adjacent to the target pixel, and is acquired from a certain direction in the scan pattern. As shown in FIG. 10A, it is assumed that there are, for example, four candidate pixels for one target pixel.
[0037]
First, it applies to the difference vector table which has in common on the encoding side and the decoding side with respect to the local decoding value of the pixel of interest. As an example, 6 bits 64 (2 ⁶ ) If there are difference vectors, 64 color vectors are created by adding the difference vectors to the local decode value of the pixel of interest as shown in FIG. 10B. These are currently available transmission pixel values.
[0038]
Next, as shown in FIG. 10C, the transmittable pixel values that can be generated from each candidate and the target pixel are compared, and the transmittable pixel value having the strongest correlation is determined for each candidate based on the comparison result. The transmittable pixel value having the strongest correlation is the optimum approximate value of each candidate. The pixel of interest pixel is supplied to the local decoder 6, and the local decoder 6 performs local decoding of the pixel of interest, processing for generating transmission pixel values that can be created as shown in FIG. 10B, and the generated 64 transmission pixel values and each candidate. A process of calculating a difference from the pixel value and a process of generating an optimum approximate value of each candidate are performed.
[0039]
Finally, as shown in FIG. 10D, a difference value between each candidate and the optimum transmittable pixel is obtained. The candidate having the smallest obtained difference value is determined as the candidate having the smallest transmission error among the four candidates, and the transmission pixel is determined. The direction of the determined transmission pixel viewed from the target pixel and the index of the difference vector are multiplexed and transmitted. As illustrated in FIG. 10D, the candidate evaluation unit 5 obtains an error after transmission of each candidate pixel and performs a process of determining a transmission pixel.
[0040]
Next, the present invention will be described. In the transmission method described above, an optimal scan order is obtained so that the difference between the pixel to be transmitted and the pixel to be transmitted next is minimized, thereby performing efficient transmission. In the present invention, the restriction of the optimal scan order is removed, and instead, information different from the transmission image is embedded.
[0041]
FIG. 11 shows a method for embedding information in a scan signal indicating the scan order. As shown in FIG. 11A, when the up / down / left / right scanning directions are possible, it is defined as (upward: (00), leftward: (01), rightward: (10), downward: (11)). . When the value of the embedded information is “151” and the binary number is (10010111), as shown in FIG. 11B, the embedded information is divided every two bits and converted into a scan signal. In this example, the scan order is (right → left → left → down). In the method of FIG. 11B, the true value of the embedded data is transmitted.
[0042]
In the previously proposed transmission method, the transmission efficiency is improved by realizing scanning adapted to the unique characteristics of the image. Embedding information with respect to scan order sacrifices this feature. Therefore, by embedding the embedded information in the scan order data, the image quality of the decoded image of the image that is originally desired to be transmitted deteriorates. However, as in one embodiment, by using a color space difference vector, it is possible to flexibly cope with the transmission of a pixel in which the difference is large, and it is possible to prevent the image quality deterioration of the transmission image from becoming large. it can.
[0043]
In another example shown in FIG. 11C, the lower 4 bits of 8 bits of the embedded information are free. Therefore, although the value of embedded information “144” (in binary, (10010000) or more is guaranteed, an error of the lower 4 bits (up to 16 errors) occurs. It is not defined by the scan signal. The lower 4 bits correspond to the most efficient scan order determined by applying the previously proposed transmission method, so compared to the method shown in FIG. The image quality of the transmitted image can be improved, and there is an error in the pixel value of the embedded image, but it is possible to determine what image is being sent, and there is little degradation depending on the image However, the lower 6 bits or 2 bits may be free.
[0044]
FIG. 12 shows three examples of embedded information. The first example of the embedded information is a reduced image B1 of transmission data (transmission image) I. The reduced image can be generated, for example, by subsampling the transmission image. For example, if ½ thinning is performed in the horizontal direction and the vertical direction, a reduced image of ¼ can be formed. The 8-bit code of each pixel of the reduced image is divided every 2 bits, and the scanning direction is defined by each 2 bits. On the receiving side, the reduced scan image can be obtained by treating the scan order data for four times as luminance data of one pixel and arranging the luminance data in the raster scan order. The reduced image can be viewed even if the receiving apparatus cannot decode the transmission data I. However, in the case where the reduced image is used as the embedded information, it is possible to transmit a true value of ¼ of the entire pixel value of the reduced image, and it is possible to improve the image quality using the true value. .
[0045]
A second example of the embedded information is an image B2 different from the transmission data I. Since an image completely different from the transmission image can be transmitted, it is possible to transmit a hidden image. If the image size is adjusted, a color image can be embedded. The embedded information is not limited to image information but may be audio information.
[0046]
A third example of the embedded information is digital data B3 used for process control and the like. For example, there is a case in which transmission data is decoded from received data, and processing for creating a resolution of the decoded transmission data is performed. The process of creating resolution is a process of creating a pixel that does not exist in the transmission image by calculating a prediction coefficient for each class that has been obtained in advance and a plurality of pixel values of the transmission image data. A class is determined by local features of an image, for example, a level distribution. When performing such resolution creation processing, the prediction coefficient is usually stored in the memory on the receiving side, but it is possible to transmit the prediction coefficient optimal for the processing of the transmission image data as embedded information. It becomes. Thereby, the memory for storing the prediction coefficient can be omitted. In addition, when a plurality of prediction coefficient tables are provided, information for switching the prediction coefficient table can be transmitted.
[0047]
Further, data that makes scanning itself more efficient may be embedded. On the transmission side (decoder side), simulation is performed considerably far in advance, and advantageous scan order data is sequentially transmitted. Alternatively, a table can be sent that will favor the transmission of the next frame.
[0048]
FIG. 13 schematically shows a system configuration to which the present invention is applied when image information is transmitted as embedded information. In the encoder denoted by reference numeral 101, the data of the embedded image 102 is converted into a scan signal 103 indicating the scan order. Pixel data 105 of the transmission image 104 stored in the buffer memory is output in the order indicated by the scan signal 103. In this case, in order to improve the efficiency of scanning, not only the pixel data itself but also a difference between pixel values may be transmitted, or a difference vector index as in the previously proposed transmission method may be transmitted. . These are collectively referred to as pixel data or pixel information. The pixel data and the scan signal are set as transmission data 106 and are transmitted or recorded on a recording medium.
[0049]
In the decoder indicated by reference numeral 201, the received transmission data 202 is divided into pixel data 203 and a scan signal 204. The pixel data 203 is stored at the pixel position indicated by the scan signal 204 in the buffer memory. Thereby, the transmission image 205 is restored. On the other hand, the scan signal 204 can be converted into the embedded image 206 as it is.
[0050]
The flow of the encoding process will be described with reference to the flowchart shown in FIG. In step S41, the embedded image data is converted into scan order data. In step S42, a pixel to be transmitted first is determined. For example, select the pixel in the middle of the image. However, the first pixel to be transmitted may be at any position. In step S43, the first pixel is transmitted. Attention is now paid to the transmitted pixel (step S44).
[0051]
In step S45, the pixel to be transmitted next is determined from the candidates in the scan order converted in step S41. In step S46, the determined transmission pixel is transmitted. In step S47, attention is now paid to the transmitted pixel. In step S48, it is determined whether all pixels have been transmitted. If all the pixels are transmitted, the encoding process ends. If all the pixels are not transmitted, the process returns to step S45 to continue the process.
[0052]
The flow of the decoding process will be described with reference to the flowchart shown in FIG. In step S51, transmission data is received. The pixel data sent at the beginning of the frame is received. In step S52, the received pixel data is reconstructed. In step S53, the reconstructed pixel data is output to a specified position, for example, the center position of the image.
[0053]
In step S54, data (after the second pixel) is received from the encoder. In step S55, the transmission data is divided into pixel data and scan order data, and the scan order data is converted into pixel values and output as an embedded image. In step S56, the scan memory is accessed to search for output positions in all directions. In step S57, with respect to the direction where there is no output position, a search is performed from the upper left of the screen in the raster scan order, and an empty pixel is applied as the output position. In step S58, the output position is determined from the scan order data. In step S59, pixel data is reconstructed. In step S60, the pixel of the transmission image is output. In step S61, it is checked whether all pixels have been output. If so, the process ends. Otherwise, the process returns to step S54, and the next pixel is received.
[0054]
FIG. 16 shows a configuration of an example of an encoder to which the present invention is applied. In FIG. 16, reference numerals 31a, 31b, and 31c are input terminals to which a color component signal (YUV) of (4: 4: 4), for example, is input. An input component signal is supplied to the vectorization unit 32. In the vectorization unit 32, the input component signal is vectorized as a point in the YUV space. The output of the vectorization unit 32 is written in a transmission image memory (frame memory) 33 having a size for one screen.
[0055]
Luminance data of the embedded image is supplied to the input terminal 34 and is written in the embedded image memory (frame memory) 35. For example, an 8-bit pixel value p read from the embedded image memory 35 is supplied to the conversion unit 36, and the pixel value p is divided every 2 bits to generate each 2-bit scan signal s. The scan signal s is supplied to the transmission pixel determination unit 37.
[0056]
The transmission pixel determining unit 37 refers to the contents of the scan memory 41 to determine the transmission pixel, reads out the determined transmission pixel vector v from the transmission image memory 33, and outputs it. In addition, the scan signal s is also output because it is transmitted. The scan memory 41 stores a flag indicating whether each pixel stored in the transmission image memory 33 is already transmitted or not transmitted.
[0057]
The transmission pixel vector v and the scan signal s are sent to the vector conversion unit 38. In the vector conversion unit 38, a difference vector between the local decode value of the previous transmission pixel and the local decode value of the current transmission pixel is calculated, and the difference vector is sent to the index determination unit 39. The index determination unit 39 determines the index i closest to the difference vector while referring to the difference vector table 40. The index i and the scan signal s from the index determination unit 39 are supplied to the multiplexing unit 42. In the multiplexing unit 42, the scan signal s and the index i of the representative vector viewed from the target pixel of the transmission pixel are multiplexed, and the transmission data is extracted to the output terminal 43. Further, the scan signal s is supplied from the index determination unit 39 to the scan memory 41, and the scan memory 41 is updated.
[0058]
FIG. 17 shows a configuration of an example of a decoder to which the present invention is applied. The decoder will be described with reference to FIG. Reference numeral 51 is an input terminal. The input data is supplied to the dividing unit 52, and the input data is divided into the scan signal s and the index information i. The scan signal s is supplied to the conversion unit 60, and for example, four consecutive scan signals are collected to generate an 8-bit pixel value p. Pixel value p is written into the embedded image memory 61. The order of writing pixel values to the embedded image memory 61 is the raster scan order. Further, the pixel values of the embedded image are read from the embedded image memory 61 in the raster scan order, and are extracted to the output terminal 62.
[0059]
A scan signal s from the dividing unit 52 is supplied to the output position determining unit 53. The output position determination unit 53 receives the scan signal s, determines the output position while referring to the scan memory 54, and outputs the output position data a. The vector reconstruction unit 55 reconstructs a vector by referring to the difference vector table 56 from the index information i, and outputs a vector v. The vector v is supplied to the componentization unit 57, and the vector v is converted into the component signal YUV. The component signal is written in the transmission image memory (frame memory) 58. The transmission image memory 58 is also supplied with the output position data a, and the pixel value at the position indicated by the output position data a is output from the transmission image memory 58 to the output terminal 59.
[0060]
FIG. 18 schematically shows a system configuration to which the present invention is applied when, for example, data for subsequent signal processing (referred to as control data) is transmitted as embedded information. In the encoder denoted by reference numeral 301, the data of the control data 302 is converted into a scan signal 303 indicating the scan order. Pixel data 305 of the transmission image 304 accumulated in the buffer memory is output in the order indicated by the scan signal 303. In this case, in order to improve the efficiency of scanning, not only the pixel data itself but also a difference between pixel values may be transmitted, or a difference vector index as in the previously proposed transmission method may be transmitted. . These are collectively referred to as pixel data or pixel information. The pixel data and the scan signal are set as transmission data 306 and are transmitted or recorded on a recording medium.
[0061]
In the decoder indicated by reference numeral 401, the received transmission data 402 is divided into pixel data 403 and a scan signal 404. Pixel data 403 is stored at a pixel position indicated by a scan signal 404 in the buffer memory. Thereby, the transmission image 405 is restored. On the other hand, the scan signal 404 is converted into the control data 406 as it is.
[0062]
The transmission image data restored by the decoder 401 is input to the signal processing unit 407. Control data 406 is supplied to the signal processing unit 407. The signal processing unit 407 is a resolution conversion unit that converts the resolution to a higher one using, for example, a class classification adaptive prediction method. As described above, the resolution conversion unit is a process of creating pixels that do not exist in the transmission image by calculating a prediction coefficient for each class that is obtained in advance and a plurality of pixel values of the transmission image data. When performing such a resolution conversion process, the control data 406 is transmitted with the optimal prediction coefficient for the transmission image data processing, information for switching the prediction coefficient table, the result of class classification in advance (class information), and the like. be able to. As a result, it is possible to speed up the processing in the signal processing unit 407, perform conversion processing with higher accuracy, and the like.
[0063]
The flow of the encoding process will be described with reference to the flowchart shown in FIG. Portions corresponding to those in the flowchart shown in FIG. 14 are encoded with the same reference numerals. In step S41 ′, the signal processing control data is converted into scan order data. In step S42, the first pixel to be transmitted is determined. In step S43, the first pixel is transmitted. In step S44, attention is paid to the pixel that has been transmitted.
[0064]
In step S45 ′, the next pixel to be transmitted is determined from the candidates in the scan order converted in step S41 ′. In step S46, the determined transmission pixel is transmitted, and in step S47, attention is paid to the transmitted pixel. In step S48, it is determined whether or not all the pixels have been transmitted. If all the pixels have been transmitted, the encoding process ends. If all the pixels have not been transmitted, the process returns to step S45 to perform the process. to continue.
[0065]
The flow of decoding processing will be described with reference to the flowchart shown in FIG. In step S51, transmission data is received, and in step S52, the received pixel data is reconstructed. In step S53, the reconstructed pixel data is output to a specified position, for example, the center position of the image.
[0066]
In step S54, data (after the second pixel) is received from the encoder. In step S55 ′, the transmission data is divided into pixel data and scan order data, and the scan order data is converted into control data and output as embedded data. In step S56, the scan memory is accessed to search for output positions in all directions, and in step S57, for the direction in which there is no output position, search is performed from the upper left of the screen in the raster scan order, and vacant pixels are assigned as output positions. . In step S58, the output position is determined from the scan order data, and in step S59, the pixel data is reconstructed. In step S60, the pixels of the transmission image are output. In step S61, it is checked whether all the pixels have been output. If so, the process ends. Otherwise, the process returns to step S54, and the next pixel is received.
[0067]
FIG. 21 shows a configuration of an example of an encoder to which the present invention is applied. Components corresponding to those of the encoder shown in FIG. 16 are denoted by the same reference numerals. Input component signals from the input terminals 31a, 31b, and 31c are supplied to the vectorization unit 32, and the input component signals are vectorized as one point in the YUV space. The output of the vectorization unit 32 is written in a transmission image memory (frame memory) 33 having a size for one screen.
[0068]
Embedded data (for example, control data for signal processing described above) is supplied to the input terminal 44 and written into the data memory 45. Data read from the data memory 45, for example, 8-bit data p is supplied to the conversion unit 36, and the data p is divided every 2 bits to generate each 2-bit scan signal s. A scan signal s is supplied to the transmission image memory 33.
[0069]
The transmission image memory 33 determines transmission pixels with reference to the contents of the scan memory 41, and outputs a vector v of the determined transmission pixels. In addition, the scan signal s is also output because it is transmitted. The scan memory 41 stores a flag indicating whether each pixel stored in the transmission image memory 33 is already transmitted or not transmitted. You may make it provide a transmission pixel determination part like the structure shown in FIG.
[0070]
The transmission pixel vector v and the scan signal s are supplied to the vector conversion unit 38, a difference vector is calculated, and the difference vector is sent to the index determination unit 39. The index determination unit 39 determines the index i closest to the difference vector with reference to the difference vector table 40, the index i and the scan signal s are supplied to the multiplexing unit 42, and the transmission data is taken out to the output terminal 43.
[0071]
FIG. 22 shows a configuration of an example of a decoder to which the present invention is applied. Components corresponding to those of the encoder shown in FIG. 17 are denoted by the same reference numerals. Input data from the input terminal 51 is supplied to the dividing unit 52, and the input data is divided into the scan signal s and the index information i. The scan signal s is supplied to the conversion unit 60, and for example, four consecutive scan signals are collected to generate 8-bit data p. Pixel value p is written to data memory 63. Data such as control data for signal processing is taken out from the data memory 63 to the output terminal 64.
[0072]
The output position determination unit 53 receives the scan signal s, determines the output position while referring to the scan memory 54, and outputs the output position data a. The vector reconstruction unit 55 reconstructs the vector v, the vector v is supplied to the componentization unit 57, and the vector v is converted into the component signal YUV. The component signal is written in the transmission image memory (frame memory) 58, and the pixel value at the position indicated by the output position data a is output from the transmission image memory 58 to the output terminal 59.
[0073]
The present invention is not limited to the above-described embodiment of the present invention, and various modifications and applications can be made without departing from the gist of the present invention. For example, in the case of random scanning for transmitting address information, the scanning method shown in FIG. 6C and FIG. 6D may be used instead of the scanning method shown in FIG. 6B. Furthermore, more complicated information can be embedded in the scan information by assigning bits in more detail according to the distance and angle between the target pixel and the transmission pixel, or by assigning a table index.
[0074]
【The invention's effect】
In the present invention, the embedded information can be transmitted using the scan information indicating the scan direction or the scan order, and more information can be transmitted. In the case of transmitting a reduced image of the transmission image, the image quality can be improved by using the true value in the reduced image when the transmission image is decoded. Further, by embedding control data for subsequent signal processing, better signal processing can be performed on the decoding side.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of an encoder in a transmission scheme previously proposed by the present applicant.
FIG. 2 is a flowchart showing a flow of an encoding process in the transmission scheme previously proposed by the applicant of the present application.
FIG. 3 is a block diagram showing a configuration of a decoder in the transmission scheme previously proposed by the present applicant.
FIG. 4 is a flowchart showing a flow of decoding processing in the transmission scheme previously proposed by the present applicant.
FIG. 5 is a schematic diagram for explaining a method of jumping over already transmitted pixels in the transmission method previously proposed by the applicant of the present application.
FIG. 6 is a schematic diagram for explaining a method for searching for a pixel to be transmitted in the transmission scheme previously proposed by the applicant of the present application.
FIG. 7 is a schematic diagram for explaining a color space difference vector in the transmission scheme previously proposed by the applicant of the present application.
FIG. 8 is a schematic diagram for explaining a color space difference vector in the transmission scheme previously proposed by the applicant of the present application.
FIG. 9 is a schematic diagram for explaining a color space difference vector in the transmission scheme previously proposed by the applicant of the present application.
FIG. 10 is a schematic diagram used for explaining the candidate selection method and local decoding in the transmission scheme previously proposed by the present applicant.
FIG. 11 is a schematic diagram for explaining a method of embedding information other than a transmission image in scan information according to the present invention.
FIG. 12 is a schematic diagram for explaining an example of embedded information in the present invention;
FIG. 13 is a block diagram for schematically explaining a system of embedding image information according to an embodiment of the present invention;
FIG. 14 is a flowchart showing a flow of an encoding process according to an embodiment of the present invention.
FIG. 15 is a flowchart showing the flow of decoding processing in one embodiment of the present invention.
FIG. 16 is a block diagram showing a configuration of an encoder according to an embodiment of the present invention.
FIG. 17 is a block diagram showing a configuration of a decoder in one embodiment of the present invention.
FIG. 18 is a block diagram schematically illustrating a system according to another embodiment of the present invention in which control data is embedded.
FIG. 19 is a flowchart showing a flow of an encoding process according to another embodiment of the present invention.
FIG. 20 is a flowchart showing the flow of decoding processing in another embodiment of the present invention.
FIG. 21 is a block diagram showing a configuration of an encoder according to another embodiment of the present invention.
FIG. 22 is a block diagram showing a configuration of a decoder according to another embodiment of the present invention.
[Explanation of symbols]
33... Transmission image memory, 35... Embedded image memory, 36... Conversion unit that converts data in scan order, 37... Transmission pixel determination unit, 45. Image memory 60... Conversion unit for converting scan signals into data 61. Embedded image memory 101, 301... Encoder, 201, 401.

Claims (7)

An image signal transmission device for transmitting a pixel together with address information indicating the position of the pixel to be transmitted when transmitting a pixel at an arbitrary position,
Vectorizing means for vectorizing the input component signal as a vector in color space ;
A memory for storing the vectorized component signal ;
A candidate selecting means for selecting a pixel at a preset position of the image as a target pixel, and selecting a plurality of adjacent pixels not yet transmitted from the target pixel as candidate pixels;
A transmission pixel determining means for acquiring a difference from the selected vector of target pixels and the selected vector of a plurality of candidate pixels and determining a candidate pixel having the smallest acquired difference as a transmission pixel;
Scan information generating means for generating scan information indicating the direction of the determined transmission pixel around the selected target pixel;
The address information is transmitted together with the quantized value of the selected pixel of interest, the embedded information consisting of several bits is embedded in a part of the generated scan information, and the determined transmission pixel vector is transmitted together with the scan information. Transmission means for
In the candidate selection means, the pixel that has been determined and transmitted as the transmission pixel in the transmission pixel determination means immediately before is newly selected as the target pixel from the memory, and has not yet been transmitted from the newly selected target pixel. A plurality of adjacent pixels are newly selected as candidate pixels,
In the transmission pixel determination means, an image signal transmission device that repeats a process of newly determining a transmission pixel from the newly selected pixel of interest and the plurality of newly selected candidate pixels . The image signal transmission apparatus according to claim 1, wherein the embedded information is reduced image information of a transmission image. The image signal transmission apparatus according to claim 1, wherein the embedded information is image information different from a transmission image. The image signal transmission apparatus according to claim 1, wherein the embedded information is information necessary for processing an image decoded on a decoding side. An image signal transmission method for transmitting a pixel together with address information indicating the position of the pixel to be transmitted when transmitting a pixel at an arbitrary position,
A vectorization step of vectorizing the input component signal as a vector in color space ;
Storing the vectorized component signal in a memory;
A candidate selection step of selecting a pixel at a preset position of the image as a target pixel and selecting a plurality of adjacent pixels not yet transmitted from the target pixel as candidate pixels;
A transmission pixel determining step of acquiring a difference from the selected vector of the target pixel and the selected vector of the plurality of candidate pixels, and determining a candidate pixel having the smallest acquired difference as a transmission pixel;
A scan information generation step for generating scan information indicating the direction of the determined transmission pixel around the selected pixel of interest;
The address information is transmitted together with the quantized value of the selected pixel of interest, the embedded information consisting of several bits is embedded in a part of the generated scan information, and the determined transmission pixel vector is transmitted together with the scan information. and a transmission step of,
In the candidate selection step, a pixel that has been determined and transmitted as a transmission pixel immediately before is newly selected as a target pixel from the memory, and a plurality of adjacent pixels that are not yet transmitted from the newly selected target pixel are selected. Newly selected as a candidate pixel,
In the transmission pixel determination step, an image signal transmission method in which processing for newly determining a transmission pixel from the newly selected pixel of interest and the plurality of newly selected candidate pixels is repeated . An input component signal is vectorized as a vector in a color space, the vectorized component signal is stored in a memory, a pixel at a preset position in the image is selected as a target pixel, and an adjacent pixel not yet transmitted from the target pixel Multiple candidate pixels are selected as candidate pixels, and a difference is acquired from the selected vector of the target pixel and the selected plurality of candidate pixel vectors, and the candidate pixel having the smallest acquired difference is transmitted. determining the pixel as the center pixel of interest above selected, it generates scan information indicating the direction of transmission pixels determined above, and transmits the address information with the quantized value of the elected target pixel, the generated It has been embedded embedded information consisting of several bits in a portion of the scanning information, and the determined together with the scan information An image signal receiving apparatus for receiving the transmitted data comprising a vector of transmission pixel,
Receiving means for receiving the transmission data;
Pixel value decoding means for decoding a pixel value from a vector of transmission pixels included in the received transmission data;
Output position determining means for determining the output position of the decoded pixel value from the scan information included in the received transmission data;
An image signal receiving apparatus comprising: embedded information restoring means for converting the scan information into the embedded information. An input component signal is vectorized as a vector in a color space, the vectorized component signal is stored in a memory, a pixel at a preset position in the image is selected as a target pixel, and an adjacent pixel not yet transmitted from the target pixel Multiple candidate pixels are selected as candidate pixels, and a difference is acquired from the selected vector of the target pixel and the selected plurality of candidate pixel vectors, and the candidate pixel having the smallest acquired difference is transmitted. determining the pixel as the center pixel of interest above selected, it generates scan information indicating the direction of transmission pixels determined above, and transmits the address information with the quantized value of the elected target pixel, the generated It has been embedded embedded information consisting of several bits in a portion of the scanning information, and the determined together with the scan information An image signal receiving method for receiving a transmission data including a vector of transmission pixel,
A receiving step for receiving the transmission data;
A pixel value decoding step of decoding a pixel value from a vector of transmission pixels included in the received transmission data;
An output position determining step for determining an output position of the decoded pixel value from the scan information included in the received transmission data;
An image signal receiving method comprising: an embedded information restoration step for converting the scan information into the embedded information.

Images