JP4548434B2 - Data processing apparatus, data processing method, and recording medium - Google Patents

Description

  The present invention relates to a data processing device, a data processing method, and a recording medium, and more particularly, to a data processing device, a data processing method, and a recording medium that enable, for example, high compression of an image or the like.

  For example, as a method for compressing images and other data, there is a method using entropy coding such as Huffman coding.

  Entropy coding is lossless coding in which a bit string is converted into a smaller number of bit strings using statistical properties, and corresponds to, for example, Huffman coding. In Huffman coding, data is compressed by assigning short codewords to values with high appearance frequency and assigning long codewords to values with low appearance frequency.

  For this reason, when an image is compressed by Huffman coding, for each pixel constituting the image, a difference from a pixel adjacent to the pixel (adjacent pixel) is calculated, and the difference value is encoded by Huffman coding. To be done.

  In other words, the frequency distribution of the pixel values of the pixels constituting the image has a certain degree of deviation, but generally does not have a great degree of deviation. On the other hand, the difference value is a difference between adjacent pixels and is generally concentrated in the vicinity of 0. Therefore, the frequency distribution has a relatively steep and large deviation with the maximum frequency near 0. It becomes.

  Therefore, by performing Huffman coding for the difference value, the compression rate can be greatly improved compared to the case where the pixel value itself is targeted.

  Here, if the original pixel value is represented by N bits, the simple difference value between the pixel values is a value represented by N + 1 bits at the maximum. That is, for example, if the pixel value is represented by 8 bits in the range of 0 to 255, such a simple difference between the pixels becomes a value in the range of -255 to +255, and the representation is 9 bits. Is required. In this case, since the number of values that the difference value can take is almost twice the number of values that the original pixel value can take, the number of codewords required in Huffman coding also doubles, and the compression rate is reduced. It will deteriorate.

  Therefore, a method for preventing the increase in the number of bits of the difference value as described above by determining the magnitude relationship between two pixel values for which a difference is obtained and changing the method for calculating the difference value based on the determination result. There is.

  That is, when the target pixel (the pixel value) A of interest for calculating the difference value is equal to or greater than the adjacent pixel (the pixel value) B adjacent thereto, the adjacent pixel B is subtracted from the target pixel A. The subtraction value is used as difference data C as it is. Specifically, for example, when the target pixel value A is 200 and the adjacent pixel value B is 10, 200−10 = 190 is set as the difference value C.

On the other hand, when the target pixel A is smaller than the adjacent pixel B, the adjacent pixel B is subtracted from the target pixel A, a certain value is added to the subtraction value, and the addition result is set as a difference value. That is, if the pixel is expressed by N bits, 2 ^N is added to the subtraction value obtained by subtracting the adjacent pixel B from the target pixel A, and the addition result is set as a difference value C. Specifically, for example, when the pixel is represented by 8 bits, the target pixel value A is 10, and the adjacent pixel value B is 200, 10−200 + 2 ⁸ = 66 is set as the difference value C.

  By obtaining the difference value C as described above, the number of bits of the difference value C is not different from the number of bits of the original pixel (if the original pixel is expressed by N bits, the difference Data can also be expressed in N bits). Therefore, it is possible to prevent a reduction in compression rate due to an increase in the number of bits of the difference value.

  Note that the difference value C obtained as described above can be restored to the original pixel (target pixel) A as follows.

That is, first, the difference value C is added to the adjacent pixel B, and the added value is obtained. Now, assuming that the pixel is represented by N bits, if the addition value is 2 ^N −1 or less, the addition value is directly used as the restoration result of the original pixel A.

On the other hand, when the addition value of the difference value C and the adjacent pixel B is larger than 2 ^N −1, 2 ^N is subtracted from the addition value. In this case, the subtraction value becomes the restoration result of the original pixel A.

  As described above, the difference between a certain pixel A and another pixel B can be expressed without increasing the number of bits, and the difference is restored to the original pixel A. Can do.

In this case, the frequency distribution of the difference values has a relatively steep and large bias with the maximum frequencies near 0 and 2 ^N −1.

  As described above, the frequency distribution of the difference values between adjacent pixels is a distribution having a relatively steep and large bias around a certain value, but the frequency distribution having a steeper and larger bias with respect to an image. Is obtained, a higher compression ratio can be realized.

  The present invention has been made in view of such a situation, and makes it possible to compress images and other data at a higher compression rate.

The data processing device according to the first aspect of the present invention selects first data, and uses other first data having a predetermined positional relationship with the selected first data as reference data. Selection means for selecting, conversion means for converting the first data into the second data based on a conversion table generated for each value of the reference data and associated with the first and second data; The frequency distribution calculating means for obtaining the frequency distribution of at least a part of the second data, and the change target data to be changed according to the third data are determined from the part of the data according to the frequency distribution. The change means data is changed to the change data according to the determination means for determining the change data that is a value for changing the change target data, and the third data, so that the second data is changed to the third data. And a coding means for embedding data relating to, the conversion table, the data of a predetermined value of the first data as the reference data, among the first data, a predetermined position with respect to the reference data A frequency distribution of the related ones is generated for each value of the reference data, and based on the frequency distribution of the first data for each value of the reference data, the frequency distribution of each value for which the frequency distribution is obtained with respect to the reference data The second data is assigned to the first data in ascending or descending order of the frequency, and is generated for each value of the reference data .

  In the data processing device according to the first aspect, the determining means can determine data having the maximum frequency as data to be changed from some data.

  In the data processing device according to the first aspect, the determining means can determine a value having a frequency of 0 as change data among the values that the second data can take.

  In the data processing device according to the first aspect, the determining means can determine a plurality of values as change data.

  In the data processing device according to the first aspect, the conversion table uses the predetermined value data of the first data as reference data, and temporally or spatially with respect to the reference data of the first data. It is assumed that a frequency distribution of those close to each other is generated for each value of the reference data, and is generated for each value of the reference data based on the frequency distribution of the first data for each value of the reference data. Can do.

  In the data processing device according to the first aspect, the conversion table uses the predetermined value data of the first data as reference data, and temporally or spatially with respect to the reference data of the first data. A frequency distribution of adjacent objects can be generated for each value of the reference data, and can be generated for each value of the reference data based on the frequency distribution of the first data for each value of the reference data. .

  In the data processing device according to the first aspect, the first data can be a pixel value constituting an image.

  The data processing apparatus according to the first aspect can further include storage means for storing a conversion table for each value of the reference data.

  The data processing apparatus according to the first aspect may further include a generating unit that generates a conversion table for each value of the reference data.

In the data processing method according to the first aspect of the present invention, the first data is selected, and other first data having a predetermined positional relationship with the selected first data is used as reference data. A selection step for selecting, a conversion step for converting the first data into the second data based on a conversion table generated for each value of the reference data and associated with the first and second data; A frequency distribution calculating step for obtaining a frequency distribution of at least a part of the second data, and determining change target data to be changed according to the third data from the part of the data according to the frequency distribution. The change data is changed to the change data according to the determination step for determining the change data that is a value for changing the change target data and the third data. In, and a coding step of embedding data relating to the third data, the conversion table, the data of a predetermined value of the first data as the reference data, among the first data, the reference data Is generated for each value of the reference data, and based on the frequency distribution of the first data for each value of the reference data, a frequency distribution is generated for the reference data. The second data is assigned to the first data of each obtained value in the ascending or descending order of the frequency, and is generated for each value of the reference data .

The recording medium according to the first aspect of the present invention selects first data and selects other first data having a predetermined positional relationship with the selected first data as reference data. A conversion step for converting the first data into the second data based on a conversion table generated for each value of the reference data and associated with the first and second data, A frequency distribution calculating step for obtaining a frequency distribution of at least a part of the data of 2 and determining a change target data to be changed according to the third data from a part of the data according to the frequency distribution; By changing the change target data to the change data according to the determination step for determining the change data that is a value for changing the change target data and the third data, Program and a coding step of embedding data relating to the third data and is recorded, the conversion table, the data of a predetermined value of the first data as the reference data, among the first data A frequency distribution of a predetermined positional relationship with respect to the reference data is generated for each value of the reference data, and based on the frequency distribution of the first data for each value of the reference data, The frequency distribution is generated for each value of the reference data by assigning the second data to the first data of each value for which the frequency distribution is obtained in ascending or descending order of the frequency. To do.

According to a second aspect of the present invention, there is provided a data processing device comprising: a frequency distribution calculating means for obtaining a frequency distribution of at least a part of encoded data; and a third data from a part of the data based on the frequency distribution. A change means for determining change data that has been changed in accordance with the change target data, and change data that is a value obtained by changing the change target data; And decoding means for decoding the third data embedded in the encoded data, selecting the second data, and having a predetermined positional relationship with the selected second data Selection means for selecting first reversely converted first data as reference data, and a conversion table generated for each value of the reference data and associated with the first and second data Based on the second data, and a reverse conversion means for inversely converting the first data, the conversion table, the data of a predetermined value of the first data as the reference data, the first data A frequency distribution of those having a predetermined positional relationship with respect to the reference data is generated for each value of the reference data, and based on the frequency distribution of the first data for each value of the reference data, the reference data Is generated for each value of the reference data by assigning the second data in ascending or descending order of the frequency to the first data of each value for which the frequency distribution is obtained for It is characterized by.

  In the data processing device according to the second aspect, the determining means can determine data in which the change in frequency is discontinuous from the encoded data as the change target data or the change data.

  In the data processing device according to the second aspect, the determining means can determine a plurality of values as change data.

  In the data processing device according to the second aspect, the conversion table uses data of a predetermined value in the first data as reference data, and temporally or spatially with respect to the reference data in the first data. It is assumed that a frequency distribution of those close to each other is generated for each value of the reference data, and is generated for each value of the reference data based on the frequency distribution of the first data for each value of the reference data. Can do.

  In the data processing device according to the second aspect, the conversion table uses data of a predetermined value in the first data as reference data, and temporally or spatially with respect to the reference data in the first data. A frequency distribution of adjacent objects can be generated for each value of the reference data, and can be generated for each value of the reference data based on the frequency distribution of the first data for each value of the reference data. .

  In the data processing device according to the second aspect, the first data can be a pixel value constituting an image.

  The data processing apparatus according to the second aspect can further include storage means for storing a conversion table for each value of the reference data.

  The data processing apparatus according to the second aspect can further include an acquisition unit that acquires a conversion table for each value of the reference data.

The data processing method according to the second aspect of the present invention includes a frequency distribution calculating step for obtaining a frequency distribution of at least a part of encoded data, a third data from a part of the data based on the frequency distribution. A determination step for determining change target data changed in accordance with the change target data and a change data that is a value obtained by changing the change target data; and changing the change data to the change target data to convert the encoded data into the second data And decoding the third data embedded in the encoded data, selecting the second data, and having a predetermined positional relationship with the selected second data A selection step of selecting the first inversely transformed first data as the reference data, and the first and second data are associated with each other, generated for each value of the reference data The Based on the conversion table, the second data, and a reverse transformation step for inverse transformation to the first data, the conversion table, the data of a predetermined value of the first data as reference data, A frequency distribution of the first data having a predetermined positional relationship with the reference data is generated for each value of the reference data, and based on the frequency distribution of the first data for each value of the reference data The second data is assigned to the first data of each value for which the frequency distribution is obtained with respect to the reference data, in ascending or descending order of the frequencies. It is characterized by being.

The recording medium according to the second aspect of the present invention includes a frequency distribution calculating step for obtaining a frequency distribution of at least a part of encoded data, and a third data from a part of the data based on the frequency distribution. Accordingly, a determination step for determining the changed change target data and the change data that is a value obtained by changing the change target data, and changing the change data to the change target data, A decoding step for decoding the data and decoding the third data embedded in the encoded data, and selecting the second data and having a predetermined positional relationship with the selected second data A selection step of selecting first data that has already been inversely converted as reference data, and each value of the reference data is generated and associated with the first and second data Based on the conversion table, the second data, and a program and an inverse conversion step of inversely transforming is recorded in the first data, the conversion table, the data of a predetermined value of the first data As the reference data, a frequency distribution of the first data having a predetermined positional relationship with the reference data is generated for each value of the reference data, and the first data for each value of the reference data is generated. By assigning the second data in ascending or descending order of the frequency to the first data of each value for which the frequency distribution is obtained with respect to the reference data based on the frequency distribution, each value of the reference data It is characterized by being generated .

In the data processing device, the data processing method, and the recording medium according to the first aspect of the present invention, the first data is selected, and the other is in a predetermined positional relationship with the selected first data The first data is selected as reference data. Further, the first data is converted into the second data based on the conversion table generated for each value of the reference data and associated with the first and second data. Further, a frequency distribution of at least a part of the second data is obtained, and change target data to be changed according to the third data is determined from the part of the data according to the frequency distribution. Then, change data that is a value for changing the change target data is determined. Then, according to the third data, the change target data is changed to the change data, whereby data related to the third data is embedded in the second data. The conversion table uses the predetermined value data of the first data as the reference data, and the frequency distribution of the first data in the predetermined positional relationship with respect to the reference data Generated for each value, and based on the frequency distribution of the first data for each value of the reference data, for the first data of each value for which the frequency distribution is obtained for the reference data, The second data is assigned in ascending or descending order, and is generated for each reference data value.

In the data processing device, the data processing method, and the recording medium according to the second aspect of the present invention, the frequency distribution of at least a part of the encoded data is obtained, and based on the frequency distribution, The change target data changed according to the third data and the change data that is a value obtained by changing the change target data are determined. Furthermore, by changing the change data to the change target data, the encoded data is decoded into the second data, and the third data embedded in the encoded data is decoded. Thereafter, the second data is selected, and is in a predetermined positional relationship with the selected second data.
The first data that has already been inversely transformed is selected as reference data. Then, based on the conversion table that is generated for each value of the reference data and the first and second data are associated with each other,
The second data is inversely converted to the first data. The conversion table uses the predetermined value data of the first data as the reference data, and the frequency distribution of the first data in the predetermined positional relationship with respect to the reference data Generated for each value, and based on the frequency distribution of the first data for each value of the reference data, for the first data of each value for which the frequency distribution is obtained for the reference data, The second data is assigned in ascending or descending order, and is generated for each reference data value.

  According to the first aspect of the present invention, a large amount of data can be embedded. Also, according to the second aspect of the present invention, encoded data in which a large amount of data is embedded can be restored to the original data.

  FIG. 1 shows a configuration example of an embodiment of an image conversion apparatus to which the present invention is applied.

  In this image conversion apparatus, the image data is converted into data having a frequency distribution that is steep and has a large deviation beyond the difference value between adjacent pixels.

  That is, digital image data to be converted is supplied to the frame memory 1 for each frame, for example, and the frame memory 1 sequentially stores the image data supplied thereto. .

  The code table creation unit 2 creates (generates) a code table used for converting the pixel values constituting the image data for each possible value of the pixel values constituting the image data, and supplies the code table to the code table storage unit 3 To do.

  The code table creation unit 2 can create a code table to be used for each frame by using the image data for each frame stored in the frame memory 1, or is prepared in advance for creating a code table. It is also possible to create a code table that is commonly used for all frames by using the image data.

  The code table storage unit 3 temporarily stores the code table supplied from the code table creation unit 2.

  The conversion target pixel acquisition unit 4 reads (acquires) the pixels (pixel values) constituting the frame stored in the frame memory 1 in the so-called raster scan order, and the read pixels are stored in the code table storage unit 3. The data is supplied to the conversion unit 6 as an object to be converted according to the code table (hereinafter, appropriately referred to as a conversion target pixel).

  The reference pixel acquisition unit 5 reads (acquires) a pixel having a predetermined positional relationship with respect to the conversion target pixel from the frame memory 1 and outputs the pixel as a reference pixel to the conversion unit 6. That is, the reference pixel acquisition unit 5 reads, as a reference pixel, a pixel at a close position, such as a pixel that is spatially or temporally adjacent to the conversion target pixel, and outputs it to the conversion unit 6. . In the present embodiment, the reference pixel acquisition unit 5 reads, for example, the pixel adjacent to the left of the conversion target pixel from the frame memory 1 as a reference pixel.

  The conversion unit 6 refers to the code table stored in the code table storage unit 3 with respect to the pixel value of the reference pixel from the reference pixel acquisition unit 5 among the code tables for each possible pixel value, and stores the code table in the code table. Therefore, the pixel value of the conversion target pixel from the conversion target pixel acquisition unit 4 is converted. Further, the conversion unit 6 supplies the converted pixel value to the frame memory 7.

  The frame memory 7 stores the pixel value after conversion of the conversion target pixel from the conversion unit 6 at the address of the corresponding position, and when the pixel value for one frame is stored, an image having such a pixel value ( Hereinafter, the converted image is output as appropriate.

  Next, FIG. 2 shows a configuration example of the code table creation unit 2 of FIG.

  The frame memory 11 is supplied with code table creation image data that is image data used to create a code table, and the frame memory 11 stores the code table creation image data.

  Here, as described above, the code table creation image data may be image data stored in the frame memory 1 of FIG. 1 or image data prepared in advance for creating the code table. Also good.

  The pixel selection unit 12 selects a pixel having a certain pixel value as a reference pixel among the pixels constituting the frame of the image data stored in the frame memory 11. Further, the pixel selection unit 12 converts the conversion target pixel acquired by the conversion target pixel acquisition unit 4 of FIG. 1 for the selected reference pixel, and the reference pixel acquired by the reference pixel acquisition unit 5 for the conversion target pixel. Are further selected from the frame memory 11.

  That is, in the present embodiment, as described above, the reference pixel acquisition unit 5 acquires the pixel adjacent to the left as the reference pixel for the conversion target pixel acquired by the conversion target pixel acquisition unit 4. Therefore, as shown in FIG. 3, the pixel selection unit 12 selects the pixel on the right side of the reference pixel selected by itself. In this way, the pixel selection unit 12 selects a pixel corresponding to the pixel that the conversion target pixel selection unit 4 in FIG. 1 acquires as the conversion target pixel.

  Here, the pixel corresponding to the conversion target pixel that the pixel selection unit 12 selects with respect to the reference pixel is hereinafter referred to as a selection pixel as appropriate.

  The reference pixel selected by the pixel selection unit 12 and the selection pixel corresponding to the reference pixel are supplied to the histogram creation unit 13.

  As shown in FIG. 4, the histogram creation unit 13 calculates the frequency distribution (histogram) of the pixel values (for example, luminance values) of the selected pixels selected by the pixel selection unit 12 with respect to the reference pixels, and the pixel values of the reference pixels. Each code is generated and supplied to the code generation unit 14.

  Based on the frequency distribution of the selected pixel for each pixel value of the reference pixel supplied from the histogram creation unit 13, the code generation unit 14 sets the pixel value of the selected pixel and the pixel value for each pixel value of the reference pixel. A code table in which the allocation code to be allocated is associated is generated and supplied to the code table storage unit 3 (FIG. 1).

  Here, the code table generated in the code generation unit 14 based on the frequency distribution of the selected pixels will be described.

  FIG. 5 shows a frequency distribution of selected pixels obtained using an actual image.

  That is, FIG. 5 shows a frequency distribution of pixel values of a selected pixel that is a pixel on the right side when a pixel having a pixel value of 20 is set as a reference pixel in an image represented by 8 bits. When a pixel having a value of 128 is set as a reference pixel, a frequency distribution of pixel values of a selected pixel which is a pixel on the right side of the pixel is shown.

  From FIG. 5, the frequency distribution of the pixel value of the selected pixel that is a pixel right next to the reference pixel of a certain pixel value is not a uniform distribution but a non-uniform distribution, and the pixel value of the reference pixel is centered. The distribution is not symmetrical (often).

  For this reason, the difference value obtained by subtracting the pixel on the left (a reference pixel in this embodiment) from a certain pixel does not necessarily have an optimal bias.

  That is, for example, assume that the frequency distribution of the pixel value of the pixel (selected pixel) on the right side of the reference pixel having a certain pixel value is as shown in FIG. Here, in FIG. 6A, the horizontal axis represents the pixel value of the right adjacent pixel based on the pixel value of the reference pixel, that is, subtracts the reference pixel from the right adjacent pixel of the reference pixel. The vertical axis represents the frequency (frequency) of the differential value.

  Representing a pixel value as a difference value with the pixel (reference pixel) on the left of that pixel means that the smaller the absolute value of the difference value, the higher the frequency, and the higher the absolute value of the difference value, the higher the frequency. This is equivalent to assigning codes to each pixel value in ascending order (or descending order) of the absolute value of the difference value.

  Therefore, in the expression using the difference value, for example, as shown in FIG. 6B, the code value 0 is used for the pixel value having the difference value 0, and the code 1 is used for the pixel value having the difference value +1. Code 2 is assigned to the pixel value of −1, code 3 is assigned to the pixel value of difference value +2, and code 3 is assigned to the pixel value of difference value −2.

  In this case, the frequency distribution in which the assigned code (assignment code) is on the horizontal axis is as shown in FIG. 6 (C), and macroscopically (as a whole), the frequency increases as the code increases. Microscopically, the frequency does not decrease monotonously but has some unevenness. This unevenness is caused by the fact that in the frequency distribution shown in FIG. 6A, the smaller (larger) absolute value of the difference value does not necessarily become higher (lower).

  As described above, the frequency distribution of the allocation code allocated based on the difference value from the reference pixel for the pixel (selected pixel) on the right side of the reference pixel having a certain pixel value has an unevenness as shown in FIG. Therefore, the frequency distribution of the allocation code for the pixel on the right side of the reference pixel of another pixel value is similarly uneven.

  Therefore, since the frequency distribution of the allocation code based on the difference value for the entire image is a macro addition of the frequency distribution with respect to the reference pixel of each pixel value, the frequency decreases as the code increases. Is a distribution having a certain degree of steepness and a certain degree of bias. However, since the frequency distribution of the allocation code based on the difference value for the entire image is a microscopic addition of the frequency distribution with respect to the reference pixel of each pixel value having the unevenness as described above, its steepness is obtained. The thickness and the deviation are dull due to unevenness.

  Therefore, the code generation unit 14 generates a code table that assigns the following assignment codes to the pixel values so that the frequency distribution is steeper and more biased. Yes.

  That is, for example, as the frequency distribution of the pixel value of the pixel (selected pixel) right next to the reference pixel having a certain pixel value, the histogram generation unit shown in FIG. 7A similar to FIG. 13 is supplied to the code generation unit 14.

  In this case, based on the frequency distribution from the histogram creation unit 13, the code generation unit 14 performs ascending order of the frequency of the pixel values (or pixel values with reference to the reference pixel) of the selected pixels (or Assign codes in descending order.

  Therefore, in the frequency distribution shown in FIG. 7A, pixel values (pixel values with reference to the reference pixel) are 0, +1, +2, -1, +3, -2, +4, +5, -3, Since the frequency decreases in the order of −4 (−5),..., In the code generation unit 14, as shown in FIG. For example, a code table to which an integer code from 0 is assigned is created.

  In this case, the frequency distribution with the assigned code (assignment code) on the horizontal axis is a monotonically decreasing distribution as shown in FIG.

  Similarly, the frequency distribution of the allocation code assigned based on the frequency distribution of the pixel values with reference to the reference pixel for the pixel (selected pixel) to the right of the reference pixel of another pixel value is also shown in FIG. The distribution is monotonically decreasing as shown.

  Accordingly, in this case, since the frequency distribution of the allocation code of the entire image is obtained by adding the frequency distributions of monotonous decrease as described above, the frequency distribution is steeper compared to the case where the difference value expression is described in FIG. And a distribution with a larger bias.

  As described above, the code assigned to the pixel value is determined based on the frequency of the pixel value of the pixel immediately adjacent to the reference pixel, that is, the existence probability of the pixel value, and converted into such a code. This frequency distribution is steeper and has a larger bias.

  As a result, based on the code table shown in FIG. 7B, an image (converted image) composed of pixels obtained by converting pixel values into allocation codes (hereinafter referred to as converted pixels as appropriate) has a higher compression rate. Can be compressed.

  Next, a conversion process for converting an image into a converted image performed by the image conversion apparatus in FIG. 1 will be described with reference to the flowchart in FIG.

  The frame memory 1 sequentially supplies and stores images to be converted, and the code table creation unit 2 is supplied with a code table creation image. Here, for example, it is assumed that the image to be converted stored in the frame memory 1 is supplied as a code table creation image used to create a code table for converting the image.

  In step S1, the code table creation unit 2 performs code table creation processing using the image of the frame of interest, which is the frame that is the object of conversion, stored in the frame memory 1, and thereby the reference pixel A code table is created for each pixel value that can be taken by the pixel. The code table for each pixel value is supplied to and stored in the code table storage unit 3, and the process proceeds to step S2.

  In step S <b> 2, the conversion target pixel acquisition unit 4 reads out pixels that have not yet been converted as pixels to be converted in the raster scan order among the pixels constituting the target frame of the frame memory 1 as a conversion target pixel. 6 is supplied. Furthermore, in step S <b> 2, the reference pixel acquisition unit 5 reads out the pixel adjacent to the left of the conversion target pixel from the frame memory 1 as a reference pixel, and supplies it to the conversion unit 6.

  Then, the process proceeds to step S3, and the conversion unit 6 converts the code table for each value that can be taken by the pixel value stored in the code table storage unit 3 with respect to the pixel value of the reference pixel from the reference pixel acquisition unit 5. With reference to the code table, the pixel value of the conversion target pixel from the conversion target pixel acquisition unit 4 is converted into an allocation code associated with the pixel value, and the process proceeds to step S4. In step S4, the conversion unit 6 supplies the conversion target pixel obtained by converting the pixel value into the allocation code, that is, the conversion pixel, to the frame memory 7 and stores it in the address of the corresponding position.

  Thereafter, the process proceeds to step S5, and it is determined whether or not the conversion of the pixel value is performed with all the pixels constituting the target frame as the conversion target pixels. In step S5, when it is determined that all of the pixels constituting the frame of interest are conversion target pixels and the pixel values have not yet been converted, the process returns to step S2, and the same processing is repeated thereafter.

  If it is determined in step S5 that all the pixels constituting the target frame have been converted as pixel values to be converted, that is, the converted image corresponding to the target frame is stored in the frame memory 7. In this case, a converted image is read from the frame memory 7 and a code table (a set of code tables for each pixel value) stored in the code table storage unit 3 and used to obtain the converted image is read. Is output.

  In step S6, it is determined whether or not the frame memory 1 stores the next frame of the frame of interest. If it is determined in step S6 that the next frame of the frame of interest is stored, the next frame is newly set as the frame of interest, and the process returns to step S1, and the same processing is repeated thereafter.

  If it is determined in step S6 that the frame next to the frame of interest is not stored in the frame memory 1, the process ends.

  According to the flowchart of FIG. 8, each time an image of a certain frame is converted into a converted image, a code table used for the conversion is created. Therefore, a code table for the frame can be obtained for each frame. become.

  However, the code table can be created for each image of two frames or more, for example.

  The code table can be created in advance using a predetermined image and stored in the code table storage unit 3. In this case, the code table creation unit 2 is unnecessary in the image conversion apparatus of FIG. 1 (therefore, the process of step S1 of FIG. 8 is also unnecessary). Furthermore, in this case, only one set of code tables is required to convert a series of images.

  Next, with reference to the flowchart of FIG. 9, the code table creation process performed by the code table creation unit 2 (FIG. 2) performed in step S1 of FIG. 8 will be further described.

  The frame memory 11 is supplied with and stored code table creation image data.

  In step S11, the pixel selection unit 12 sets one of the possible values of the pixel values constituting the image data as the pixel value of the reference pixel, and proceeds to step S12. In step S12, the pixel selection unit 12 searches the code table creation image data in the frame memory 11 for a pixel having the pixel value set in step S11 (hereinafter referred to as a set pixel value as appropriate) as a reference pixel. The pixel on the right side of the reference pixel is selected as the selected pixel and supplied to the histogram creation unit 13.

  In step S <b> 13, the histogram creation unit 13 creates a frequency distribution (histogram) of the pixel values of the selected pixels from the pixel selection unit 12 and supplies the frequency distribution to the code generation unit 14. In step S14, the code generation unit 14 creates a code table for the pixel value (set pixel value) that is currently the reference pixel based on the frequency distribution from the histogram creation unit 13 as described in FIG. To do.

  That is, based on the frequency distribution from the histogram creation unit 13, the code generation unit 14 calculates, for each pixel value (pixel value based on the reference pixel) of the pixel (selected pixel) on the right side of the reference pixel, that pixel. Codes are assigned in ascending order of the frequency of values, thereby creating a code table for pixel values (set pixel values) that are now reference pixels.

  Then, the process proceeds to step S15, and the pixel selection unit 12 determines whether or not processing has been performed using all the possible values of the pixel values constituting the image data as the set pixel values. In step S15, when it is determined that all the values that can be taken by the pixel values constituting the image data are not set as the set pixel values, the process returns to step S11, and the pixel selection unit 12 determines that the image data Among the possible values of the pixel values constituting the pixel value, one of the pixel values that are not yet set pixel values is set as a new set pixel value, and the same processing is repeated thereafter.

  On the other hand, if it is determined in step S15 that all of the values that can be taken by the pixel values constituting the image data have been processed as the set pixel values, that is, all the values that can be taken by the pixel values that make up the image data. On the other hand, when the code table is obtained, the code generation unit 14 outputs the code table to the code table storage unit 3 (FIG. 1) and returns.

  In the image conversion apparatus of FIG. 1 (the same applies to the image reverse conversion apparatus of FIG. 10 described later), if a pixel adjacent to the left or right of the leftmost or rightmost pixel of the frame is necessary, the left adjacent or right The adjacent pixel is processed, for example, as having the same pixel value as the pixel at the left end or the right end, or having a predetermined pixel value (for example, 0).

  Next, FIG. 10 shows a configuration example of an embodiment of an image reverse conversion device that reversely converts the converted image output from the image conversion device of FIG. 1 into the original image.

  The converted image and code table output from the image conversion apparatus in FIG. 1 are supplied to the frame memory 23 and the code table acquisition unit 21, respectively.

  The code table acquisition unit 21 acquires a code table for each pixel value supplied thereto and supplies the code table to the code table storage unit 22. The code table storage unit 22 stores the code table from the code table acquisition unit 21.

  The frame memory 23 sequentially stores the converted images supplied thereto, for example, in units of one frame. The inverse conversion target pixel acquisition unit 24 acquires the conversion pixels constituting the conversion image stored in the frame memory 23 sequentially as an object of reverse conversion (hereinafter referred to as an inverse conversion target pixel as appropriate), for example, in raster scan order. And supplied to the inverse transform unit 25.

Inverse conversion target pixels are supplied from the reverse conversion target pixel acquisition unit 24 to the reverse conversion unit 25, and reference pixels are supplied from the reference pixel acquisition unit 27. The inverse conversion unit 25 refers to the code table for each reference pixel value from the reference pixel acquisition unit 27 out of the code tables for each value that can be taken by the pixel value stored in the code table storage unit 22, and the code table. Accordingly, the allocation code as the pixel value of the inverse conversion target pixel from the inverse conversion target pixel acquisition unit 24 is inversely converted to the original pixel value.
Further, the inverse transform unit 25 supplies the pixel value after the inverse transform to the frame memory 26.

  The frame memory 26 stores the pixel value (original pixel value) of the inversely converted pixel from the inverse conversion unit 25 at the address of the corresponding position, and stores the pixel value for one frame. When the image is restored, the restored original image is output.

  The reference pixel acquisition unit 27 is used as a reference pixel in the image conversion apparatus in FIG. 1 when a reverse conversion target pixel is a conversion target pixel among pixels that have already been reversely converted and stored in the frame memory 26. Are obtained and supplied to the inverse conversion unit 25 as reference pixels. That is, in the present embodiment, the reference pixel acquisition unit 27 acquires, as a reference pixel, a pixel that is stored in the frame memory 26 and is adjacent to the left side of the pixel to be inversely converted among the pixels that have already been inversely converted. This is supplied to the conversion unit 25.

  In the present embodiment, the inverse conversion target pixel acquisition unit 24 acquires the conversion pixels constituting the conversion image stored in the frame memory 23 as the reverse conversion target pixels in the raster scan order. The pixel on the left side of the pixel has already been inversely transformed, restored to the original pixel, and stored in the frame memory 26.

  Next, with reference to the flowchart of FIG. 11, an inverse transform process performed by the image inverse transform device of FIG. 10 to inversely transform a converted image into an original image will be described.

  When the converted image output from the image conversion apparatus of FIG. 1 and the code table for each pixel value are supplied, in step S21, the frame memory 23 stores the converted image, and the code table acquisition unit 21 converts the converted image. The code table supplied together with the code table is acquired and supplied to the code table storage unit 22. The code table storage unit 22 stores the code table from the code table acquisition unit 21.

  Then, the process proceeds to step S22, and among the conversion pixels constituting the converted image stored in the frame memory 23 in the reverse conversion target pixel acquisition unit 24, those that have not yet been set as the reverse conversion target pixels in the raster scan order. , Read out as a reverse conversion target pixel, and supplied to the reverse conversion unit 25. Further, in step S22, the reference pixel acquisition unit 27 acquires, as the reference pixel, a pixel adjacent to the inverse conversion target pixel among the already inversely converted pixels stored in the frame memory 26, and the inverse conversion unit. 25.

  In step S23, the inverse conversion unit 25 refers to the code table stored in the code table storage unit 22 for each value that can be taken by the pixel value and corresponding to the pixel value of the reference pixel from the reference pixel acquisition unit 27. In accordance with the code table, the allocation code as the pixel value of the reverse conversion target pixel from the reverse conversion target pixel acquisition unit 24 is reversely converted to the original pixel value. In step S24, the inverse conversion unit 25 supplies the pixel value after the inverse conversion to the frame memory 26, stores it in the address of the corresponding position, and proceeds to step S25.

  In step S25, the inverse conversion target pixel acquisition unit 24 determines whether all the pixels constituting the converted image stored in the frame memory 23 have been subjected to inverse conversion of the pixel values as the inverse conversion target pixels. . If it is determined in step S25 that all of the pixels constituting the converted image have not been converted as pixel values for inverse conversion, the process returns to step S22, and the same processing is repeated thereafter. .

  Further, when it is determined in step S25 that all the pixels constituting the converted image have been subjected to inverse conversion of the pixel values as the inverse conversion target pixels, that is, the original image obtained by inversely converting the converted image in the frame memory 26. Is stored, the original image is read from the frame memory 26, and the process ends.

  Note that the inverse transformation process according to the flowchart of FIG. 11 is performed each time a converted image and a code table for each pixel value are supplied to the image inverse transformation apparatus.

  Here, the code table is supplied together with the converted image to the image reverse conversion device in FIG. 10, but the code table is created in advance in the image conversion device in FIG. 1 as described above. In addition, in the case where the code table storage unit 3 stores the same code table, the same code table may be stored in the code table storage unit 22 also in the image reverse conversion device of FIG. it can. In this case, the image reverse conversion apparatus of FIG. 10 can be configured without providing the code table acquisition unit 21.

  Next, as described above, the conversion image output from the image conversion apparatus in FIG. 1 has a distribution in which the allocation code, which is a pixel value constituting the conversion image, is steep and has a large deviation. By performing entropy coding such as, it is possible to compress at a high compression rate.

  FIG. 12 shows an example of the configuration of an embodiment of an image transmission system that compresses and transmits an image in such a manner.

  This image transmission system includes a compression device 31 that compresses image data and a decompression device 34 that decompresses the compressed image data to the original image data.

  The compression device 31 includes an image conversion device 41, an entropy encoding unit 42, and a multiplexer 43.

  The image conversion apparatus 41 is configured in the same manner as the image conversion apparatus shown in FIG. 1, converts the image data to be compressed into a converted image as described above, and outputs the converted image data to the entropy encoding unit 42 together with the code table. To do. The entropy encoding unit 42 compresses the converted image and the code table by performing entropy encoding processing such as Huffman encoding, and outputs the result to the multiplexer 43.

  Here, as described above, since the allocation code, which is the pixel value constituting the converted image, has a steep and large distribution, the entropy encoding unit 42 achieves high compression, as described above. The

  The multiplexer 43 multiplexes the compressed converted image from the entropy encoding unit 42 and the code table, and the data obtained as a result of the multiplexing is, for example, a terrestrial wave, a satellite line, or a CATV (Cable Television) network. Or transmitted via a transmission medium 32 such as the Internet or a public line, or recorded on a recording medium 33 such as a semiconductor memory, a magneto-optical disk, a magnetic disk, an optical disk, a magnetic tape, or a phase change disk. , Provided to the stretching device 34.

  The expansion device 34 includes a demultiplexer 44, an entropy decoding unit 45, and an image inverse conversion device 46.

  The demultiplexer 44 receives data provided via the transmission medium 32 or the recording medium 33, separates the data into an entropy-encoded converted image and a code table, and supplies the data to the entropy decoding unit 45. . The entropy decoding unit 45 decodes the output of the demultiplexer 44 into the original converted image and the code table, respectively, and supplies the decoded image to the image inverse conversion device 46.

  The image reverse conversion device 46 is configured in the same manner as the image reverse conversion device of FIG. 10, and as described above, the converted image supplied thereto is converted into the original image using the code table supplied thereto. Reverse transform and output.

  Next, as described above, the method of converting the image into a pixel whose pixel value is a steep and large distribution and performing the inverse conversion is proposed by the present applicant, for example, The present invention can be applied to the case of performing the embedded encoding / decoding disclosed in Japanese Patent Application No. 11-284199.

  FIG. 13 shows a configuration example of the image transmission system previously proposed by the applicant.

  The image transmission system includes an encoding device 110 and a decoding device 120. The encoding device 110 encodes, for example, an image as an encoding target, and outputs encoded data. The decoding device 120 The encoded data is decoded into the original image.

  That is, the image database 101 stores digital image data to be encoded. Then, the image stored in the image database 101 is read out and supplied to the embedded encoder 103.

  The additional information database 102 stores additional information (digital data) as information to be embedded in the image to be encoded. The additional information stored in the additional information database 102 is read out and supplied to the embedded encoder 103.

  The embedded encoder 103 receives an image from the image database 101 and additional information from the additional information database 102. Further, the embedded encoder 103 encodes the image according to the additional information from the additional information database 102 so that the image can be decoded by using the energy bias of the image from the image database 101. Output. That is, the embedded encoder 103 encodes the image by embedding additional information in the image so that decoding can be performed using the energy bias of the image, and outputs encoded data. The encoded data output by the embedded encoder 103 is recorded on a recording medium 104 made of, for example, a semiconductor memory, a magneto-optical disk, a magnetic disk, an optical disk, a magnetic tape, a phase change disk, or the like. The signal is transmitted via a transmission medium 105 including a wave, a satellite line, a CATV (Cable Television) network, the Internet, a public line, and the like, and provided to the decoding device 120.

  The decoding device 120 is configured by an embedded decoder 106, in which encoded data provided via the recording medium 104 or the transmission medium 105 is received. Further, the embedded decoder 106 decodes the encoded data into the original image and the additional information using the energy bias of the image. The decoded image is supplied to a monitor (not shown) and displayed, for example.

  As the additional information, for example, text data related to the original image, audio data, a reduced image obtained by reducing the image, and data unrelated to the original image can be used.

  Next, the principles of encoding (embedded encoding) in the embedded encoder 103 in FIG. 13 and decoding (embedded decoding) in the embedded decoder 106 will be described.

  In general, what is called information has a bias (universality) of energy (entropy), and this bias is recognized as information (worthy information). That is, for example, an image obtained by shooting a certain landscape is recognized by a person as being an image of such a landscape because the image (pixel value of each pixel constituting the image, etc.) This is because the image has a corresponding energy bias, and an image without the energy bias is merely noise or the like and is not useful as information.

  Therefore, even if some operation is performed on valuable information and the original energy bias of the information is destroyed, so to speak, some operation is performed by restoring the destroyed energy bias. Information can also be restored to the original information. That is, encoded data obtained by encoding information can be decoded into original valuable information by utilizing the original energy bias of the information.

  Here, examples of the energy (bias) of information include correlation, continuity, and similarity.

  Information correlation is the correlation (for example, autocorrelation between one component and another component) (for example, in the case of an image, pixels and lines constituting the image). Distance). For example, the correlation between the images includes the correlation between the lines of the image, and the correlation value indicating the correlation includes, for example, the sum of squares of the differences between the corresponding pixel values in the two lines. (In this case, a small correlation value represents a large correlation between lines, and a large correlation value represents a small correlation between lines).

  That is, for example, when there is an image having H lines as shown in FIG. 14, the correlation between the first line from the top (first line) and other lines is generally shown in FIG. As shown in (A), the closer the distance to the first line (the upper line in the image in FIG. 14) is, as shown as the correlation for the Mth line, the greater the distance from the first line. The line (the lower line in the image in FIG. 14) becomes smaller as shown as the correlation for the Nth line. Accordingly, there is a correlation bias in which the closer to the first line, the greater the correlation with the first line and the smaller the distance.

  Therefore, in the image of FIG. 14, an operation of exchanging the pixel values of the Mth line that is relatively close to the first line and the Nth line that is relatively far from the first line is performed (1 <M <N ≦ H). When the correlation between the first line and the other lines is calculated for the image after the replacement, it is as shown in FIG. 15B, for example.

  That is, in the image after replacement, the correlation with the Mth line (Nth line before replacement) close to the first line becomes small, and the correlation with the Nth line (Mth line before replacement) far from the first line. Becomes larger.

  Therefore, in FIG. 15B, the correlation bias that the correlation increases as the distance from the first line decreases and the correlation decreases as the distance from the first line decreases. However, with respect to an image, generally, the correlation bias that the correlation becomes larger as it is closer to the first line and the correlation becomes smaller as it is farther can be used to restore the broken correlation bias. That is, in FIG. 15B, the correlation with the Mth line near the first line is small and the correlation with the Nth line far from the first line is large because of the inherent correlation bias of the image. For example, it is clearly unnatural (funny), and the Mth line and the Nth line should be interchanged. Then, by replacing the M-th line and the N-th line in FIG. 15B, an image having an original correlation bias as shown in FIG. 15A, that is, an original image can be decoded. it can.

  Here, in the case described with reference to FIG. 14 and FIG. 15, the replacement of the lines will encode the image. In the encoding, the embedded encoder 103 determines, for example, which line is to be moved and which line is to be exchanged according to the additional information. On the other hand, in the embedded decoder 106, the encoded image, that is, the image in which the line is replaced, can be restored to the original image by replacing the line with the original position using the correlation. Will be decrypted. Further, in the decoding, for example, detecting which line has been moved or which lines have been exchanged in the embedded decoder 106 will decode the additional information embedded in the image.

  Next, regarding information continuity, for example, when attention is paid to one line of an image, a change pattern of pixel values as shown in FIG. Assuming that the waveform is observed, a change pattern of pixel values different from that of the target line is observed in other lines apart from the target line. Therefore, the pixel value change pattern is different between the attention line and another line away from the attention line, and there is also a bias in continuity. That is, when attention is paid to the pixel value change pattern of a certain part of the image, a similar pixel value change pattern exists in a part adjacent to the target part. There is a continuity bias that exists.

  Therefore, a part of the waveform in which the change pattern of pixel values in a certain line of the image shown in FIG. 16A is continuous is shown in FIG. 16B, for example. Replace with part of the waveform at.

  In this case, the continuity bias of the image is destroyed. However, by using the continuity bias that the change pattern of pixel values of adjacent parts is continuous and the change pattern of pixel values is different as the distance is increased, the destroyed continuity bias can be restored. it can. That is, in FIG. 16B, the change pattern of the pixel value of a part of the waveform is significantly different from the change pattern of the pixel value of the other part. Therefore, a portion that is clearly unnatural and is different from the change pattern of the pixel value of the other portion should be replaced with a waveform similar to the change pattern of the pixel value of the other portion. Then, by performing such replacement, the continuity bias is restored, so that the original waveform shown in FIG. 16A can be decoded from the waveform shown in FIG. .

  Here, in the case described with reference to FIG. 16, replacing a part of the waveform with a waveform of a change pattern of pixel values that is significantly different from the change pattern of the surrounding pixel values is to encode an image. Become. In the encoding, the embedded encoder 103 determines, for example, which part of the waveform the pixel value change pattern is to be replaced, how much the pixel value change pattern is to be changed, etc. according to the additional information. Will be decided. On the other hand, in the embedded decoder 106, the encoded signal, that is, a waveform having a change pattern of a pixel value that differs greatly in part, the change pattern of the surrounding pixel values is continuous, and the pixel value increases as the distance increases. Returning to the original waveform by using the continuity bias that the change patterns are different means decoding the original waveform. Further, at the time of decoding, the embedded decoder 106 detects, for example, which part of the waveform the pixel value change pattern has changed greatly and how much the pixel value change pattern has changed. To decode the embedded additional information.

  Next, regarding the similarity of information, for example, it is known that a part of an image or the like obtained by photographing a landscape can be generated using the fractal nature (self-similarity) of the image. That is, for example, in an image obtained by photographing the sea and forest as shown in FIG. 17A, the similarity between the change pattern of pixel values of the entire sea and the change pattern of pixel values of a part of the sea is Although it is high, there is a similarity bias that the similarity between these change patterns and the change pattern of pixel values of forests away from the sea is low. Here, the similarity of images may be considered not by comparing pixel value change patterns as described above but by comparing edge shapes.

  Therefore, a part of the sea shown in FIG. 17A is replaced with a part of the forest.

  In this case, the image similarity bias is destroyed, and an image as shown in FIG. 17B is obtained. However, the variation pattern of the pixel values of the adjacent parts has a high similarity, and the similarity deviation of the pixel value change pattern decreases as the distance from the pattern increases. Can be restored. That is, in FIG. 17B, a part of the sea image is a part of a forest image having low similarity to the sea, and a part of the forest image has low similarity to the forest. Being part of the ocean image is clearly unnatural, given the inherent similarity bias of the image. Specifically, in FIG. 17B, the similarity of a part of the forest image in the sea image is extremely low compared to the similarity of the other part of the sea. Also, the similarity of a part of the sea image in the forest image is extremely low compared to the similarity of the other part of the forest.

  Therefore, according to the bias of similarity that the image originally has, a part of the sea image that is a part of the sea image and a part of the sea image that is a part of the forest image Should be replaced. Then, by performing such replacement, the similarity similarity of the images is restored, thereby decoding the original image shown in FIG. 17A from the image shown in FIG. Can do.

  Here, in the case described with reference to FIG. 17, a part of the sea image and a part of the forest image are replaced with each other to encode the image. In the encoding, the embedded encoder 103 determines, for example, which part of the sea image (position on the screen) and which part of the forest image to replace according to the additional information. It will be. On the other hand, in the embedded decoder, an encoded signal, that is, an image in which a part of the sea is a part of the forest and a part of the forest is a part of the sea, The similarity of the change pattern of the pixel value is high, and using the similarity bias that the similarity of the change pattern of the pixel value decreases as the distance increases, the original image can be restored to the original image. It will be decrypted. Further, in the decoding, the embedded decoder 106 detects, for example, which part of the sea image has been replaced with which part of the forest image. It will be.

  As described above, the embedded encoder 103 encodes the image in accordance with the additional information so that decoding can be performed using the energy bias of the image to be encoded, and the encoded data , The embedded decoder 106 can decode the encoded data into the original image and the additional information without using overhead for decoding by utilizing the energy bias of the image. it can.

  In addition, since the additional information is embedded in the image to be encoded, the image obtained as a result of the embedding is different from the original image and is not an image that can be recognized as valuable information by a person. Therefore, it is possible to realize encryption without overhead for the image to be encoded.

  Furthermore, a completely reversible digital watermark can be realized. That is, in the conventional digital watermark, for example, the low-order bits of the pixel value that do not significantly affect the image quality are simply changed to values corresponding to the digital watermark. In this case, the low-order bits are changed to the original values. It is difficult to return. Therefore, the image quality of the decoded image deteriorates not a little due to the change of the lower bits as the digital watermark. On the other hand, when the encoded data is decoded using the energy bias of the original image, the original image and the additional information without deterioration can be obtained. Therefore, the image quality of the decoded image does not deteriorate due to the digital watermark.

  Further, since the embedded additional information can be extracted by decoding the image from the encoded data, the side information can be provided without any overhead along with the image encoding result. In other words, since the additional information can be embedded in the image without the overhead for extracting the additional information, the encoded data obtained as a result of the embedding is compressed (embedded compressed) by the amount of the additional information. be able to. Therefore, for example, if half of a certain image is to be encoded and the other half is additional information, the remaining half of the image can be embedded in the half of the image to be encoded. The image is simply compressed to ½.

  Furthermore, since the encoded data is decoded using a so-called statistic, that is, an energy bias of the original image, it is highly resistant to errors. That is, robust encoding (statistical encoding) that is highly robust encoding can be realized.

  In addition, since the encoded data is decoded using the energy bias of the original image, the more characteristic the energy bias is, that is, for example, the higher the activity of the image, or The lower the redundancy, the more additional information can be embedded. Here, as described above, the encoded data obtained as a result of embedding the additional information can be said to be compressed by the amount of the additional information. From the viewpoint of this compression, the information to be encoded is According to the method of encoding the information according to the additional information (embedded encoding method) so that decoding can be performed using the bias of energy possessed, the higher the activity of the image, or the The lower the redundancy, the higher the compression ratio. In this regard, the embedded coding method is significantly different from the conventional coding method (in the conventional coding method, for example, MPEG (Moving Picture Experts Group) method, etc., basically, the higher the activity of the image, or The lower the image redundancy, the lower the compression rate).

Furthermore, for example, as described above, an image is to be encoded, and as additional information, for example, audio of a medium different from the image is used, and the image is provided using the audio as a key. It becomes possible. That is, on the encoding device 110 side, for example, the voice “open sesame” uttered by the contractor is embedded in the image as additional information, and on the decoding device 120 side, the user utters the voice “open sesame”. Then, speaker recognition is performed using the voice and the voice embedded in the image. In this way, for example, as a result of speaker recognition, it is possible to automatically present an image only when the user is a contractor.
In this case, the sound as the additional information can use the sound waveform itself, not so-called feature parameters.

  In addition, for example, while audio is an encoding target, audio is provided using an image as a key by using, for example, an image of a medium different from audio as additional information (for example, Voice response after face recognition). That is, on the encoding device 110 side, for example, an image of the user's face is embedded in the voice as a response to the user, and on the decoding device 120 side, the user's face is photographed and matched with the resulting image. By outputting the voice in which the face image is embedded, it is possible to realize a voice response system that performs a different voice response for each user.

  Furthermore, it is also possible to embed information of the same medium in information of a certain medium, such as embedding sound in an audio or embedding an image in an image. Alternatively, if the voice and face image of the contractor are embedded in the image, the image is presented only when the voice and face image of the user match those embedded in the image. In other words, a double key system can be realized.

  In addition, for example, it is possible to embed the other in one of the synchronized images and sounds constituting the television broadcast signal. In this case, the information of different media is integrated. Integrated coding can be realized.

  Further, in the embedded coding system, as described above, the information can be embedded with more additional information, as the information has a characteristic of energy bias. Therefore, for example, it is possible to control the total amount of data by adaptively selecting one of the two types of information that has a characteristic in energy bias and embedding the other in the selected one. It becomes. That is, between two pieces of information, one piece of information can absorb the other amount of information. As a result of controlling the total amount of data in this way, information transmission (environmentally responsive network transmission) with a data amount suitable for the transmission band and usage status of the transmission path and other transmission environments becomes possible.

  Further, for example, by embedding a reduced image of an image in an image (or by embedding a sound obtained by thinning out the sound), so-called hierarchical encoding (lower hierarchy) is performed without increasing the amount of data. (Encoding for generating higher layer information) can be realized.

  Furthermore, for example, by embedding an image as a key for searching for an image in the image, a database for searching for an image based on the image as the key can be realized.

  Next, FIG. 18 shows a configuration example of the embedded encoder 103 in FIG. 13 in the case of performing embedded encoding in which additional information is embedded in an image so that it can be restored using the continuity of the image. ing.

  An image supplied from the image database 101 is supplied to a frame memory 131. The frame memory 131 temporarily stores an image from the image database 101, for example, in units of frames.

  A CPU (Central Processing Unit) 132 executes a program stored in the program memory 133 to perform an embedded encoding process to be described later. That is, the CPU 132 receives additional information supplied from the additional information database 102 via the memory 134 in accordance with the program stored in the program memory 133, and converts the additional information into an image stored in the frame memory 131. It is made to embed. Specifically, the CPU 132 embeds the additional information in the pixel value by changing the pixel value constituting the image stored in the frame memory 131 according to the additional information stored in the memory 134. ing. The image in which the additional information is embedded is output as encoded data.

  The program memory 133 is composed of, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), and the like, and stores a computer program and necessary data for causing the CPU 132 to perform embedded encoding processing.

  The memory 134 reads out the additional information stored in the additional information database 102 and temporarily stores it. The output buffer 135 temporarily stores encoded data obtained by performing the embedded encoding processing by the CPU 132 and outputs the encoded data.

  The frame memory 131 is configured by a plurality of banks so that a plurality of frames can be stored. By switching the banks, the frame memory 131 stores images supplied from the image database 101, In addition, the image that is the target of the embedded encoding process by the CPU 132 can be simultaneously stored. As a result, even if the image supplied from the image database 101 is a moving image, the encoded data can be output in real time.

  Next, FIG. 19 shows a functional configuration example of the embedded encoder 103 of FIG. The functional configuration shown in FIG. 19 is realized by the CPU 132 executing a computer program stored in the program memory 133.

  Each of the frame memory 131, the memory 134, and the output buffer 135 is the same as that described in FIG.

  The frequency distribution calculation unit 141 obtains the frequency distribution (histogram) of the pixel values of the encoding target image stored in the frame memory 131, for example, in units of one frame, and supplies it to the change target pixel value / change value determination unit 142. It is made to do. Based on the frequency distribution of the pixel values supplied from the frequency distribution calculation unit 141, the change target pixel value / change value determination unit 142 selects, from among the pixel values constituting the encoding target image stored in the frame memory 131, A pixel value to be changed according to the additional information (hereinafter referred to as a change target pixel value as appropriate) is determined, and a value to be changed (hereinafter referred to as a change value as appropriate) is determined, and a pixel value change unit 143. The pixel value changing unit 143 reads the pixel values constituting the encoding target frame stored in the frame memory 131 in a predetermined order, for example, in the raster scan order, and the read pixel value is the change target pixel. If the values do not coincide with each other, they are supplied as they are to the output buffer 135 and stored in the corresponding addresses. Further, when the read pixel value matches the change target pixel value, the pixel value changing unit 143 reads the additional information stored in the memory 134 and changes the change target pixel value according to the additional information. Or is left as it is and supplied to the output buffer 135 to be stored at the corresponding address.

  Next, the embedded encoding process performed in the embedded encoder 103 in FIG. 19 will be described with reference to the flowchart in FIG.

  The image stored in the image database 101 is read out and supplied to the frame memory 131 and stored in sequence. Further, the additional information stored in the additional information database 102 is read out and sequentially supplied and stored in the memory 134.

  On the other hand, in the frequency distribution calculation unit 141, in step S101, pixel values constituting one frame to be encoded stored in the frame memory 131 are read, and the frequency distribution is calculated.

  Here, FIG. 21 shows a frequency distribution of pixel values of an actual image. FIG. 21 shows the frequency distribution of the R component of an image of one frame composed of RGB component signals each assigned 8 bits. Here, as described above, 8 bits are assigned to the R component, and therefore, pixel values in a range of 256 such as 0 to 255 can be taken. In FIG. 21, pixels having pixel values of 250 or more are used. Does not exist. Furthermore, as is apparent from FIG. 21, in general, there are not only the same number of pixel values (here, values of 0 to 255) but also a large number of pixel values in the image. For example, some pixel values do not exist at all. However, changes in the frequency distribution of pixel values are generally continuous.

  The frequency distribution of pixel values obtained by the frequency distribution calculation unit 141 is supplied to the change target pixel value / change value determination unit 142. When the change target pixel value / change value determination unit 142 receives the frequency distribution of the pixel values, the process proceeds to step S102. Based on the frequency distribution, the change target pixel value / change value determination unit 142 determines the pixel values constituting the encoding target image stored in the frame memory 131. A change target pixel value is determined from among them, and a change value that is a value for changing the change target pixel value is determined. That is, the change target pixel value / change value determination unit 142 detects, for example, the pixel value with the highest frequency from the frequency distribution and determines it as the change target pixel value. Furthermore, the change target pixel value / change value determination unit 142 detects a pixel value having a frequency of 0 from the frequency distribution and determines it as a change value (the pixel value having the frequency of 0 is If there are a plurality, for example, one of them is selected and determined as a change value).

  Here, in the present embodiment, additional information is embedded in the change target pixel value. Therefore, in order to increase the data amount of the additional information that can be embedded, the pixel value having the highest frequency is set as the change target pixel value. However, the pixel value to be changed is not limited to the highest pixel value.

  When the change target pixel value and the change value are determined in the change target pixel value / change value determination unit 142, the change target pixel value and the change value are supplied to the pixel value change unit 143.

  When the pixel value changing unit 143 receives the change target pixel value and the change value, in step S103, the pixel value changing unit 143 reads out the upper left pixel value constituting the encoding target frame stored in the frame memory 131, and proceeds to step S104. In step S104, the pixel value changing unit 143 determines whether or not the pixel value read in step S104 is a change target pixel value. If it is determined that the pixel value is not a change target pixel value, steps S105 to S107 are skipped. In step S108, the pixel value is supplied to the output buffer 135 as it is and written to the corresponding address.

If it is determined in step S104 that the pixel value read in step S104 is a change target pixel value, the process proceeds to step S105, where the pixel value changing unit 143 reads additional information from the memory 134, and step S106. Proceed to
Here, in step S105, it is assumed that the additional information is read in units of 1 bit, for example.

  In step S106, the pixel value changing unit 143 determines whether the 1-bit additional information read in step S105 is 0 or 1. If it is determined in step S106 that the additional information is 0 or 1, for example, 0, step S107 is skipped and the process proceeds to step S108. In the pixel value changing unit 34, the pixel value to be changed is Then, it is supplied to the output buffer 135 as it is and written to the corresponding address. That is, the additional information that is 0 is embedded in the pixel having the change target pixel value without changing the change target pixel value.

  If it is determined in step S106 that the additional information is, for example, one of 0 or 1, the process proceeds to step S107, and the pixel value changing unit 143 changes the change target pixel value to the change value. The process proceeds to step S108. In step S108, the pixel value changing unit 143 supplies the changed value changed in step S107 to the output buffer 135 and writes it to the corresponding address. That is, the additional information that is 1 is embedded in the pixel having the change target pixel value by changing the change target pixel value to the change value.

  After the process of step S108, the process proceeds to step S109, and it is determined whether or not all the pixels in the frame that is the current encoding target have been read. If it is determined in step S109 that reading of all the pixels of the frame that is the current encoding target has not been completed yet, the process returns to step S103, and the pixel values to be processed next are read in the raster scan order. Thereafter, the same processing is repeated.

  If it is determined in step S109 that reading of all the pixels of the frame that is the current encoding target has been completed, the process proceeds to step S110, and the frame in which the additional information stored in the output buffer 135 is embedded. Are read out as encoded data and output. Then, the process proceeds to step S111, where it is determined whether or not the frame to be processed next is stored in the frame memory 131. If it is determined that the frame is stored, the process returns to step S101 to set the frame as an encoding target. The same process is repeated.

  If it is determined in step S111 that the frame to be processed next is not stored in the frame memory 131, the embedded encoding process is terminated.

  According to the embedded encoding process as described above, an image of a certain frame is encoded into the following encoded data.

  That is, for example, assuming that the change target pixel values are distributed in the encoding target frame as indicated by · in FIG. 22A, among the change target pixel values, 0 in the raster scan order. Those at the position corresponding to the additional information are left as they are, and those at the position corresponding to one additional information are present in the frame of FIG. 22 (A) as indicated by x in FIG. 22 (B). The pixel value that is not changed, that is, the changed value that is a pixel value with a frequency of 0 is changed.

  As described above, the change target pixel value, which is the highest pixel value among the pixel values constituting the image stored in the frame memory 131, is a pixel value that does not exist in the image according to the additional information. When embedding additional information by changing to a value, the change value is converted into the original change target pixel value by utilizing the bias of image continuity (continuity of the frequency distribution of pixel values constituting the image). By changing to, the original image can be decoded and the additional information can be decoded. Therefore, it is possible to embed additional information in an image while minimizing degradation of the image quality of the image and without increasing the amount of data.

  That is, the change value is decoded into the original pixel value (change target pixel value) without overhead by using the continuity of the image, that is, the continuity of the frequency distribution of the pixel values constituting the image. The additional information can be decoded by detecting the change value and the change target pixel value. Accordingly, in the decoded image (reproduced image) obtained as a result, image quality deterioration due to embedding additional information basically does not occur.

  If there is no pixel value that does not exist in the encoding target frame, changing the change target pixel value distinguishes the pixel value after the change from the pixel value that originally exists in the encoding target frame. Difficult to do. Therefore, after calculating the frequency distribution in step S101 of the embedded encoding process in FIG. 20, if it is determined from the frequency distribution that there is no pixel value that does not exist in the frame, additional information is added to the frame. It is desirable to perform the embedded encoding process with the next frame as an encoding target without being embedded.

  Next, FIG. 23 shows a configuration example of the embedded decoder 106 in FIG. 13 that decodes the encoded data output from the embedded encoder 103 in FIG. 19 into the original image and additional information using the continuity of the image. Is shown.

  The encoded data, that is, an image in which additional information is embedded (hereinafter also referred to as an embedded image as appropriate) is supplied to the input buffer 151. The input buffer 151 converts the embedded image into, for example, a frame. Temporary storage is made in units. The input buffer 151 is also configured in the same manner as the frame memory 131 in FIG. 18, and by performing bank switching, real-time processing is possible even if the embedded image is a moving image.

  The CPU 152 executes an embedded decoding process to be described later by executing a program stored in the program memory 153. That is, the CPU 152 is configured to decode the embedded image stored in the input buffer 151 into the original image and the additional information using the continuity of the image. Specifically, the CPU 152 decodes the additional information by detecting the change target pixel value and the change value from the pixel values of the decoding target frame stored in the input buffer 151, and further, the change value is obtained. By changing to the change target pixel value, the original image is decoded.

  The program memory 153 is configured similarly to the program memory 133 of FIG. 18, for example, and stores a computer program and necessary data for causing the CPU 152 to perform embedded decoding processing.

  The frame memory 154 stores and outputs the image decoded by the CPU 152, for example, in units of one frame. The memory 155 temporarily stores the additional information decoded by the CPU 152 and outputs it.

  Next, FIG. 24 shows a functional configuration example of the embedded decoder 106 of FIG. The functional configuration shown in FIG. 24 is realized by the CPU 152 executing a computer program stored in the program memory 153.

  Each of the input buffer 151, the frame memory 154, and the memory 155 is the same as that described in FIG.

  The frequency distribution calculation unit 161 reads the pixel values constituting the frame of the decoding target embedded image stored in the input buffer 151, obtains the frequency distribution, and supplies the frequency distribution to the change target pixel value / change value determination unit 162. Has been made. The change target pixel value / change value determination unit 162 changes the change target pixel value and the change target pixel value in the embedded encoder 103 from the pixel values constituting the embedded image based on the frequency distribution of the pixel values from the frequency distribution calculation unit 161. A pixel value determined for each value is obtained (determined) and supplied to the pixel value changing unit 163. The pixel value changing unit 163 recognizes the change target pixel value and the change value from the output of the change target pixel value / change value determining unit 162, and changes the frame in the decoding target embedded image stored in the input buffer 151. By detecting the target pixel value and the change value, the additional information embedded in the embedded image is decoded and supplied to the memory 155. Further, the pixel value changing unit 163 changes the change value in the embedded image to the change target pixel value, thereby decoding the embedded image into the original image and supplying the decoded image to the frame memory 154. .

  Next, the embedded decoding process performed in the embedded decoder 106 in FIG. 24 will be described with reference to the flowchart in FIG.

  In the input buffer 151, embedded images (encoded data) supplied thereto are sequentially stored, for example, in units of one frame.

  On the other hand, in step S121, the frequency distribution calculation unit 161 reads the frame of the embedded image to be decoded stored in the input buffer 151, and obtains the frequency distribution of the pixel values constituting the frame. This frequency distribution is supplied to the change target pixel value / change value determination unit 162.

  When the change target pixel value / change value determination unit 162 receives the frequency distribution from the frequency distribution calculation unit 161, the change target pixel value and the change determined by the embedded encoder 103 based on the frequency distribution in step S122. Find the value.

That is, if the pixel values constituting the original image encoded in the embedded image have a continuous frequency distribution as shown in FIG. 26A, for example, the embedded encoder 103 The highest pixel value P ₁ is determined as the change target pixel value. In FIG. 26, the range of pixel values that can be taken is 0 to MAX, and the original image has no pixels that are equal to or less than the pixel value P _min and pixels that are equal to or greater than the pixel value P _max. (0 <P _min <P _max <MAX).

Further, in this case, if the embedded encoder 103 determines a pixel value P ₂ that is equal to or less than the pixel value P _min and does not exist in the original image as a change value and performs embedded encoding, the result is obtained. The frequency distribution of the pixel values of the embedded image is, for example, as shown in FIG. That is, the frequency of changing the target pixel value P ₁ is close to it the pixel values (e.g., pixel values on both sides) becomes extremely low compared to the frequency of, the frequency of the change value P _2, on the contrary, it is close pixel values It becomes extremely high compared to the frequency of.

As a result, for example, the difference between the frequency of the pixel value n and the frequency of the pixel value n + 1 (= the frequency of the pixel value n−the frequency of the pixel value n + 1) (hereinafter, appropriately referred to as a frequency difference) from the smallest pixel value. As a result, the frequency difference becomes extremely large as shown in FIG. 27A near the change target pixel value P ₁ (when n = P ₁ and n + 1 = P ₁ ). After that, it becomes extremely small (discontinuous). Further, the frequency difference becomes extremely large after being extremely small as shown in FIG. 27B in the vicinity of the change value P ₂ (when n = P ₂ and n + 1 = P ₂ ). Become (becomes discontinuous).

  Therefore, the change target pixel value and the change value determined in the embedded encoder 103 can be obtained by searching the frequency difference, and the change target pixel value / change value determination unit 162 does so. Based on the frequency distribution of the pixel values of the embedded image, the change target pixel value and the change value determined by the embedded encoder 103 are obtained.

  Returning to FIG. 25, when the change target pixel value / change value determination unit 162 obtains the change target pixel value and the change value determined by the embedded encoder 103, the change target pixel value / change value determination unit 162 outputs them to the pixel value change unit 163. The process proceeds to S123.

  In step S123, the pixel value changing unit 163 reads the upper left pixel value constituting the frame of the embedded image to be decoded stored in the input buffer 151, and proceeds to step S124 to determine the pixel value. . If it is determined in step S124 that the read pixel value is neither the change target pixel value nor the change value, the pixel value changing unit 163 proceeds to step S128 and stores the pixel value in the frame memory 154. And store it at the corresponding address. Here, since the additional information is not embedded in the pixel that is neither the change target pixel value nor the change value, the decoding is not performed (it is not necessary).

  When it is determined in step S124 that the read pixel value is the change target pixel value, the process proceeds to step S125, and the pixel value changing unit 163 stores 0 in the memory 155 as a decoding result of the additional information. Alternatively, 0 corresponding to the change target pixel value is supplied and stored. In step S128, the pixel value changing unit 163 supplies the change target pixel value to the frame memory 154 as it is and stores it in the corresponding address.

  Furthermore, when it is determined in step S124 that the read pixel value is a change value, the process proceeds to step S126, and the pixel value changing unit 163 changes the change value to the change target pixel value, thereby The original pixel value is decoded, and the process proceeds to step S127. In step S127, in the pixel value changing unit 163, 1 corresponding to the changed value of 0 or 1 is supplied and written to the memory 155 as the decoding result of the additional information. In step S128, the pixel value changing unit 163 supplies the change target pixel value (in which the change value is changed in step S126) to the frame memory 154 and stores it in the corresponding address.

  After the process of step S128, the process proceeds to step S129, and it is determined whether or not all the pixels in the frame of the embedded image that is the decoding target have been read. If it is determined in step S129 that reading of all the pixels of the frame of the embedded image that is to be decoded has not been completed yet, the process returns to step S123, and the pixel value to be processed next is determined in raster scan order. Thereafter, the same processing is repeated.

  If it is determined in step S129 that reading of all the pixels of the embedded image that is the decoding target has been completed, one frame of decoded image or additional information stored in the frame memory 154 or the memory 155 is read. Is output. In step S130, it is determined whether the frame of the embedded image to be processed next is stored in the input buffer 151. If it is determined that the frame is stored, the process returns to step S121 to decode the frame. Similar processing is repeated as a target.

  If it is determined in step S130 that the frame of the embedded image to be processed next is not stored in the input buffer 151, the embedded decoding process ends.

  As described above, encoded data, which is an image in which additional information is embedded, is decoded into the original image and additional information using the continuity of the image, so there is no overhead for decoding. However, the encoded data can be decoded into the original image and additional information. Accordingly, the decoded image basically does not deteriorate in image quality due to the embedded additional information.

  In the present embodiment, in the embedded encoding process, the change target pixel value is left as it is (changed to the change target pixel value) or changed to the change value according to the additional information. When there are two or more pixel values that do not exist in the image to be encoded, for example, two of them are set as a first change value and a second change value, and the change target pixel value is set according to the additional information, It is possible to change to the first change value or to change to the second change value. Here, when the change target pixel value is left as it is or is changed to the change value, it is sufficient if there is one pixel value that does not exist in the encoding target image. In the case of changing to the first or second change value, it is necessary that the encoding target image has two or more pixel values that do not exist. However, when the change target pixel value is changed to the first or second change value, since the change target pixel value does not exist in the embedded image, the embedded decoder 106 determines from the frequency distribution of the embedded image. Thus, it is possible to obtain the change target pixel value with higher accuracy.

  Further, when there are a plurality of non-existing pixel values in the encoding target image, all the plurality of pixel values are changed values, and the change target pixel value is selected from any of the plurality of changed values according to additional information. It is possible to change it. In this case, additional information of 2 bits or more can be embedded in one pixel.

  In this embodiment, in the embedded encoding process, the pixel values of the image to be encoded are processed in the raster scan order. However, the order of the process is not limited to the raster scan order. That is, as long as the pixel value for obtaining the frequency distribution can be read out, the processing order may be the column direction or the oblique direction.

  Furthermore, in the present embodiment, the frequency distribution is obtained for each frame in the embedded encoding process and the embedded decoding process, but the frequency distribution is divided into several blocks, for example, by dividing one frame into several blocks. It may be obtained in units of blocks or may be obtained in units of a plurality of frames.

  In this embodiment, the embedded decoder 106 obtains the change target pixel value and the change value determined by the embedded encoder 103. However, the change target pixel value and the change value have a slight amount. It is data and may be included in the embedded image as overhead.

  Furthermore, when the image to be embedded is, for example, a color image composed of RGB component signals, it is possible to perform embedded encoding on each of RGB.

  The information used as the additional information is not particularly limited, and for example, an image, sound, text, computer program, control signal, or other data can be used as the additional information. If a part of the image in the image database 101 is set as additional information and the rest is a supply target to the frame memory 131, a part of the image set as the additional information is embedded in the remaining part. Compression will be realized.

  Furthermore, in the present embodiment, the additional information is embedded in the image. However, the additional information can be embedded in other media, such as audio, in which the shape of the frequency distribution of values is continuous. is there.

  Here, the additional information itself is embedded in the image. However, for example, the feature amount of the additional information (for example, a histogram of additional information, variance, dynamic range, etc.) is embedded in the image. It is also possible to do.

  Next, FIG. 28 shows a functional configuration example when the embedded encoder 103 of FIG. 13 is configured using the image conversion apparatus of FIG. In the figure, portions corresponding to those in FIG. 19 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, the embedded encoder 103 in FIG. 28 has the same configuration as that in FIG. 19 except that an image conversion device 171 is newly provided.

  The image conversion device 171 is configured in the same manner as the image conversion device in FIG. 1, converts the image to be encoded stored in the frame memory 131 into a converted image as described above, and stores it in the frame memory 131. It comes to memorize.

  Here, in order to simplify the description, it is assumed that the image conversion apparatus 171 stores a code table for each possible value that is created in advance as a pixel value.

  Therefore, in the embodiment of FIG. 28, the frequency distribution calculation unit 141, the change target pixel value / change value determination unit 142, and the pixel value change unit 143 perform the above-described embedded encoding process on the converted image. Done.

  Next, the embedded encoding process in the embedded encoder 103 of FIG. 28 will be described with reference to the flowchart of FIG.

  In the embedded encoder 103 of FIG. 28, first, in step S140, the image conversion apparatus 171 converts the encoding target image stored in the frame memory 131 into a converted image and stores the converted image in the frame memory 131. .

  Then, in steps S141 to S151, the same processing as in steps S101 to S111 in FIG. 20 is performed, whereby the same embedded encoding processing as that described in FIG. 20 is performed on the converted image. Additional information is embedded.

  As described above, it is possible to perform more efficient embedded coding by embedding additional information in the converted image instead of the image itself.

  That is, as described above, the distribution of the pixel values of the converted pixels constituting the converted image obtained by the image conversion device 171 has a steep and large deviation. Therefore, when embedding additional information in the converted image, the number of conversion pixels (the pixel value) having the highest frequency as the pixel value to be changed is compared with that when embedding additional information in the pixel value itself of the original image. As a result, a large number of additional information can be embedded in such a large number of converted pixels.

  Next, FIG. 30 illustrates a functional configuration example of the embedded decoder 106 in FIG. 13 when the embedded encoder 103 is configured as illustrated in FIG. In the figure, portions corresponding to those in FIG. 24 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, the embedded decoder 106 in FIG. 30 is configured in the same manner as in FIG. 24 except that an image inverse conversion device 181 is newly provided.

  The image reverse conversion device 181 is configured in the same manner as the image reverse conversion device in FIG. 10, and restores the converted image stored in the frame memory 154 by inversely converting it into the original image, and writes it into the frame memory 154. It has become.

  Next, the embedded decoding process by the embedded decoder 106 of FIG. 30 will be described with reference to the flowchart of FIG.

  In the embedded decoder 106 in FIG. 30, the same processing as in steps S121 through S130 in FIG. 25 is performed on encoded data obtained by embedding additional information in the converted image as described above in steps S161 through S170. To be done. However, while proceeding from step S169 to step S170, in step S171, the image reverse conversion device 181 reversely converts the pixel value of each conversion pixel constituting the converted image stored in the frame memory 154 to the original pixel value. Write to the frame memory 154.

  That is, the encoded data output from the embedded encoder 103 in FIG. 28 is obtained by embedding additional information in the converted image as described above. By performing the processing of steps S161 to S168 similar to steps S121 to S128, the converted image is written in the frame memory 154. In step S 171, the converted image is inversely converted in the image inverse conversion device 181 as described above, and the original image obtained as a result is written in the frame memory 154.

  According to the embedded decoding process as described above, as described above, an image in which a large amount of additional information is embedded can be decoded into the original image and the additional information.

  In the present embodiment, the CPU 132 or 152 is caused to execute a computer program to perform the embedded encoding process or the embedded decoding process. However, these processes are performed by dedicated hardware. Is also possible.

  Next, the above-described series of processing can be performed by installing a program in a general-purpose computer or the like.

  Therefore, FIG. 32 shows a configuration example of an embodiment of a general-purpose computer in which a program for executing the series of processes described above is installed.

  The program can be recorded in advance in a hard disk 205 or ROM 203 as a recording medium built in the computer.

  Alternatively, the program is stored in a removable recording medium 211 such as a floppy (registered trademark) disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto optical) disk, a DVD (Digital Versatile Disc), a magnetic disk, or a semiconductor memory. It can be stored (recorded) temporarily or permanently. Such a removable recording medium 211 can be provided as so-called package software.

  The program is installed on the computer from the removable recording medium 211 as described above, or transferred from the download site to the computer wirelessly via a digital satellite broadcasting artificial satellite, LAN (Local Area Network), The program can be transferred to a computer via a network such as the Internet. The computer can receive the program transferred in this way by the communication unit 208 and install it in the built-in hard disk 205.

  The computer includes a CPU (Central Processing Unit) 202. An input / output interface 210 is connected to the CPU 202 via the bus 201, and the CPU 202 operates the input unit 207 including a keyboard, a mouse, a microphone, and the like by the user via the input / output interface 210. When a command is input as a result of this, the program stored in a ROM (Read Only Memory) 203 is executed accordingly. Alternatively, the CPU 202 also receives a program stored in the hard disk 205, a program transferred from a satellite or a network, received by the communication unit 208 and installed in the hard disk 205, or a removable recording medium 211 attached to the drive 209. The program read and installed in the hard disk 205 is loaded into a RAM (Random Access Memory) 204 and executed. Thereby, the CPU 202 performs processing according to the above-described flowchart or processing performed by the configuration of the above-described block diagram. Then, the CPU 202 outputs the processing result from the output unit 206 configured with an LCD (Liquid Crystal Display), a speaker, or the like, for example, via the input / output interface 210, or from the communication unit 208 as necessary. Transmission, and further recording on the hard disk 205 is performed.

  Here, in the present specification, the processing steps for describing a program for causing the computer to perform various processes do not necessarily have to be processed in time series in the order described in the flowcharts, but in parallel or individually. This includes processing to be executed (for example, parallel processing or processing by an object).

  Further, the program may be processed by one computer or may be distributedly processed by a plurality of computers. Furthermore, the program may be transferred to a remote computer and executed.

  In this embodiment, the pixel adjacent to the left of a certain pixel is used as the reference pixel. However, for example, the pixel adjacent to the right, the pixel adjacent above or below it, the pixel that is not adjacent but is close to the pixel, etc. Can be used as a reference pixel. Furthermore, for example, a pixel at the same position in the previous frame or a pixel at the same position in the previous frame after motion compensation can be used as the reference pixel. It is also possible to obtain an average value of the pixel values of several pixels at close positions, and to assume that a pixel having the average value exists virtually, and to use the virtual pixel as a reference pixel. .

  Furthermore, in the present embodiment, the image data is targeted for conversion. However, for example, audio data, computer programs, and the like can be targeted for conversion. However, the present invention is particularly useful when conversion is performed on data that has some correlation between data such as image data and audio data.

It is a block diagram which shows the structural example of one Embodiment of the image conversion apparatus to which this invention is applied. It is a block diagram which shows the structural example of the code table preparation part 2 of FIG. It is a figure for demonstrating the process of the pixel selection part 12 of FIG. It is a figure for demonstrating the process of the histogram preparation part 13 of FIG. It is a figure which shows the histogram of the pixel value obtained from the actual image. It is a figure for demonstrating the process of the code generation part 14 of FIG. It is a figure for demonstrating the process of the code generation part 14 of FIG. 3 is a flowchart for explaining conversion processing by the image conversion apparatus in FIG. 1. It is a flowchart for demonstrating the detail of the process of FIG.8 S1. It is a block diagram which shows the structural example of one Embodiment of the image reverse transformation apparatus to which this invention is applied. It is a flowchart for demonstrating the reverse transformation process by the image reverse transformation apparatus of FIG. It is a block diagram which shows the structural example of one Embodiment of the image transmission system to which this invention is applied. It is a block diagram which shows the structural example of the image transmission system which the present applicant previously proposed. It is a figure which shows the image of encoding object. It is a figure for demonstrating the encoding / decoding using a correlation. It is a figure for demonstrating the encoding / decoding using a continuity. It is a figure for demonstrating the encoding / decoding using similarity. It is a block diagram which shows the structural example of the hardware of the embedded encoder 103 of FIG. FIG. 19 is a block diagram illustrating a functional configuration example of an embedded encoder 103 in FIG. 18. It is a flowchart for demonstrating an embedded encoding process. It is a figure which shows the frequency distribution of the pixel value of an actual image. It is a figure for demonstrating the result of an embedded encoding process. It is a block diagram which shows the structural example of the hardware of the embedded decoder 106 of FIG. FIG. 24 is a block diagram illustrating a functional configuration example of an embedded decoder 106 in FIG. 23. It is a flowchart for demonstrating an embedded decoding process. It is a figure for demonstrating the process of step S122 of FIG. It is a figure for demonstrating the process of step S122 of FIG. It is a block diagram which shows the structural example of one Embodiment of the embedded encoder 103 of FIG. 13 to which this invention is applied. It is a flowchart for demonstrating the embedded encoding process which the embedded encoder 103 of FIG. 28 performs. It is a block diagram which shows the structural example of one Embodiment of the embedded decoder 106 of FIG. 13 to which this invention is applied. It is a block diagram for demonstrating the embedding decoding process which the embedding decoder 106 of FIG. 30 performs. It is a block diagram which shows the structural example of one Embodiment of the computer to which this invention is applied.

Explanation of symbols

  1 frame memory, 2 code table creation unit, 3 code table storage unit, 4 conversion target pixel acquisition unit, 5 reference pixel acquisition unit, 6 conversion unit, 7 frame memory, 11 frame memory, 12 pixel selection unit, 13 histogram creation unit , 14 code generation unit, 21 code table acquisition unit, 22 code table storage unit, 23 frame memory, 24 inverse conversion target pixel acquisition unit, 25 inverse conversion unit, 26 frame memory, 27 reference pixel acquisition unit, 31 compression device, 32 Transmission medium, 33 recording medium, 34 decompression device, 41 image conversion device, 42 entropy encoding unit, 43 multiplexer, 44 demultiplexer, 45 entropy decoding unit, 46 image inverse conversion device, 101 image database, 102 additional information Database, 103 embedded encoder, 104 recording medium, 105 transmission medium, 106 embedded decoder, 110 encoding device, 120 decoding device, 131 frame memory, 132 CPU, 133 program memory, 134 memory, 135 output buffer, 141 frequency Distribution calculation unit, 142 Change target pixel value / change value determination unit, 143 Pixel value change unit, 151 Input buffer, 152 CPU, 153 Program memory, 154 Frame memory, 155 memory, 161 Frequency distribution calculation unit, 162 Change target pixel value / Change value determination unit, 163 pixel value change unit, 171 image conversion device, 181 image reverse conversion device, 201 bus, 202 CPU, 203 ROM, 204 RAM, 205 hard disk, 206 output , 207 input unit, 208 communication unit, 209 drive, 210 input-output interface, 211 removable recording medium

Claims (21)

A data processing device that encodes second data obtained by converting first data according to third data,
Selecting means for selecting the first data and selecting the other first data having a predetermined positional relationship with the selected first data as reference data;
Conversion means for converting the first data into the second data based on a conversion table generated for each value of the reference data and associated with the first and second data;
Frequency distribution calculating means for determining a frequency distribution of at least a part of the second data;
A determining means for determining change target data to be changed according to the third data from the partial data according to the frequency distribution, and for determining change data that is a value for changing the change target data When,
Coding means for embedding data related to the third data in the second data by changing the change target data to the change data according to the third data ;
The conversion table uses a predetermined value of the first data as the reference data, and a frequency distribution of the first data in a predetermined positional relationship with respect to the reference data. Generated for each value of the reference data, and based on the frequency distribution of the first data for each value of the reference data, the first value of each value for which the frequency distribution is obtained for the reference data A data processing device generated for each value of the reference data by assigning the second data to the data in ascending or descending order of frequency . The data processing apparatus according to claim 1, wherein the determination unit determines data having the highest frequency as the data to be changed from the partial data. The data processing apparatus according to claim 1, wherein the determination unit determines, as the change data, a value having a frequency of 0 among the values that the second data can take. The data processing apparatus according to claim 1, wherein the determination unit determines a plurality of values as the change data. The conversion table is located at a position temporally or spatially close to the reference data of the first data, with the predetermined value data of the first data as the reference data. A frequency distribution is generated for each value of the reference data, and is generated for each value of the reference data based on the frequency distribution of the first data for each value of the reference data. The data processing apparatus according to claim 1 , wherein: The conversion table uses a predetermined value of the first data as the reference data, and the frequency distribution of the first data that is temporally or spatially adjacent to the reference data Is generated for each value of the reference data, and is generated for each value of the reference data based on the frequency distribution of the first data for each value of the reference data. The data processing apparatus according to claim 1 . The data processing apparatus according to claim 1, wherein the first data is a pixel value constituting an image. The data processing apparatus according to claim 1, further comprising storage means for storing the conversion table for each value of the reference data. The data processing apparatus according to claim 1, further comprising a generating unit that generates the conversion table for each value of the reference data. A data processing method for encoding second data obtained by converting first data according to third data,
A selection step of selecting the first data and selecting the other first data having a predetermined positional relationship with the selected first data as reference data;
A conversion step of converting the first data into the second data based on a conversion table generated for each value of the reference data and associated with the first and second data;
A frequency distribution calculating step for obtaining a frequency distribution of at least a part of the second data;
A determination step of determining change target data to be changed according to the third data from the partial data according to the frequency distribution, and determining change data that is a value for changing the change target data When,
An encoding step of embedding data related to the third data in the second data by changing the change target data to the change data according to the third data ,
The conversion table uses a predetermined value of the first data as the reference data, and a frequency distribution of the first data in a predetermined positional relationship with respect to the reference data. Generated for each value of the reference data, and based on the frequency distribution of the first data for each value of the reference data, the first value of each value for which the frequency distribution is obtained for the reference data A data processing method characterized by being generated for each value of the reference data by assigning the second data to the data in ascending or descending order of frequency . A recording medium on which a program for causing a computer to perform data processing for encoding second data obtained by converting the first data according to the third data is recorded,
A selection step of selecting the first data and selecting the other first data having a predetermined positional relationship with the selected first data as reference data;
A conversion step of converting the first data into the second data based on a conversion table generated for each value of the reference data and associated with the first and second data;
A frequency distribution calculating step for obtaining a frequency distribution of at least a part of the second data;
A determination step of determining change target data to be changed according to the third data from the partial data according to the frequency distribution, and determining change data that is a value for changing the change target data When,
According to the third data, there is recorded a program comprising: an encoding step of embedding data related to the third data in the second data by changing the change target data to the change data. and,
The conversion table uses a predetermined value of the first data as the reference data, and a frequency distribution of the first data in a predetermined positional relationship with respect to the reference data. Generated for each value of the reference data, and based on the frequency distribution of the first data for each value of the reference data, the first value of each value for which the frequency distribution is obtained for the reference data A recording medium generated for each value of the reference data by assigning the second data to the data in ascending or descending order of frequency . A data processing apparatus for decoding encoded data obtained by encoding second data obtained by converting first data according to third data,
Frequency distribution calculating means for determining a frequency distribution of at least a part of the encoded data;
Based on the frequency distribution, a determination unit that determines, from among the partial data, change target data changed according to the third data, and change data that is a value obtained by changing the change target data;
Decoding means for decoding the encoded data into the second data and decoding the third data embedded in the encoded data by changing the change data into the change target data; ,
Selecting means for selecting the second data and selecting the first data that has already been inversely transformed and is in a predetermined positional relationship with the selected second data as reference data;
Inverse conversion means for inversely converting the second data into the first data based on a conversion table generated for each value of the reference data and associated with the first and second data. Prepared ,
The conversion table uses a predetermined value of the first data as the reference data, and a frequency distribution of the first data in a predetermined positional relationship with respect to the reference data. Generated for each value of the reference data, and based on the frequency distribution of the first data for each value of the reference data, the first value of each value for which the frequency distribution is obtained for the reference data A data processing device generated for each value of the reference data by assigning the second data to the data in ascending or descending order of frequency . The data processing apparatus according to claim 12 , wherein the determining means determines data in which a change in frequency is discontinuous from the encoded data as the change target data or change data. . The data processing apparatus according to claim 12 , wherein the determination unit determines a plurality of values as the change data. The conversion table is located at a position temporally or spatially close to the reference data of the first data, with the predetermined value data of the first data as the reference data. A frequency distribution is generated for each value of the reference data, and is generated for each value of the reference data based on the frequency distribution of the first data for each value of the reference data. The data processing apparatus according to claim 12 , characterized in that: The conversion table uses a predetermined value of the first data as the reference data, and the frequency distribution of the first data that is temporally or spatially adjacent to the reference data Is generated for each value of the reference data, and is generated for each value of the reference data based on the frequency distribution of the first data for each value of the reference data. The data processing device according to claim 12 . The data processing apparatus according to claim 12 , wherein the first data is a pixel value constituting an image. The data processing apparatus according to claim 12 , further comprising storage means for storing the conversion table for each value of the reference data. The data processing apparatus according to claim 12 , further comprising an acquisition unit that acquires the conversion table for each value of the reference data. A data processing method for decoding encoded data obtained by encoding second data obtained by converting first data according to third data,
A frequency distribution calculating step for obtaining a frequency distribution of at least a part of the encoded data;
A determination step for determining change target data changed according to the third data, and change data that is a value obtained by changing the change target data, based on the frequency distribution;
A decoding step of decoding the encoded data into the second data and decoding the third data embedded in the encoded data by changing the changed data into the change target data; ,
A selection step of selecting the second data and selecting, as reference data, the first data that has already been inversely transformed and has a predetermined positional relationship with the selected second data;
An inverse conversion step of inversely converting the second data into the first data based on a conversion table generated for each value of the reference data and associated with the first and second data. Prepared ,
The conversion table uses a predetermined value of the first data as the reference data, and a frequency distribution of the first data in a predetermined positional relationship with respect to the reference data. Generated for each value of the reference data, and based on the frequency distribution of the first data for each value of the reference data, the first value of each value for which the frequency distribution is obtained for the reference data A data processing method characterized by being generated for each value of the reference data by assigning the second data to the data in ascending or descending order of frequency . A recording medium on which a program for causing a computer to perform data processing for decoding encoded data obtained by converting second data obtained by converting first data according to third data is recorded. ,
A frequency distribution calculating step for obtaining a frequency distribution of at least a part of the encoded data;
A determination step for determining change target data changed according to the third data, and change data that is a value obtained by changing the change target data, based on the frequency distribution;
A decoding step of decoding the encoded data into the second data and decoding the third data embedded in the encoded data by changing the changed data into the change target data; ,
A selection step of selecting the second data and selecting, as reference data, the first data that has already been inversely transformed and has a predetermined positional relationship with the selected second data;
An inverse conversion step of inversely converting the second data into the first data based on a conversion table generated for each value of the reference data and associated with the first and second data. The program to be prepared is recorded ,
The conversion table uses a predetermined value of the first data as the reference data, and a frequency distribution of the first data in a predetermined positional relationship with respect to the reference data. Generated for each value of the reference data, and based on the frequency distribution of the first data for each value of the reference data, the first value of each value for which the frequency distribution is obtained for the reference data A recording medium generated for each value of the reference data by assigning the second data to the data in ascending or descending order of frequency .

Images