JP2007028658A - Data processing apparatus, data processing method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable embedding more data and correct restoration of input data. <P>SOLUTION: A data processing apparatus, when first data to be embedded is image data, subjects the image data as the first data to be embedded to a embedding process (A) which is operated with respect to characteristic data using, for example, a line rotation technique. Thereby, the characteristic data is embedded into the image data as the first data to be embedded, so that embedded encoded data is produced. Thereafter, the image data as data to be embedded is subjected to a embedding process which is operated with respect to characteristic embedded data using, for example, the line rotation technique (a first embedding process for the data to be embedded). Thereby, the characteristic embedded data is embedded into the image data as the data to be embedded, so that first embedded encoded data is produced. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a data processing device, a data processing method, and a program. In particular, for example, a large amount of data is embedded in image data, and the image data in which the data is embedded and the embedded data are restored. The present invention relates to a data processing apparatus, a data processing method, and a program that can perform the above.

  For example, as a method of embedding another data series as other information without increasing the amount of data with respect to a certain data series as information, the least significant bit or lower order of digital audio data (audio data) There are things that convert 2 bits into information to be embedded. This method uses the fact that the lower bits of the digital audio data do not significantly affect the sound quality, and simply replaces the lower bits with data constituting another data series. The digital audio data in which another data series is embedded is output as it is without returning the lower bits. That is, in this case, it is difficult to restore the lower bits converted to another data series, and the lower bits do not significantly affect the sound quality. The data series is output in an embedded state.

  However, according to the above method, audio data different from the original audio data is output. In other words, the other data embedded in the original audio data becomes noise for the original audio data, and the sound quality of the audio data embedded with the other data is deteriorated.

  In addition to audio data, image data can be embedded with other data in the same manner as in the above-described audio data. In this case as well, image data embedded with other data is The image quality is deteriorated.

  Therefore, the applicant of the present invention first described an embedding method that can restore the original image data and other data embedded in the original image data without degrading the image quality (minimizing degradation of the image quality). Propose to.

  In other words, image data or the like to be embedded with other data is embedded in the data to be embedded, the other data embedded in the data to be embedded, the data to be embedded, and the data to be embedded in the data to be embedded. The data obtained by the above is referred to as embedded encoded data.

  In the embedding process for embedding embedded data in the data to be embedded, the image data as the data to be embedded is operated in accordance with the embedded data according to a predetermined operation rule. That is, for example, according to an operation rule that the horizontal line (or vertical line) constituting the image data as the embedding target data is rotated leftward by a value corresponding to the embedded data, the image as the embedding target data is Data is manipulated. As a result, as shown in FIG. 1A, embedded encoded data in which embedded data is embedded in the data to be embedded is generated.

  On the other hand, in the restoration process for restoring the embedded encoded data into the embedding target data and the embedded data, the embedded encoded data uses the correlation of the image as the embedding target data, as shown in FIG. The original data to be embedded and the data to be embedded are restored. That is, for example, the horizontal lines constituting the image data set as embedded encoded data are sequentially set as the target lines, and the target lines are rotated pixel by pixel in the left direction, which is the opposite direction to that in the embedding process. However, the correlation with the reference line, which is the horizontal line one line above the target line, is calculated, and the rotation amount that maximizes the correlation is obtained. Then, the attention line is rotated by the rotation amount that maximizes the correlation with the reference line, thereby returning to the original position, and the rotation amount is restored as embedded data embedded in the attention line. The

  From the above, according to the previously proposed embedding process, the embedding target data and the embedding data are compressed into embedding encoded data corresponding to the data amount of the embedding target data, that is, the embedding data is compressed. Can do. Furthermore, according to the previously proposed restoration process, by using the correlation of the image data as the embedding target data, the embedded encoded data can be compared with the original embedding target data without any particular overhead for restoration. Can be restored to embedded data.

  By the way, it is convenient if more embedded data can be embedded, but as described above, the amount of embedded data that can be embedded in image data is determined by the operation rule of rotating a line. Limited by the number of lines that make up the data.

  The present invention has been made in view of such a situation, and an embedding process for embedding more embedded data into embedding target data, and embedded encoded data obtained by the embedding process, It is possible to perform restoration processing for restoring data to be embedded and data to be embedded.

  According to a first aspect of the present invention, there is provided a data processing device that extracts feature data representing feature data from input data, and wherein the other information is embedded using the energy bias of the information. In accordance with a predetermined operation rule for performing an operation so that information is restored to the original information, the feature data is embedded in other arbitrary data, and the first feature data is embedded in the input data It embeds other arbitrary data and further includes an embedding means for embedding arbitrary data.

  The data processing method according to the first aspect of the present invention includes a feature data extraction step for extracting feature data representing the feature from the input data, and the other information embedded using the energy bias of the information In accordance with a predetermined operation rule for performing an operation so that information is restored to the original information, the feature data is embedded in other arbitrary data, and the first feature data is embedded in the input data And embedding any other data, and further embedding any data.

  The program according to the first aspect of the present invention includes: a feature data extraction step for extracting feature data representing the feature from input data; and the information in which other information is embedded by utilizing the energy bias of the information. According to a predetermined operation rule for performing an operation so as to restore the original information, the first feature data is embedded in other arbitrary data, and the other data in which the feature data is embedded in the input data Arbitrary data is embedded, and further, a data process including an embedding step of embedding arbitrary data is performed by a computer.

In the data processing device, the data processing method, or the program according to the first aspect of the present invention, feature data representing the feature is extracted from the input data, and other information is embedded using the energy bias of the information. In accordance with a predetermined operation rule for performing an operation so that the information is restored to the original information, the first feature data is embedded in other arbitrary data, and the feature data is included in the input data. The other arbitrary data that is embedded is embedded, and further arbitrary data is embedded.

  A data processing device according to a second aspect of the present invention is a data processing device for restoring processing data obtained by processing the input data transmitted together with the first feature data representing the characteristics of the input data. , By applying a predetermined restoration process to the processed data, the input data is restored, and the restored input data is output as restored data; Feature data extraction means for extracting feature data; and control means for determining the coincidence of the first and second feature data, and controlling restoration processing in the restoration means based on the determination result, The processing data is based on a predetermined operation rule that performs an operation so that the information embedded with other information is restored to the original information by utilizing the energy bias of the information. Yes, the first feature data is embedded in other arbitrary data, the other arbitrary data in which the first feature data is embedded is embedded in the input data, and further, arbitrary data is embedded. The restoration means restores the input data in which the other arbitrary data in which the first feature data is embedded and the arbitrary data are restored from the processing data, and the first data is restored. The other arbitrary data in which the other feature data is embedded is restored to the other arbitrary data in which the first feature data is embedded, and the input data. The other arbitrary data in which the first feature data is embedded is restored to the first feature data and the other arbitrary data.

  The control means can control the restoration process so that the first and second feature data match.

  The restoration means restores the input data by performing a restoration process that manipulates the processing data according to a predetermined operation rule, and the control means controls an operation on the processing data in the restoration process. Can do.

  The restoration means restores the input data by changing a value of the processing data and performing a restoration process in which the changed processing data is manipulated according to a predetermined operation rule. The change of the value of the processing data in the restoration process can be controlled.

  The second data processing method of the present invention is a data processing method for restoring the processing data obtained by processing the input data, which is transmitted together with the first feature data representing the characteristics of the input data, A restoration step of restoring the input data by performing a predetermined restoration process on the processing data, and outputting the restored input data as restoration data; and second feature data representing the feature from the restoration data And a control step for determining the coincidence of the first and second feature data and controlling the restoration process in the restoration step based on the result of the judgment. The data is a predetermined operation route for performing an operation so that the information embedded with other information is restored to the original information by utilizing the energy bias of the information. Accordingly, the first feature data is embedded in other arbitrary data, the other arbitrary data in which the first feature data is embedded is embedded in the input data, and further arbitrary data is embedded. In the restoration step, the input data in which the other arbitrary data in which the first feature data is embedded is embedded from the processing data, and the arbitrary data is restored, and the first data is restored. Restoring the input data in which the other arbitrary data embedded with one feature data is embedded into the other arbitrary data in which the first characteristic data is embedded, and the input data; The other arbitrary data in which the first feature data is embedded is restored to the first feature data and the other arbitrary data.

  The second program of the present invention is a program for causing a computer to perform data processing for restoring processing data obtained by processing the input data, which is transmitted together with the first feature data representing the characteristics of the input data. Then, by performing a predetermined restoration process on the processing data, the input data is restored, a restoration step of outputting the restored input data as restoration data, and a second characteristic representing the characteristics from the restoration data. A feature data extraction step for extracting two feature data; and a control step for determining matching between the first and second feature data and controlling a restoration process in the restoration step based on a result of the determination. The processing data includes the information embedded with other information using the energy bias of the information, and the original information is restored. The first feature data is embedded in other arbitrary data, and the other arbitrary data in which the first feature data is embedded is embedded in the input data in accordance with a predetermined operation rule for performing the operation. Further, arbitrary data is embedded, and in the restoration step, the input data in which the other arbitrary data in which the first feature data is embedded is embedded from the processing data, and the arbitrary data And the other arbitrary data in which the first feature data is embedded, the other arbitrary data in which the first feature data is embedded, and the other arbitrary data in which the first feature data is embedded, Restoring to input data, and further restoring the other arbitrary data in which the first feature data is embedded into the first feature data and the other arbitrary data That.

  In the second data processing apparatus, data processing method, or program of the present invention, the input data is restored by applying a predetermined restoration process to the processed data, and the restored input data is used as restored data. Output, the second feature data representing the feature is extracted from the restored data, the consistency between the first and second feature data representing the feature of the input data is determined, and based on the determination result, Restoration processing is controlled, and the processing data is in accordance with a predetermined operation rule that performs an operation so that the information embedded with other information is restored to the original information by using the energy bias of the information, The first feature data is embedded in other arbitrary data, and the other arbitrary data in which the first feature data is embedded is embedded in the input data. Furthermore, arbitrary data is embedded, and from the processing data, the input data in which the other arbitrary data in which the first feature data is embedded is embedded, and the arbitrary data is restored, The input data in which the other arbitrary data in which the first feature data is embedded is embedded, the other arbitrary data in which the first feature data is embedded, and the input data are restored, Further, the other arbitrary data in which the first feature data is embedded is restored to the first feature data and the other arbitrary data.

  According to the first aspect of the present invention, a lot of data can be embedded.

  According to the second aspect of the present invention, input data can be accurately restored.

  FIG. 2 shows a configuration example of an embodiment of an embedded encoding / decoding system to which the present invention is applied.

  This embedded encoding / decoding system includes an encoding device 11 and a decoding device 12. The encoding device 11 generates and outputs embedded encoded data by embedding embedded data in, for example, image data as data to be embedded, and the decoding device 12 outputs the embedded encoded data. The original data to be embedded and the data to be embedded are decrypted.

  That is, the encoding device 11 includes an embedding target database 1, an embedded database 2, and an embedded encoder 3.

  The embedding target database 1 stores, for example, digital image data as embedding target data. Then, the image data stored therein is read from the embedding target database 1 and supplied to the embedding encoder 3.

  The embedded database 2 stores embedded data (digital data). The embedded data stored in the embedded database 2 is read out and supplied to the embedded encoder 3.

  The embedded encoder 3 receives the image data from the embedding target database 1 and the embedded data from the embedded database 2. Further, the embedded encoder 3 converts the image data into the embedded data from the embedded database 2 so that the image data from the embedded target database 1 can be decoded using the energy bias. By performing the corresponding operation, the embedded data is embedded in the image data, and the embedded encoded data is generated and output. The embedded encoded data output from the embedded encoder 3 is recorded on a recording medium 4 formed of, for example, a semiconductor memory, a magneto-optical disk, a magnetic disk, an optical disk, a magnetic tape, a phase change disk, or the like. The signal is transmitted via a transmission medium 5 including a terrestrial wave, a satellite line, a CATV (Cable Television) network, the Internet, a public line, and the like, and is provided to the decoding device 12.

  The decoding device 12 is configured by a decompressor 6, in which embedded encoded data provided via the recording medium 4 or the transmission medium 5 is received. Further, the restoring unit 6 restores the embedded encoded data to the original image data and the embedded data by using the energy bias of the image data as the embedding target data. The restored image data is supplied to a monitor (not shown) and displayed, for example.

  The embedded data includes, for example, text data related to the original image data as the embedding target data, audio data, reduced image data obtained by reducing the image data, and data unrelated to the original image data. Can also be used.

  Next, FIG. 3 shows a configuration example of the embedded encoder 3 of FIG.

  The image data as the embedding target data supplied from the embedding target database 1 is supplied to the frame memory 21. The frame memory 21 receives the image data from the embedding target database 1 in units of frames, for example. Memorize temporarily.

  The embedding unit 22 receives the embedded data stored in the embedded database 2 and performs an embedding process for embedding the embedded data in the image data as the embedding target data stored in the frame memory 21. Embedding encoded data that can be restored using the energy bias of the image data is generated.

  The frame memory 21 is composed of a plurality of banks so that a plurality of frames (or fields) can be stored. By switching the bank, the frame memory 21 is supplied from the embedding target database 1. Storage of image data, storage of image data to be embedded by the embedding unit 22, and output of image data (embedded encoded data) after the embedding process can be performed simultaneously. . Thus, even if the image data supplied from the embedding target database 1 is a moving image, real-time output of embedded encoded data can be performed.

  Next, FIG. 4 shows the decompressor of FIG. 2 that restores the embedded encoded data output from the embedded encoder 3 of FIG. 3 to the original image data and embedded data by using the energy bias of the image data. 6 shows a configuration example.

  Embedded encoded data, that is, image data in which embedded data is embedded (hereinafter also referred to as embedded image data as appropriate) is supplied to the frame memory 31, and the frame memory 31 stores embedded image data. Is temporarily stored in units of frames, for example. The frame memory 31 is also configured in the same manner as the frame memory 21 in FIG. 3, and by performing bank switching, real-time processing is possible even if the embedded image data is a moving image.

  The restoration unit 32 performs a restoration process for restoring the embedded image data stored in the frame memory 31 to the original image data and the embedded data using the energy bias of the image data.

  Next, the principles of the embedding process performed by the above-described embedded encoder 3 and the restoration process performed by the decompressor 6 will be described.

  In general, what is called information has a bias (universality) of energy (entropy), and this bias is recognized as information (worthy information). That is, for example, an image obtained by shooting a certain landscape is recognized by a person as being an image of such a landscape because the image (pixel value of each pixel constituting the image, etc.) This is because the image has a corresponding energy bias, and an image without the energy bias is merely noise or the like and is not useful as information.

  Therefore, even if some operation is performed on valuable information and the original energy bias of the information is destroyed, so to speak, some operation is performed by restoring the destroyed energy bias. Information can also be restored to the original information. That is, data obtained by manipulating information can be restored to original valuable information by utilizing the original energy bias of the information.

  Here, examples of the energy (bias) of information include correlation, continuity, and similarity.

  Information correlation refers to the correlation (for example, auto-correlation, certain components and other components) of the components of the information (for example, in the case of an image, pixels and horizontal lines constituting the image). Mean distance). For example, there is a correlation between horizontal lines of an image that represents the correlation between images, and the correlation value representing this correlation is, for example, 2 of the difference between pixel values of corresponding pixels in two horizontal lines. A reciprocal such as a sum of multipliers can be used.

  That is, for example, when there is an image having H horizontal lines as shown in FIG. 5, the correlation between the horizontal line of the first row from the top (first horizontal line) and the other horizontal lines. In general, as shown in FIG. 6A, the horizontal line closer to the first horizontal line (the horizontal line on the upper side of the image in FIG. 5) becomes larger as shown as the correlation with respect to the Mth horizontal line. The horizontal line that is farther from one horizontal line (the lower horizontal line in the image in FIG. 5) becomes smaller as shown as the correlation for the Nth horizontal line. Accordingly, there is a correlation bias in which the closer to the first horizontal line, the greater the correlation with the first horizontal line and the smaller the distance.

  Therefore, now, in the image of FIG. 5, an operation is performed to replace the pixels of the Mth horizontal line that is relatively close to the first horizontal line and the Nth horizontal line that is relatively far from the first horizontal line (1 <M <N ≦ H) When the correlation between the first horizontal line and another horizontal line is calculated for the image after the replacement, it is as shown in FIG. 6B, for example.

  That is, in the image after replacement, the correlation with the Mth horizontal line close to the first horizontal line (Nth horizontal line before replacement) decreases, and the Nth horizontal line far from the first horizontal line (Mth before replacement). The correlation with the horizontal line is increased.

  Therefore, in FIG. 6B, the correlation bias that the correlation increases as the distance from the first horizontal line decreases and the correlation decreases as the distance from the first horizontal line decreases. However, with respect to images, generally, the correlation bias that the correlation becomes larger as it is closer to the first horizontal line and the correlation becomes smaller as it is farther can be used to restore the broken correlation bias. . That is, in FIG. 6B, the correlation with the Mth horizontal line close to the first horizontal line is small and the correlation with the Nth horizontal line far from the first horizontal line is large because of the inherent correlation bias of the image. If this is the case, it is clearly unnatural (funny), and the Mth and Nth horizontal lines should be interchanged. Then, by replacing the Mth horizontal line and the Nth horizontal line in FIG. 6B, it is possible to restore the image having the original correlation bias as shown in FIG. 6A, that is, the original image.

  Here, in the case described with reference to FIGS. 5 and 6, the horizontal line replacement operation embeds the embedded data in the image data. When embedding the embedded data, for example, an operation on the image data such as how many horizontal lines are moved and which horizontal lines are exchanged is determined in accordance with the embedded data. On the other hand, replacing the horizontal line of the embedded image data (embedded encoded data) in which the embedded data is embedded to the original position using the correlation of the original image data restores the original image data. become. In the restoration, for example, the embedded data embedded in the image data is restored based on an operation on the embedded image data such as how many horizontal lines are moved and which horizontal lines are replaced. .

  Next, an example of embedding processing and restoration processing using the correlation of image data will be described with reference to FIGS.

  7 to 9, in the embedding process, some pixels constituting the image data as the embedding target data are selected as the processing target pixels, and the selected pixels correspond to the embedded data. Then, by operating in the level direction (by operating the pixel value), the embedded data is embedded in the pixel. On the other hand, in the restoration process, some pixels (pixels at the same position as selected in the embedding process) constituting the embedding image data obtained by the embedding process are selected as processing target pixels, as shown in FIG. In addition, the pixel value is changed by performing an operation opposite to that in the embedding process on the processing target pixel. Further, as shown in FIG. 7, a correlation value R1 between the pixel to be processed before the pixel value is changed and its peripheral pixels (pixels adjacent to the left and right in the embodiment of FIG. 7) is calculated, and the pixel A correlation value R2 between the processing target pixel whose value has been changed and the surrounding pixels of the pixel is calculated. Then, based on the magnitude relationship between the correlation values R1 and R2, either one of the processing target pixels before or after the change of the pixel value is set as the restoration result of the original image data, and the process The embedded data embedded in the target pixel is restored.

  FIG. 8 is a flowchart showing an example of an embedding process performed by the embedding encoder 3 of FIG. 3 using the correlation of image data.

  In the embedding target database 1, image data as embedding target data stored therein is read out, and supplied to the frame memory 21 and stored, for example, in units of one frame.

  On the other hand, the embedding unit 22 receives the data to be embedded from the embedded database 2, for example, bit by bit, and in step S1, selects a pixel (processing target pixel) that is a processing target for embedding the 1-bit embedded data. The image is selected from the images stored in the frame memory 21.

  Here, in the present embodiment, for example, pixels are sequentially selected from the image data stored in the frame memory 21 in the form of a five-eye lattice so as to be processed pixels.

  Thereafter, in step S2, the embedding unit 22 determines whether the embedded data received from the embedded database 2 is 1 or 0. When it is determined in step S2 that the embedded data is 1 or 0, for example, 0, the process returns to step S1. In other words, when the embedded data is 0, the embedding unit 22 does not perform any operation on the processing target pixel (adds 0 as a predetermined constant), returns to step S1, and returns to the next 1 Waiting for the bit embedded data to be transmitted from the embedded database 2, the pixel to be processed next is selected, and the same processing is repeated thereafter.

  If it is determined in step S2 that the embedded data is 1 or 0, for example, 1, the process proceeds to step S3 and the embedding unit 22 performs a predetermined operation on the processing target pixel. . That is, the embedding unit 22 adds, for example, 2 as the predetermined constant to the number of bits assigned to the pixels constituting the image data minus the first power, to the pixel value of the processing target pixel.

Therefore, for example, when 8 bits are allocated as the pixel value of the pixels constituting the image data, 2 ⁷ is added to the pixel value of the processing target pixel in step S3.

  This addition may be performed for either the luminance component Y or the color components U and V when the pixel value is expressed in, for example, YUV. The addition may be performed on any of R, G, and B when the pixel value is expressed in RGB, for example. That is, when the pixel value is composed of a plurality of components, the addition in step S3 may be performed on one or more of the plurality of components, or may be performed on all of the components. good. Further, the addition in step S3 can be performed independently for each of a plurality of components.

In step S3, after 2 ⁷ is added to the pixel value of the pixel to be processed, the process proceeds to step S4, and it is determined whether or not the addition result has overflowed. If it is determined in step S4 that the addition result has not overflowed, step S5 is skipped, and the process proceeds to step S6. The embedding unit 22 uses the addition result as the pixel value of the processing target pixel, and the frame memory 21 Is written (overwritten), and the process returns to step S1.

Further, in step S4, if the addition result is determined to be overflowed, i.e., if the present embodiment, since the pixel value is 8 bits, the addition result is 2 after ⁸ or more, step S5 Then, the embedding unit 22 corrects the added value. That is, in step S5, the addition value overflows, for example, is corrected to the overflowed amount (value obtained by subtracting 2 ⁸ from the added value obtained by overflow). In step S 6, the embedding unit 22 writes the corrected addition result as the pixel value of the processing target pixel in the frame memory 21, and the next 1-bit embedded data is transmitted from the embedded database 2. After waiting for this, the process returns to step S1.

  After embedding processing is performed on one frame of image data stored in the frame memory 21, the image data of one frame, that is, embedded image data is read from the frame memory 21 and output. The Then, the embedding unit 22 continues the embedding process for the next one frame of image data stored in the frame memory 21.

  Next, with reference to the flowchart of FIG. 9, the restoration process by the restoration unit 6 for restoring the embedded image data obtained by the embedding process of FIG. 8 will be described.

  In the frame memory 31, the embedded image data supplied thereto is sequentially stored, for example, in units of one frame.

  On the other hand, the restoration unit 32 selects a pixel to be restored (processing target pixel) from embedded image data of a certain frame stored in the frame memory 31 in step S11.

  Here, in the restoration unit 32, similarly to the embedding unit 22, pixels are sequentially selected from the embedded image data stored in the frame memory 31 in the form of a five-eye lattice. That is, in the embedding process and the restoration process, pixels at the same position are selected as processing target pixels.

  Thereafter, the process proceeds to step S12, and the restoration unit 32 performs an operation opposite to the operation performed by the embedding unit 22 of FIG. That is, the embedding unit 22 subtracts, for example, 2 as a predetermined constant, the number of bits assigned to the pixels constituting the image minus the first power, from the pixel value of the processing target pixel.

Therefore, as described above, as the pixel values of pixels constituting the embedded image data, for example, in a case where 8 bits are assigned, in step S12, 2 ⁷ is subtracted from the pixel value of the processing pixel It will be.

  Note that this subtraction may be performed on either the luminance component Y or the color components U and V when the pixel value is expressed in, for example, YUV. Further, the subtraction may be performed on any of R, G, and B when the pixel value is expressed in RGB, for example. However, the subtraction in step S12 needs to be performed on the same thing as that in step S3 of FIG. That is, when the pixel value is expressed by, for example, YUV and the addition in step S3 in FIG. 8 is performed on, for example, the Y component of YUV, the subtraction in step S12 is: After all, it is necessary to carry out with respect to the Y component.

In step S12, the pixel value of the pixel to be processed, after the 2 ⁷ is subtracted, the process proceeds to step S13, the subtraction result is whether the underflow is determined. If it is determined in step S13 that the subtraction result does not underflow, step S14 is skipped and the process proceeds to step S15.

If it is determined in step S13 that the subtraction result is underflowing, that is, if the addition result is less than 0, the process proceeds to step S14, and the restoration unit 32 corrects the subtraction value. . That is, in step S14, the subtraction value underflows, for example, is corrected to a value obtained by adding 2 ⁸ to the subtracted value, the process proceeds to step S15.

In step S15, the pixel value of the pixel to be processed (the pixel value not subtracted 2 ⁷ in step S12) (hereinafter referred to as the first pixel value as appropriate) P1 and the subtraction value obtained by subtracting 2 ⁷ from the pixel value ( In the following, for each of P2 (hereinafter also referred to as the second pixel value as appropriate), for example, pixels adjacent to the left and right of the pixel to be processed. A correlation value representing the correlation between is calculated.

  That is, in step S15, for example, the absolute value of the difference between the first pixel value P1 of the pixel to be processed and the pixel values of the left and right pixels is calculated, and the resulting sum of the two absolute values is calculated. The reciprocal is obtained as the correlation value R1 for the first pixel value P1 of the pixel to be processed. Further, in step S15, for the second pixel value P2 of the processing target pixel, the reciprocal of the addition value of the absolute values of the differences between the respective pixel values of the left and right pixels is calculated, and this is calculated. It is obtained as the correlation value R2 of the second pixel value P2.

  Note that, in step S15, the pixels used for obtaining the correlation with the processing target pixel are not limited to the pixels adjacent to the left and right, but may be pixels adjacent vertically and temporally. It may be a pixel adjacent to. Further, the pixels need not necessarily be adjacent in terms of space or time. However, in obtaining the correlation with the processing target pixel, it is desirable to use a pixel in which embedded data is not embedded in the embedded image data. This is because even if the correlation between the pixel to be processed and the pixel in which the embedded data is embedded is obtained, the correlation with respect to the original image data cannot be obtained, and therefore the correlation of the image data can be used. This is because it is difficult to accurately restore the original pixel value and the embedded data from the pixel in which the embedded data is embedded. In addition, as long as the processing target pixel is restored using the correlation of the image data, the pixel used for obtaining the correlation value with the processing target pixel has a close spatial or temporal distance to the processing target pixel. It is desirable that

  After calculating the correlation value R1 for the first pixel value P1 and the correlation value R2 for the second pixel value P2, the process proceeds to step S16, and the restoration unit 32 compares the correlation values R1 and R2. .

  If it is determined in step S16 that the correlation value R1 is greater than (or greater than) the correlation value R2, the process proceeds to step S17, where the restoration unit 32 outputs 0 as the restoration result of the embedded data. Return to S11. In this case, the stored value of the frame memory 31 is not rewritten (or the pixel value P1 is overwritten). Therefore, the restoration result of the pixel value of the pixel to be processed remains the pixel value P1.

That is, the fact that the correlation value R1 for the first pixel value P1 is larger than the correlation value R2 for the second pixel value P2 means that the pixel value of the pixel to be processed is a pixel value rather than the pixel value P2. Since the value P1 is more likely, the restoration result of the pixel value of the processing target pixel is the likely pixel value P1. Further, since 2 ⁷ is not subtracted in step S12, it is considered that 2 ⁷ is not added in step S3 of FIG. In the embedding process of FIG. 8, when the embedded data is 0, 2 ⁷ is not added. Therefore, the correlation value R1 for the first pixel value P1 is larger, and the pixel value P1 is When the pixel value of the processing target pixel is probable, the embedded data embedded therein is 0.

On the other hand, if it is determined in step S16 that the correlation value R2 is greater than (or greater than) the correlation value R1, the process proceeds to step S18, and the restoration unit 32 stores the pixel value of the processing target pixel stored in the frame memory 31. Is rewritten to a subtraction value obtained by subtracting 2 ⁷ from the pixel value, that is, the second pixel value P2. Therefore, in this case, the restoration result of the pixel value of the processing target pixel is the pixel value P2. In step S19, the restoration unit 32 outputs 1 as the restoration result of the embedded data, and the process returns to step S11.

That is, the fact that the correlation value R2 for the second pixel value P2 is larger than the correlation value R1 for the first pixel value P1 means that the pixel value of the pixel to be processed is a pixel value rather than the pixel value P1. Since the value P2 is more likely, the restoration result of the pixel value of the processing target pixel is the likely pixel value P2. Further, since the pixel value P2 is obtained by subtracting 2 ⁷ from the pixel value P1 in step S12, it is considered that 2 ⁷ is added to the original pixel value in step S3 of FIG. . In the embedding process shown in FIG. 8, when the embedded data is 1, 2 ⁷ is added. Therefore, the correlation value R2 for the second pixel value P2 is larger, and the pixel value P2 is If the pixel value of the processing target pixel is probable, the embedded data embedded therein is 1.

  Here, when the difference between the correlation values R1 and R2 obtained as described above is small, it is generally determined which of the pixel values P1 and P2 is likely to be the pixel value of the processing target pixel. I can not say. Therefore, in such a case, not only the pixels adjacent to the right and left of the pixel to be processed but also other pixels are used to obtain correlation values for the pixel values P1 and P2, and compare the correlation values. Thus, it can be determined which of the pixel values P1 and P2 is likely to be the pixel value of the processing target pixel.

  As described above, the embedded image data, which is the image data in which the embedded data is embedded, is restored to the original image data and the embedded data using the correlation between the images. Even without the overhead, the embedded image data can be restored to the original image data and the embedded data. Therefore, in the restored image (decoded image), basically, image quality deterioration due to embedding embedded data does not occur.

That is, according to the embedding process of FIG. 8, a predetermined value (0 or 2 ^{7 in} the embodiment of FIG. 8) corresponding to the value of the data to be embedded is added to the pixel value of the image data. In the case of overflow, the embedded data is embedded in the image data and the embedded image data is generated in accordance with the operation rule that the correction is performed.

  On the other hand, according to the restoration process of FIG. 9, according to the operation rule in the embedding process, an operation opposite to that in the embedding process is performed on the embedded image data, and based on the correlation of the data after the operation, In other words, the operation performed in the embedding process is specified. Then, according to the specified operation, the original image data and the embedded data are restored from the embedded image data.

  Therefore, it can be said that the operation rule in the embedding process of FIG. 8 is an operation rule that can restore the embedded image data in which the embedded data is embedded using the correlation of the image data. In the processing, by adopting such an operation rule, the embedded data is embedded in the image data without degrading the image quality of the image data (suppressing the minimum degradation) and without increasing the data amount. be able to.

  That is, the pixel in which the embedded data is embedded is compared with the original pixel without overhead by using the correlation of the image, that is, the correlation with the pixel in which the embedded data is not embedded. It can be restored (returned) to embedded data. Therefore, in the restored image (decoded image) obtained as a result, the image quality is basically not deteriorated by embedding the embedded data.

  In the embodiment of FIGS. 7 to 9, the reciprocal of the absolute value of the difference between the pixel values is used as the correlation value representing the correlation between the pixel to be processed and the other pixels. The correlation value is not limited to this.

  In the embodiment shown in FIGS. 7 to 9, pixels are selected from the image data in the form of a fifth grid, and the embedded data is embedded in the pixels. The selection pattern is not limited to this. However, in the restoration of the pixel in which the embedded data is embedded, as described above, it is desirable to obtain the correlation using the pixel in which the embedded data is not embedded, and the correlation between the pixels is basically The smaller the spatial or temporal distance between them, the smaller. Therefore, from the viewpoint of accurate restoration, it is desirable to select the pixels in which the embedded data is embedded so as to be sparse in terms of space or time. On the other hand, from the viewpoint of embedding a large amount of embedded data, that is, from the viewpoint of the compression rate, it is necessary to increase the number of pixels in which embedded data is embedded to some extent. Therefore, it is desirable to select a pixel in which to embed data to be embedded while balancing the accuracy of restoration and the compression rate.

Furthermore, in the embodiment shown in FIGS. 7 to 9, 1-bit embedded data is embedded in one pixel selected as a processing target pixel, but 2-bit or more embedded data is embedded in one pixel. It is also possible to do so. For example, when embedding 2-bit embedded data in one pixel, according to the 2-bit embedded data, for example, any one of 0, 2 ⁶ , 2 ⁷ , 2 ⁶ +2 ⁷ What is necessary is just to add to a value.

In addition, in the embodiment of FIGS. 7 to 9, by adding either 0 or 2 ^{7 to} the pixel value (without adding or adding 2 ⁷ to the pixel value), was to embed the embedded data, the value to be added to the pixel value is not limited to 2 ^7. However, when adding a value that only affects the lower bits of the pixel value, the added value and the original pixel value do not differ so much, and therefore step S15 in FIG. Correlation values R1 and R2 obtained in step 1 are not so different. This degrades the accuracy of the restoration result of the pixel value and the embedded data, so that the value added to the pixel value according to the embedded data is a value that affects the upper bits of the original pixel value. Is desirable.

  Furthermore, in the embodiment of FIGS. 7 to 9, the embedded data is embedded by adding a predetermined value to the pixel value. However, the embedded data is embedded by an operation other than addition (for example, It is also possible to perform bit inversion and the horizontal line rotation described above on the pixel value. However, as described above, from the viewpoint of preventing deterioration in accuracy of the restoration result of the pixel value and the embedded data, the operations performed on the pixel value are the correlation with respect to the original image and the correlation with respect to the image subjected to the operation. It is desirable for the to be very different.

  In the embodiment shown in FIGS. 7 to 9, 1-bit embedded data is embedded in one pixel selected as a processing target pixel. However, 1-bit embedded data is embedded in a plurality of pixels. It is also possible to make it. That is, it is possible to perform the same operation corresponding to 1-bit embedded data on two or more adjacent pixels.

  Next, an embedding process and a restoration process using information continuity will be described.

  For example, when attention is paid to one horizontal line in the image data, if a waveform WAVE # 1 having a continuous pixel value change pattern as shown in FIG. In other horizontal lines that are far from each other, a change pattern of pixel values different from that of the target line is observed. Therefore, the pixel value change pattern is different between the attention line and another horizontal line away from the attention line, and the continuity is also biased. That is, when attention is paid to the pixel value change pattern of a certain part of the image, a similar pixel value change pattern exists in a part adjacent to the target part. There is a continuity bias that exists.

  Therefore, now, a part of the waveform WAVE # 1 in which the change pattern of the pixel value in the horizontal line with the image data shown in FIG. 10A is continuous is separated from the horizontal line as shown in FIG. 10B, for example. Is replaced with the waveform WAVE # 2.

  In this case, the continuity bias of the image data is destroyed. However, by using the continuity bias that the change pattern of pixel values of adjacent parts is continuous and the change pattern of pixel values is different as the distance is increased, the destroyed continuity bias can be restored. it can. That is, in FIG. 10B, the change pattern of the pixel value of the part WAVE # 2 of the waveform is greatly different from the change pattern of the pixel value of the other part. Therefore, the portion WAVE # 2 that is clearly unnatural and is different from the change pattern of the pixel value of the other portion should be replaced with a waveform similar to the change pattern of the pixel value of the other portion. Then, by performing such replacement, the continuity bias is restored, whereby the original waveform shown in FIG. 10A can be restored from the waveform shown in FIG. 10B.

  Here, in the case described with reference to FIG. 10, it is possible to replace part of the waveform with a waveform of a pixel value change pattern that is significantly different from the surrounding pixel value change pattern. Data will be embedded. Further, when embedding, for example, which part of the waveform the pixel value change pattern is to be replaced and how much the pixel value change pattern is changed are determined according to the embedded data. On the other hand, embedded image data in which embedded data is embedded (embedded encoded data), that is, a waveform having a partially different pixel value change pattern in a part thereof, the peripheral pixel value change patterns are continuous and separated. The original waveform is restored by returning to the original waveform by utilizing the continuity bias that the pixel value change patterns are different. Then, at the time of the restoration, for example, which part of the waveform the change pattern of the pixel value has changed greatly, how much the change pattern of the pixel value has changed, that is, the embedded image data, The embedded data embedded in the image data is restored depending on what operation is used to restore the original image data (waveform).

  Next, FIG. 11 is a flowchart illustrating an example of an embedding process when embedding using the continuity of images is performed in the embedded encoder 3.

  First, in step S <b> 21, the embedding unit 22 causes the frame memory 21 to store image data for one frame as the embedding target data stored in the embedding target database 1. Further, in step S21, the embedding unit 22 detects a continuous area of the image data stored in the frame memory 21, and stores continuous area data indicating the position of the continuous area in a memory for a work (not shown) (hereinafter referred to as appropriate). , Called work memory).

  That is, for example, as shown in FIG. 12A, in the case where the continuous area is detected for image data composed of 224 (= 7 × 32) × 1600 pixels in width × length, For example, the image data is divided into image blocks of 32 × 1 pixels, and DCT (Discrete Cosine Transform) processing is performed for each image block to calculate a DCT coefficient for each image block.

  Further, the embedding unit 22 scans the image blocks in the raster scan order and sequentially sets the blocks of interest. For example, the absolute value of the difference between the DCT coefficient corresponding to the block of interest and the corresponding DCT coefficient of the adjacent image block on the left The sum is calculated in order and stored in the work memory as the continuity evaluation value of the block of interest. In addition, the embedding unit 22 recognizes an image block whose calculated continuity evaluation value (difference value) is equal to or smaller than a predetermined threshold value as a continuous region, and stores the position of the image block in the work memory.

  Here, since there is a possibility that a part of the image block that is supposed to be a continuous area is not a continuous area due to the influence of noise or the like, in order to prevent such a determination from being made, After detecting the continuous area, the embedding unit 22 performs a correction process, and converts the image block adjacent to the continuous area as a non-continuous area into an image block of the continuous area. In other words, according to the correction process, for example, an image block of a non-continuous area in a short section sandwiched between two continuous areas or an image block of one (or a small number) of non-continuous areas adjacent to the continuous area is continuous. Converted into an image block of a region (considered as an image block of a continuous region), whereby two continuous regions and a region sandwiched between them (non-continuous region image block) as a whole The image blocks of one continuous area and a non-continuous area adjacent to the one continuous area are corrected to one continuous area as a whole.

  Thereafter, the process proceeds to step S22, where the embedding unit 22 receives the embedded data from the embedded data database 2, for example, 6 bits (3 bits + 3 bits), and then proceeds to step S23.

  In step S23, the embedding unit 22 selects and extracts a processing target image as a pixel group in which the above-described 6-bit embedded data is embedded. That is, in step S23, for example, one frame of image data stored in the frame memory 21 is virtually divided into upper and lower half areas (hereinafter, appropriately referred to as upper and lower areas). Then, each of the upper and lower regions, for example, a continuous region of horizontal lines at the same position from the top is selected and extracted.

  Specifically, for example, when one frame of image data stored in the frame memory 21 is composed of 1600 horizontal lines as shown in FIG. The area up to the horizontal line is the upper area, and the area from the 801st horizontal line to the 1600th horizontal line is the lower area. Then, when the image data stored in the frame memory 21 is first subjected to the process of step 23, as shown by hatching in FIG. 12A, the first horizontal line which is the first row in the upper area is shown. A continuous region of lines and the 801st horizontal line, which is the first row in the lower region, is selected and extracted.

  The embedding unit 22 refers to the continuous area data stored in the work memory in step S21, and selects and extracts only the continuous area of the first horizontal line and the 801st horizontal line. Here, in the embodiment of FIG. 12, the entire area of the first horizontal line and the 801st horizontal line is selected and extracted as a continuous area.

  In step S24, the embedding unit 22 embeds the data to be embedded in the image data by exchanging the first horizontal line image and the 801 horizontal line image, which are the processing target images.

  That is, FIG. 12B shows the pixel values of the image data of the first horizontal line before the embedded data is embedded. FIG. 12C shows pixel values of the image data of the 801st horizontal line before the embedded data is embedded. As shown in FIGS. 12B and 12C, the change pattern (frequency characteristic) of the pixel value is different between the first horizontal line region and the 801st horizontal line region.

  For example, assuming that 6-bit embedded data with the upper 3 bits being 2 and the lower 3 bits being 6 is embedded in the image data, the embedding unit 22 embeds the upper 3 bits 2 so that FIG. Select the second image block from the left of the first horizontal line shown, and select the sixth image block from the left of the 801 horizontal line shown in FIG. To do. Further, the embedding unit 22 performs an operation of replacing the second image block of the selected first horizontal line with the sixth image block of the 801 horizontal line. Thereby, the change pattern of the pixel value of the first horizontal line is changed as shown in FIGS. 12B to 12D, and the change pattern of the pixel value of the 801 horizontal line is changed as shown in FIGS. 12C to 12E. As a result, 6-bit embedded data in which the upper 3 bits represent 2 and the lower 3 bits represent 6 is embedded in the first horizontal line and the 801st horizontal line.

  Thereafter, in step S25, the embedding unit 22 writes (overwrites) the image data of the first horizontal line and the 801st horizontal line in which the embedded data is embedded in the frame memory 21, and proceeds to step S26.

  In step S <b> 26, the embedding unit 22 determines whether to embed data to be embedded in all horizontal lines of one frame of image data stored in the frame memory 21. If it is determined in step S26 that the embedded data has not yet been embedded in all horizontal lines, the process returns to step S22, and the embedding unit 22 receives new embedded data, and proceeds to step S23. In step S23, the embedding unit 22 selects the next horizontal line from the upper area and the lower area, that is, in this case, selects the second horizontal line and the 802 horizontal line. repeat.

  On the other hand, if it is determined in step S26 that the embedded data is embedded in all horizontal lines of the image data of one frame stored in the frame memory 21, that is, in step S23, the first horizontal line and the 801st When the horizontal line set to the 800th line and 1600th line set are selected as the processing target images, the embedding unit 22 embeds the image data stored in the frame memory 21, that is, the embedded data. The image data (embedded image data) is read and output, and the embedding process is terminated.

  Note that the embedding process in FIG. 11 is repeatedly performed in units of frames for the image data stored in the frame memory 21.

  Next, FIG. 13 is a flowchart illustrating an example of a restoration process that restores embedded encoded data in which embedded data is embedded in image data serving as embedding target data using image continuity.

  First, in step S <b> 31, the restoration unit 32 stores embedded encoded data (embedded image data) for each frame in the frame memory 31. Further, in step S31, the restoration unit 32 extracts a continuous area from the embedded encoded data in the same manner as in step S21 of FIG. 11, and the position of the image block is illustrated as continuous area data. Store in the work memory.

  Here, in the present embodiment, as described with reference to FIG. 12, embedded data of 3 bits is embedded in one horizontal line, but this embedding is performed for one image block in one horizontal line and other horizontal blocks. This is done by an operation of changing to a line image block. Accordingly, at least one of the image blocks constituting one horizontal line is a non-continuous region due to a change operation for embedding embedded data. However, since the correction process is performed at the time of extracting the continuous area as described in Step 21 of FIG. 11, the image block that has been changed to the non-continuous area by the change operation for embedding the embedded data is Converted to continuous area (considered as continuous area).

  Thereafter, the process proceeds to step S32, and the restoration unit 32 virtually divides one frame of embedded image data stored in the frame memory 31 into an upper area and a lower area, as in step S23 of FIG. Referring to the continuous area data stored in the work memory in step S31, the continuous area of the horizontal line at the same position is sequentially selected and extracted from the upper area and the lower area.

  Therefore, when the process of step S32 is first performed on the embedded image data stored in the frame memory 31, the first horizontal line that is the first row in the upper region and the first row that is the first row in the lower region. The 801 horizontal line is selected. Here, as described in step S23 of FIG. 11, if the first horizontal line of the image data as the embedding target data and the entire area (all image blocks) of the 801 horizontal line are continuous areas, the step In S32, all areas of the first horizontal line and the 801st horizontal line are extracted.

  Then, the process proceeds to step S33, and the restoration unit 32 forms the continuous area of the first horizontal line extracted from each of the upper area and the lower area in step S32 and the continuous area of the 801st horizontal line, and the 32 × 1 described above. The image block in pixel units is subjected to, for example, DCT processing, thereby calculating a DCT coefficient as an evaluation value for evaluating the continuity of the image data, and the process proceeds to step S34.

  In step S34, the restoration unit 32 sequentially sets the image blocks constituting the continuous region of the first horizontal line extracted from the upper region as the attention block, for example, from the left side. The DCT coefficient of the image block adjacent to the left is subtracted from the DCT coefficient, and the absolute value sum of the difference values of the DCT coefficient is calculated. Further, the restoration unit 32 sequentially sets the image blocks constituting the continuous area as the attention block for the continuous area of the 801st horizontal line extracted from the lower area, and the DCT coefficient difference for the attention block. Calculate the sum of absolute values. Then, the restoration unit 32 stores the absolute value sum of the calculated difference values of the DCT coefficients in the work memory.

  When there is no image block adjacent to the left of the target block, that is, when the leftmost image block constituting the continuous area is the target block, the absolute value of the difference value of the DCT coefficient for the target block The sum is, for example, 0 as a predetermined value.

  Thereafter, the process proceeds to step S35, and the restoration unit 32 calculates, for each of the two horizontal lines stored in the work memory, the absolute value sum (difference absolute value sum) of the difference values of the DCT coefficients of the image blocks constituting the continuous area. Two image blocks, the first largest image block and the second largest image block, are detected, and the positions of the image blocks are stored in the work memory. That is, for example, in the work memory, the sum of absolute differences of DCT coefficients for the image blocks constituting the continuous area of the first horizontal line and the difference of DCT coefficients for the image block constituting the continuous area of the 801st line When absolute value sums are stored, in the first horizontal line, the image block having the first largest absolute value difference of DCT coefficients (hereinafter referred to as the first image block as appropriate) and second A large image block (hereinafter referred to as the second image block as appropriate) is detected, and the positions of the first and second image blocks are stored in the work memory, and also in the 801 horizontal line, The first image block and the second image block are detected, and the positions of the first image block and the second image block are stored in the work memory.

  In step S36, the restoration unit 32 restores 6-bit embedded data based on the positions of the first and second image blocks for each of the two horizontal lines stored in the work memory. And output.

  In other words, the restoration unit 32 has the first and second image blocks adjacent to each other, and the difference absolute value sums of the DCT coefficients for the first and second image blocks are the same value (difference absolute value). When the difference between the sums is equal to or less than a predetermined threshold), the embedded image data is embedded in the image block position located on the left side of the adjacent first and second image blocks by the embedding process of FIG. Is stored in the work memory as the position of the image block replaced with (hereinafter referred to as an embedding position as appropriate), and a value corresponding to the embedding position is output as a restoration result of the embedded data.

  Specifically, for example, in the embedding process of FIG. 11, for the continuous region of the first horizontal line, as shown in FIGS. 12B and 12D, the third image block from the left and the image of the 801 horizontal line When the replacement operation with the block is performed, the sum of absolute differences of the DCT coefficients is increased for the second and third image blocks from the left. Therefore, for the first horizontal line, in step S35 of FIG. 13, the third and fourth adjacent image blocks from the left are the first and second image blocks (or the second and second image blocks). The top 3 of the 6-bit embedded data described above based on the position (embedding position) of the third image block from the left, that is, the image block that is detected as the first image block) The bit is restored to 2 (= 3-1).

  Also, for example, in the embedding process of FIG. 11, as shown in FIGS. 12C and 12E, for the continuous area of the 801st horizontal line, the seventh image block from the left and the image block of the first horizontal line When the replacement operation is performed, the difference absolute value sum of the DCT coefficients becomes large for the seventh and eighth image blocks from the left. Therefore, for the 801 horizontal line, in step S35 of FIG. 13, the seventh and eighth adjacent image blocks from the left are the first and second image blocks (or the second and second image blocks). Lower-order 3 bits of the above-mentioned 6-bit embedded data based on the position (embedding position) of the seventh image block from the left, ie, the image block located on the left side of the detected image block (the first image block) Is restored to 6 (= 7-1).

  Here, in the embedding process of FIG. 11, when the embedded data is embedded by replacing the left end image block or the right end image block of the continuous area of the horizontal line with the image block of another horizontal line. The difference absolute value sum of the DCT coefficients is increased to some extent only for the image block in which the replacement has been performed, and is not so large for the other blocks. For this reason, the restoration unit 32 has the same absolute value sum of the DCT coefficients for the first and second image blocks when the positions of the first and second image blocks are not adjacent to each other. If the value is not a value (when the difference between the sums of absolute differences is equal to or greater than a predetermined threshold value), the embedded data is restored based on the position of the first image block. That is, in this case, when the first image block is the left end or right end image block, 0 or 7 is restored as the embedded data.

  As described above, when the embedded data is restored based on the embedding position, the process proceeds to step S37, and the restoration unit 32 restores the 6-bit embedded data stored in the work memory in step S35. According to the two embedding positions used in the above, the positions of the image blocks in the two horizontal lines selected in step S32 of the embedding image data stored in the frame memory 31 are exchanged.

  That is, for example, in the embedding process of FIG. 11, as described in FIG. 12, the third image block from the left of the first horizontal line that is the first row in the upper region and the first row that is the first row in the lower region. When the seventh image block from the left of the 801th line is replaced, in step S35 in FIG. 13, for the first horizontal line, the embedding position representing the third from the left is stored, and for the 801st horizontal line. Stores the seventh embedding position from the left. Accordingly, in this case, in step 37, the third image block from the left of the first horizontal line and the seventh image block from the left of the 801 horizontal line are interchanged, whereby the first horizontal line and the 801th image block are replaced. The horizontal line is restored.

  Thereafter, the restoration unit 32 writes the restored two horizontal lines in the form of overwriting the frame memory 31, and proceeds to step S38. In step S38, the restoration unit 32 determines whether or not the restoration of the image data as the embedding target data and the embedded data embedded therein is completed for one frame of the embedded image data stored in the frame memory 31. Determine. If it is determined in step S38 that the restoration of one frame of the embedded image data stored in the frame memory 31 has not been completed, the process returns to step S32, and the upper region in the embedded image data stored in the frame memory 31 From each lower region, the horizontal line of the next row is selected, and the same processing is repeated thereafter.

  If it is determined in step S38 that the restoration of one frame of embedded image data stored in the frame memory 31 has been completed, the process proceeds to step S39, and the restoration unit 32 stores the image data stored in the frame memory 31. That is, the restored image data as the embedding target data is output, and the process ends.

  Note that according to the embodiment of FIG. 13, the image data as the embedding target data is restored for every two horizontal lines, that is, for each horizontal line in the same row in the upper area and the lower area. Therefore, the restored image data can be output after all the image data of one frame stored in the frame memory 31 is restored, as described above, but the two horizontal lines as restoration units are restored. It is also possible to output every time.

  As described above, the embedded image data, which is the image data in which the embedded data is embedded, is restored to the original image data and the embedded data using the continuity of the image. Even without the overhead, the embedded image data can be restored to the original image data and the embedded data. Therefore, in the restored image (decoded image), basically, image quality deterioration due to embedding embedded data does not occur.

  That is, according to the embedding process of FIG. 11, an image block at a position corresponding to the upper 3 bits of 6-bit embedded data in one horizontal line and 6-bit embedded data in another horizontal line. In accordance with the operation rule of replacing the image block at the position corresponding to the lower 3 bits, embedded data is embedded in the image data, and embedded image data is generated.

  On the other hand, according to the restoration process in FIG. 13, the DCT coefficient of the embedded image data is used as an evaluation value of continuity, and the continuity is used. The reverse operation is performed in the case of. As a result, the original image data and the embedded data are restored from the embedded image data.

  Therefore, it can be said that the operation rule in the embedding process of FIG. 11 is an operation rule that can restore the embedded image data in which the embedded data is embedded using the continuity of the image data. In the processing, by adopting such an operation rule, the embedded data is embedded in the image data without degrading the image quality of the image data (suppressing the minimum degradation) and without increasing the data amount. be able to.

  In the above-described case, the DCT coefficient of the embedded image data is used as the continuity evaluation value. However, as the continuity evaluation value, a value other than the DCT coefficient can be used.

  Next, an embedding process and a restoration process using information similarity will be described.

  For example, it is known that a part of an image or the like obtained by photographing a landscape can be generated using the fractal nature (self-similarity) of the image. For example, in an image in which the sea R1 and the forest R2 are displayed as shown in FIG. 14A, a change pattern (for example, an edge shape) of the pixel value of the entire sea R1 and some pixels of the sea R1. Although the similarity with the value change pattern is high, there is a similarity bias that the similarity between the change pattern and the pixel value change pattern of the forest R2 away from the sea R1 is low.

  Therefore, a part R3 of the sea R1 and a part R4 of the forest R2 shown in FIG. 14A are exchanged.

  In this case, the image similarity bias is destroyed, and an image as shown in FIG. 14B is obtained. However, the variation pattern of the pixel values of the adjacent parts has a high similarity, and the similarity deviation of the pixel value change pattern decreases as the distance from the pattern increases. Can be restored. That is, in FIG. 14B, a part of the image of the sea R1 is a part R4 of the image of the forest R2 having low similarity to the sea R1, and a part of the image of the forest R2 is similar to the forest R2. The fact that it is a part R3 of the low-quality image of the sea R1 is obviously unnatural in view of the inherent similarity bias of the image. Specifically, in FIG. 14B, the similarity for part R4 of the image of forest R2 in the image of sea R1 is extremely low compared to the similarity for other parts of sea R1. In addition, the similarity for the part R3 of the image of the sea R1 in the image of the forest R2 is extremely low compared to the similarity for the other part of the forest R2. On the other hand, the similarity between the part R4 of the forest R2 in the sea R1 and the forest R2, and the part R3 of the sea R1 in the forest R2 and the similarity of the sea R1 are all high.

  Therefore, from the similarity bias inherent in the image, part of the image of the sea R1, part R4 of the image of the forest R2, and part of the image of the forest R2, the sea R1. Should be replaced with part R3 of the image. Then, by performing such replacement, the image similarity bias is restored, whereby the original image shown in FIG. 14A can be restored from the image shown in FIG. 14B.

  Here, in the case described with reference to FIG. 14, replacing the part R3 of the image of the sea R1 as the embedding target data with the part R4 of the image of the forest R2 converts the embedded data into the embedding target data. Will be embedded. In the embedding, for example, which part (position on the screen) of the image of the sea R1 and which part of the image of the forest R2 are to be replaced is determined according to the embedded data. On the other hand, embedded image data (embedded encoded data) in which embedded data is embedded, that is, a part of the sea R1 is a part R4 of the forest R2, and a part of the forest R2 is a part of the sea R1. The similarity of the change pattern of the surrounding pixel values of the image that is partly R3 is high, and the similarity bias that the similarity of the change pattern of the pixel values decreases as the distance from the image increases. Returning to the original image restores the original image. In the restoration, for example, which part of the sea image was replaced with which part of the forest image, that is, how the embedded image data was restored to the original image data. Thus, the embedded data embedded in the image data is restored.

  Next, the similarity will be further described with reference to FIG.

  For example, when a fractal image as shown in FIG. 15A is used as data to be embedded and embedded data is embedded therein, a part of the fractal image is similar to the fractal image corresponding to the embedded data. By being replaced with a non-image, embedding of data to be embedded in the fractal image is performed. That is, in FIG. 15A, a fractal image in the shape of a tree leaf is used as the embedding target data. However, embedding of embedded data is performed by converting a part of the fractal image into a triangle as shown in FIG. 15B, for example. It is done by being replaced. Here, in FIG. 15, the portions indicated by D <b> 1 and D <b> 2 in FIG. 15 </ b> B are triangles. Thus, the position of the fractal image replaced with the triangle, the size and number of triangles replaced with the fractal image, It is determined according to the embedded data.

  On the other hand, the embedded encoded data in which the embedded data is embedded in the fractal image as the embedding target data as described above is restored to the fractal image and the embedded data as follows, for example. That is, for example, in FIG. 15B, the portion surrounded by the dotted rectangle is not replaced with the triangle, and the portion surrounded by the dotted rectangle is similar to the teacher image as the teacher image. The part which is not (here, a triangle) is searched. Furthermore, the triangle that is not similar to the teacher image is replaced with an image (fractal image) generated from the reference graphic of the fractal image included in the teacher image, thereby restoring the original fractal image (FIG. 15A). Is done. Then, the embedded data to be embedded is restored based on the searched triangle position, size, number, and the like.

  In the case described above, it is necessary to detect the reference graphic of the fractal image included in the teacher image. This is performed, for example, as follows. That is, based on the similarity between the teacher image and the other parts of the image in FIG. 15B and the self-similarity of the teacher image, the reference graphic of the fractal image is searched for the image in FIG. 15B. The figure that can most efficiently represent the part other than the teacher image is detected as the reference figure.

  In the embodiment of FIG. 15, the original fractal image generation rule is recognized, and an image to be replaced with a triangle is generated using the reference graphic based on the generation rule. That is, when restoring the fractal image as the data to be embedded and the embedded data, the size, position, rotation amount, etc. of the reference graphic for generating the image to be replaced with the triangle are specified based on the generation rule, It is assumed that the reference graphic is operated according to the size, position, rotation amount, etc. of the specified reference graphic, and an image (fractal image) to be replaced with a triangle is generated.

  Next, FIG. 16 is a flowchart illustrating an example of processing when embedding using the similarity of images is performed in the embedded encoder 3.

  In step S51, the embedding unit 22 transfers and stores image data for one frame from the embedding target database 1 to the frame memory 21, further detects a similar region of the image data, and indicates the position of the similar region. The similar area data is stored in the work memory.

  That is, for example, when embedding data to be embedded using a fractal image consisting of 56 × 1600 pixels as shown in FIG. 17A as data to be embedded, the embedding unit 22 first starts from the 56 × 1600 pixels. The fractal image is divided into, for example, image blocks of 8 × 8 pixels, and the image blocks are sequentially set as the target block. Further, the embedding unit 22 scans the pixels constituting the target block, for example, in the raster scan order, and calculates the similarity between the target block and each of the two neighboring image blocks that are image blocks adjacent to the left and right.

  Specifically, the embedding unit 22 changes the size, position, and rotation amount of the target block by a predetermined amount, and matches the target block with each of the left or right adjacent image blocks (peripheral image blocks). Thus, for example, the reciprocal of the sum of absolute differences between each pixel constituting the target block and the corresponding pixel constituting the peripheral image block is obtained as the matching degree. Further, the embedding unit 22 averages the two matching degrees between the block of interest and each of the two neighboring image blocks, and the average value is an evaluation value (similarity evaluation value) representing similarity with the periphery of the block of interest. Is stored in the work memory. Here, when the image block at the left end or the right end is the target block, there is no peripheral image block on the left side or the right side thereof. The matching degree is directly used as the similarity evaluation value of the block of interest. The embedding unit 22 recognizes an image block in which the calculated similarity evaluation value (matching degree) is equal to or greater than a predetermined threshold as a similar region, and stores the position of the image block in the work memory.

  Here, since there is a possibility that a part of the image block that is supposed to be a similar region is not a similar region due to the influence of noise or the like, in order to prevent such a determination from being made, After detecting the similar area, the embedding unit 22 performs a correction process, and converts the image block adjacent to the similar area, which is an unsimilar area, into an image block in the similar area. That is, according to the correction processing, for example, an image block of a non-similar region in a short section sandwiched between two similar regions, or an image block of one (or a small number) of non-similar regions adjacent to the similar region is similar. It is converted into an image block of a region (considered as an image block of a similar region), so that two similar regions and a region sandwiched between them (an image block of a non-similar region) are combined into one similar Alternatively, the image blocks of one similar region and a non-similar region adjacent thereto are corrected to one similar region as a whole.

  Next, in step S <b> 52, the embedding unit 22 receives embedded data from the embedded database 2, for example, 6 bits (3 bits + 3 bits).

  In step S53, the embedding unit 22 selects and extracts a processing target image as an image block group in which the above-described 6-bit embedded data is embedded. That is, in step S53, for example, one frame of image data stored in the frame memory 21 is virtually divided into an upper half area and a lower half area (upper area and lower area). For example, similar regions in the same row from above are selected and extracted.

  Specifically, for example, one frame of image data stored in the frame memory 21 is composed of 1600 horizontal lines as shown in FIG. 17A, for example, and is converted into an 8 × 8 pixel image block in step S51. In the case of division, the image block area from the first line to the 100th line is the upper area, and the image block area from the 101st line to the 200th line is the lower area. When image data stored in the frame memory 21 is first subjected to the processing in step 53, the similar region between the image block in the first row in the upper region and the image block in the first row in the lower region. Are selected and extracted.

  The embedding unit 22 refers to the continuous area data stored in the work memory in step S51, and selects and extracts only the similar area between the image block in the first row and the image block in the 101st row. Here, in the embodiment of FIG. 17, all image blocks in the first row and the 101st row are selected and extracted as being similar regions.

  Thereafter, the process proceeds to step S54, where the embedding unit 22 replaces, for example, one of the image blocks in the first row and one of the image blocks in the 101st row, which is the processing target image, to thereby store the frame memory. The embedded data is embedded in the image data stored in 21.

  That is, FIG. 17A shows image data as data to be embedded before embedding data is embedded, and FIG. 17B shows embedded encoded data after embedding data is embedded.

  In the embedding target data shown in FIG. 17A, the pixel value change pattern, that is, the similarity of graphics in the image block is different between the image block area in the first row and the image block area in the 101st row.

  For example, if 6-bit embedded data in which the upper 3 bits are 2 and the lower 3 bits is 6 is embedded in the image blocks of the first row and the 101st row, the embedding unit 22 has 6 The third image block from the left in the first row in FIG. 17A is selected corresponding to the upper 3 bits 2 of the embedded data of bits, and the lower 3 bits of the embedded data of 6 bits. Corresponding to 6, the seventh image block from the left in the 101st row of FIG. 17A is selected. Further, the embedding unit 22 replaces the selected third image block from the left in the first row and the seventh image block from the left in the 101st row, thereby, as shown in FIG. The embedded data is embedded in the image blocks in the first row and the 101st row.

  Here, the i-th image block from the left is hereinafter appropriately referred to as the i-th image block.

  In step S54, after embedding the embedded data, the process proceeds to step S55, and the embedding unit 22 overwrites the frame memory 21 with the image block in the first row and the image block in the 101st row in which the embedded data is embedded. The program proceeds to step S56.

  In step S <b> 56, the embedding unit 22 determines whether to embed data to be embedded in all image block rows of one frame of image data stored in the frame memory 21. If it is determined in step S56 that the embedded data has not yet been embedded in all the image block rows, the process returns to step S52, and the embedding unit 22 receives the new embedded data, and proceeds to step S53. . In step S53, the embedding unit 22 selects an image block of the next row from the upper region and the lower region, that is, in this case, selects an image block of the second row and an image block of the 102nd row, Thereafter, the same processing is repeated.

  On the other hand, when it is determined in step S56 that the embedded data is embedded in all the image block rows of the image data of one frame stored in the frame memory 21, that is, in step S53, the first row and the first row are embedded. When a set of image blocks of 101 rows to a set of image blocks of the 100th row and the 200th row is selected as the processing target image, the embedding unit 22 stores the image data stored in the frame memory 21, that is, Image data (embedded image data) in which the data to be embedded is embedded is read and output, and the embedding process is terminated.

  Note that the embedding process in FIG. 16 is repeatedly performed as appropriate for the frame of the image data as the embedding target data.

  Next, FIG. 18 is a flowchart illustrating an example of a restoration process that restores embedded encoded data in which embedded data is embedded in image data serving as embedding target data using image similarity.

  First, in step S61, the restoration unit 32 sequentially stores embedded encoded data (embedded image data) for each frame in the frame memory 31. Further, in step S31, the restoration unit 32 extracts a similar region from the embedded encoded data in the same manner as in step S51 of FIG. 16, and illustrates the position of the image block as similar region data. Store in the work memory.

  Here, in the present embodiment, as described with reference to FIG. 16, 3-bit embedded data is embedded in one row of an image block. This embedding is performed by one of the image blocks constituting one row. This is performed by an operation (changing operation) for changing the image block. Accordingly, at least one of the image blocks constituting one row is a dissimilar region due to a change operation for embedding data to be embedded. However, since the correction process is performed as described in step 51 of FIG. 16 when extracting the similar area, the image block that has been changed to the non-similar area by the change operation for embedding the embedded data is It is converted to a similar area (considered as a similar area). Accordingly, the similar region extracted in step S31 includes the image block that has been replaced in the embedding process of FIG.

  Thereafter, the process proceeds to step S62, and the restoration unit 32 virtually divides one frame of embedded image data stored in the frame memory 31 into an upper area and a lower area, as in step S53 of FIG. Referring to the similar area data stored in the work memory in step S61, the similar areas of the image blocks in the same row are sequentially selected and extracted from the upper area and the lower area.

  Therefore, when the process of step S62 is first performed on the embedded image data stored in the frame memory 31, the first row in the upper region and the first row in the lower region are selected. Here, as described in step S53 in FIG. 16, if all of the image block in the first row and the image block in the 101st row of the image data as the embedding target data are similar regions, in step S62, , All image blocks in the first row and the 101st row are extracted as similar regions.

  Then, the process proceeds to step S63, and the restoration unit 32 performs processing similar to the case described in step S51 of FIG. 16 and the similar region of the first row extracted from each of the upper region and the lower region in step S62 and the 101st row. The image blocks of 8 × 8 pixel units constituting the similar region are sequentially set as the target block, and the target block is matched with the surrounding image blocks. As a result, the restoration unit 32 obtains the matching degree between the target block and the surrounding image block, and further calculates the similarity evaluation value from the matching degree in the same manner as described with reference to FIGS. 16 and 17. Ask.

  In the above-described case, the average value of the matching degrees of the target block and the left and right image blocks (peripheral image blocks) is set as the similarity evaluation value of the target block. The larger or smaller matching degree between the left and right image blocks may be adopted as the similarity evaluation value.

  As described above, the restoration unit 32 obtains the similarity evaluation values of the similarity regions of the first row extracted from each of the upper region and the lower region and the image blocks constituting the similarity region of the 101st row. The similarity evaluation value is stored in the work memory, and the process proceeds to step S65.

  In step S65, the restoration unit 32 obtains from the similarity regions of the image blocks of two rows (one row in the upper region and one row in the lower region corresponding to the row) stored in the work memory, respectively. The image block having the maximum similarity evaluation value is detected, and the position of the image block is stored in the work memory. That is, for example, the similarity evaluation value for the image block constituting the similarity region of the first row and the similarity evaluation value for the image block constituting the similarity region of the 101st row are stored in the work memory. In the first row, the image block having the maximum similarity evaluation value (hereinafter referred to as the maximum similarity image block as appropriate) and the maximum similarity image block are detected in the 101st row, and the maximum similarity is detected. The position of the sex image block is stored in the work memory.

  In step S66, the restoration unit 32 restores and outputs 6-bit embedded data based on the positions of the maximum similarity image blocks in the two rows stored in the work memory.

  That is, when the maximum similarity image block in the upper region or the lower region is the i-th image block, the restoration unit 32 sets i−1 as the upper 3 bits or the lower 3 bits of the 6-bit embedded data. Each is restored, and further, 6-bit embedded data is constructed and output from the upper 3 bits and the lower 3 bits.

  Here, for example, in the case shown in FIG. 17B, the maximum similarity image block in the first row is the third image block, and the maximum similarity image block in the 101st row is the seventh image block. Therefore, the upper 3 bits and the lower 3 bits of the 6-bit embedded data are 2 and 6, respectively.

  As described above, when the embedded data is restored based on the position of the maximum similarity image block, the process proceeds to step S67, and the restoration unit 32 stores the 6-bit encoded data stored in the work memory in step S65. In accordance with the positions of the two maximum similarity image blocks used for restoring the embedded data, the positions of the image blocks in the two rows selected in step S62 of the embedded image data stored in the frame memory 31 are switched.

  That is, for example, in the embedding process in FIG. 16, as described in FIG. 17, the third image block in the first row in the upper region and the seventh image block in the first row in the lower region (the entire 101st row). Are replaced with each other, in step S65 of FIG. 18, the position of the third image block is stored for the first row, and the position of the seventh image block is stored for the 101st row. Accordingly, in this case, in step 67, the third image block in the first row and the seventh image block in the 101st row are interchanged, thereby restoring the image data in the first and 101st rows.

  Thereafter, the restoration unit 32 writes the restored two rows of image data (image blocks) in the form of overwriting the frame memory 31, and proceeds to step S68. In step S68, the restoration unit 32 determines whether or not restoration of the image data as the embedding target data and the embedded data embedded therein has been completed for one frame of the embedded image data stored in the frame memory 31. Determine. If it is determined in step S68 that the restoration of one frame of the embedded image data stored in the frame memory 31 has not been completed, the process returns to step S62, and the upper region in the embedded image data stored in the frame memory 31 The next row of image blocks is selected from each of the lower regions, and the same processing is repeated thereafter.

  If it is determined in step S <b> 68 that the restoration of one frame of embedded image data stored in the frame memory 31 has been completed, the process proceeds to step S <b> 69, and the restoration unit 32 stores the image data stored in the frame memory 31. That is, the restored image data as the embedding target data is output, and the process ends.

  In the embodiment of FIG. 18 as well, in the same way as in the embodiment of FIG. 13, the restored image data is output after all the image data of one frame stored in the frame memory 31 is restored. It is also possible to output each time two rows of image blocks that are restoration units are restored.

  As described above, the embedded image data, which is the image data in which the embedded data is embedded, is restored to the original image data and the embedded data using the similarity of the images. Even without the overhead, the embedded image data can be restored to the original image data and the embedded data. Therefore, in the restored image (decoded image), basically, image quality deterioration due to embedding embedded data does not occur.

  That is, according to the embedding process of FIG. 16, the image block at the position corresponding to the upper 3 bits of the 6-bit embedded data in the image block of one row and the 6-bit embedded data constituting another row In accordance with the operation rule of replacing the image block at the position corresponding to the lower 3 bits, embedded data is embedded in the image data, and embedded image data is generated.

  On the other hand, according to the restoration process of FIG. 18, the degree of matching of embedded image data is used as an evaluation value of similarity, and the similarity is used, and the embedded process is performed on the embedded image data according to the operation rule in the embedding process. The reverse operation is performed in the case of. As a result, the original image data and the embedded data are restored from the embedded image data.

  Therefore, it can be said that the operation rule in the embedding process of FIG. 16 is an operation rule that can restore the embedded image data in which the embedded data is embedded using the similarity of the image data. In the processing, by adopting such an operation rule, the embedded data is embedded in the image data without degrading the image quality of the image data (suppressing the minimum degradation) and without increasing the data amount. be able to.

  In the above case, the matching degree of the embedded image data is used as the similarity evaluation value. However, as the similarity evaluation value, a value other than the matching degree can be used.

  As described above, in the embedded encoder 3, the image data is embedded in accordance with the operation rule for performing the operation so that the restoration can be performed using the energy bias of the image data as the embedding target data. When the embedded data is embedded in the image data and the embedded encoded data is output by operating in accordance with the data, the decompressor 6 converts the embedded encoded data into the energy bias of the image data. By using this, it is possible to restore the original image data and embedded data without the overhead for restoration.

  In addition, by embedding data in the image data as the embedding target data, the embedded image data (embedded encoded data) obtained as a result of the embedding is different from the original image data. Since it is no longer an image that can be recognized as valuable information, it is possible to realize encryption without overhead for the image data as the embedding target data.

  Furthermore, a completely reversible digital watermark can be realized. That is, in the conventional digital watermark, for example, the low-order bits of the pixel value that do not significantly affect the image quality are simply changed to values corresponding to the digital watermark. In this case, the low-order bits are changed to the original values. It is difficult to return. Therefore, the image quality of the restored image is deteriorated to some extent due to the change of the lower bits as the digital watermark. On the other hand, when the embedded encoded data is restored using the energy bias of the image data as the embedding target data, the original image data and embedded data without deterioration can be obtained. Therefore, by using the embedded data as a digital watermark, the image quality of the restored image does not deteriorate due to the digital watermark.

  Further, since the embedded data embedded in the embedding target data can be taken out (restored) by restoring the embedding target data from the embedded encoded data, side information is provided without any overhead along with the embedding target data. be able to. In other words, since the embedded data can be embedded in the data to be embedded without the overhead for extracting the embedded data, the embedded encoded data obtained as a result of the embedding is compressed (embedded) by the amount of the embedded data. It can be said that it is compressed. Therefore, for example, if half of an image is to be embedded and the other half is to be embedded, the remaining half of the image can be embedded in the half of the image to be embedded. In this case, the image is simply compressed to ½.

  Furthermore, since the embedded encoded data is restored using so-called statistics, that is, the energy bias of the information, it is highly resistant to errors. That is, robust encoding (statistical encoding) that is highly robust encoding can be realized.

  In addition, for example, while image data is set as data to be embedded, image data is provided using sound as a key by using, for example, audio data of a medium different from the image data as embedded data. It becomes possible. That is, on the encoding device 11 side in FIG. 2, for example, the embedded data such as the voice “open sesame” spoken by the contractor is embedded in the image data that is the embedding target data, and the restoration device 12 side asks the user. The voice “open sesame” is uttered, and the speaker recognition is performed using the voice and the voice embedded in the image data. In this way, for example, as a result of speaker recognition, it is possible to automatically present an image only when the user is a contractor. In this case, the voice as the embedded data can use the voice waveform itself, not so-called feature parameters.

  In addition, for example, while audio data is set as data to be embedded, for example, image data of a medium different from audio is used as embedded data, and audio is provided using an image as a key. (For example, voice response after face recognition). That is, on the encoding device 11 side in FIG. 2, for example, an image of the user's face is embedded in the voice as a response to the user, and on the restoration device 12 side, the user's face is photographed, and the resulting image It is possible to realize a voice response system that performs a different voice response for each user by outputting a voice in which a matching face image is embedded.

  Furthermore, it is also possible to embed information of the same medium in information of a certain medium, such as embedding sound in an audio or embedding an image in an image. Alternatively, if the voice and face image of the contractor are embedded in the image, the image is presented only when the voice and face image of the user match those embedded in the image. In other words, a double key system can be realized.

  In addition, for example, it is possible to embed the other in one of the synchronized images and sounds constituting the television broadcast signal. In this case, the information of different media is integrated. Integrated coding can be realized.

  Further, in the embedding process, as described above, as the information has a characteristic in the bias of energy, more embedded data can be embedded. Therefore, for example, it is possible to control the total amount of data by adaptively selecting one of the two types of information that has a characteristic in energy bias and embedding the other in the selected one. It becomes. That is, between two pieces of information, one piece of information can absorb the other amount of information. As a result of controlling the total amount of data in this way, information transmission (environmentally responsive network transmission) with a data amount suitable for the transmission band and usage status of the transmission path and other transmission environments becomes possible.

  Further, for example, by embedding a reduced image of an image in an image (or by embedding a sound obtained by thinning out the sound), so-called hierarchical encoding (lower hierarchy) is performed without increasing the amount of data. By reducing the amount of information, it is possible to realize (encoding for generating higher layer information).

  Furthermore, for example, by embedding an image as a key for searching for an image in the image, a database for searching for an image based on the image as the key can be realized.

  As described above, there are correlation, continuity, similarity, and the like that represent energy (bias) of information, but in the following, for the sake of simplicity, for example, attention is paid only to correlation. Then, explanation will be given. Therefore, first, three methods (first, second, and third methods) are used as other examples of embedding processing and restoration processing that use image data as embedding target data and use the correlation of the image data. introduce.

  In the first method, in the embedding process, the positions of the pixels constituting the image data as the embedding target data are exchanged corresponding to the embedded data, for example, in units of one column (pixel column aligned in the vertical direction) (pixels). The data to be embedded is embedded in each column by operating the image data as the data to be embedded in accordance with the operation rule.

  That is, in the embedding process of the first method, pixel swap information indicating how to replace the position of each column of image data of one frame as embedding target data is generated based on the embedded data. Is done. Specifically, for example, one frame of image data as embedding target data is composed of pixels of M rows and N columns, and the nth column (the nth column from the left) of the image data is the n'th column. In this case, pixel swap information in which n and n ′ are associated with each other is generated (n and n ′ are integers of 1 or more and N or less).

Here, when the number of columns of image data of one frame is N columns as described above, the replacement method is N! There are only (! Represents the factorial). Therefore, in this case, it is possible to embed embedded data of up to log ₂ (N!) Bits (however, rounded down after the decimal point) in one frame.

  After the pixel swap information is generated, according to the pixel swap information, the position of each column of the image data as the embedding target data is switched (pixel swapped), so that the embedded data is embedded in the embedding target data. Then, embedded encoded data (embedded image data) is generated.

  Here, in the first method, as described above, the embedded data is embedded in accordance with the operation rule of performing pixel swap for each column of the image data as the embedding target data corresponding to the embedded data. Hereinafter, this first method is appropriately referred to as a pixel swap method.

  Next, the embedding process when the embedding process by the pixel swap method is performed in the embedding encoder 3 of FIG. 3 will be described with reference to the flowchart of FIG.

  From the embedding target database 1, image data as embedding target data stored therein is read, for example, in units of one frame, and sequentially supplied to the frame memory 21 and stored.

In step S81, the embedding unit 22 reads from the embedded database 2 data to be embedded having a data amount that can be embedded in an image of one frame. That is, for example, as described above, when the number of columns of an image in one frame is N columns, and all the columns are subjected to pixel swap (replacement), log ₂ is the maximum in one frame. Since it is possible to embed (N!) Bits of embedded data, embedded data having such a number of bits (below) is read from the embedded database 2.

  Then, the embedding unit 22 proceeds to step S82, and generates pixel swap information based on the embedded data read in step S81. That is, the embedding unit 22 selects, for example, the second column to the Nth column excluding the first column from the first column to the Nth column of the frame of the image data stored in the frame memory 21 based on the embedded data. Pixel swap information indicating the number of columns in which each pixel is swapped is generated.

  Thereafter, the process proceeds to step S83, and the embedding unit 22 performs pixel swap on the position of each column of the processing target frame stored in the frame memory 21 in accordance with the pixel swap information. Then, the frame on which the column pixel swap is performed is read from the frame memory 21 and output as embedded encoded data.

  Note that the pixel swap of each column of the frame can be performed by changing the storage position of the image data (the pixels constituting the frame memory 21) in the frame memory 21, but, for example, when reading a frame from the frame memory 21 By controlling the address, as a result, it is possible to read out the frame in which the column pixel swap has been performed from the frame memory 21.

  In addition, here, the pixel swap information includes information indicating to which column the pixel swap is performed for each of the second to Nth columns, but the first column is replaced with the pixel swap. It does not contain information that indicates what to do. Therefore, in the embedding unit 22, the pixel swap for each of the second column to the Nth column is performed, but the pixel swap for the first column is not performed. The reason for not performing pixel swap in the first column will be described later.

  When the embedded encoded data is output from the frame memory 21 in step S83, the process proceeds to step S84, and it is determined whether or not a frame not yet processed is stored in the frame memory 21 and stored. If it is determined that there is a frame, the process returns to step S81, and the same processing is repeated for a frame that has not been processed yet.

  If it is determined in step S84 that no frame not yet processed is stored in the frame memory 21, the embedding process is terminated.

  As described above, the positions of the pixels in each column as a set of one or more pixels constituting the image stored in the frame memory 21 are subjected to pixel swap in accordance with the embedded data. When embedding data to be embedded, the reverse image swap (in this case, column replacement) can be performed to restore the original image data, and what kind of pixel swap has been performed. The embedded data can be restored based on the above. Therefore, it is possible to embed data to be embedded in the image data while minimizing the deterioration of the image quality of the image data and without increasing the data amount.

  That is, each column of image data (embedded image data) that has been subjected to pixel swap at the column position, which is image data in which embedded data is embedded, is the correlation of the image data, that is, here, for example, By utilizing the correlation between the original image data and the columns in the same correct position, it is possible to swap pixels to the original position without any overhead. The embedded data can be restored depending on the column in which the pixel swap is performed. Therefore, in the restored image obtained as a result, there is basically no deterioration in image quality due to embedding data to be embedded.

  Note that when there is no column at the correct position in the embedded encoded data, it is generally difficult to restore the image data and the embedded data using the correlation between the image data as described above. It is. Therefore, here, in the embedding process of FIG. 19, the first column of each frame is output as embedded encoded data without being replaced.

  However, it is also possible to perform embedded encoding with all the columns including the first column as an object of replacement. In this case, for example, at least one original position of the replaced column is embedded as overhead. By including it in the encoded data, the original image data and the embedded data can be easily restored.

  Also, the embedded data can be embedded in the image data by replacing all the columns other than the first column of the image data at once, or by sequentially replacing the columns other than the first column of the image data. It is also possible to embed in the image data.

  Next, the embedded image data obtained by the embedding process by the pixel swap method can be restored to the original image and the embedded data by the restoration process by the pixel swap method using the correlation of the image data as follows. it can.

  That is, in the restoration process using the pixel swap method, the column that makes up the correlation between the latest restored column and other columns among the columns that constitute the embedded image data and the other restored column is maximized. Is performed for all the columns constituting the embedded image data, and the original image data is restored, and the embedded image data is further restored to the original image. The embedded data is restored based on the pixel swap method for each column of the embedded image data when the data is restored.

  Next, with reference to the flowchart of FIG. 20, the restoration process when the restoration process by the pixel swap method is performed in the restoration unit 6 of FIG. 4 will be described.

  In the frame memory 31, the embedded image data supplied thereto is sequentially stored, for example, in units of one frame.

  In step S91, the restoring unit 32 sets, for example, 1 as an initial value to the variable n for counting the number of columns in the frame, and proceeds to step S92, where the variable n is the number of columns in the frame. It is determined whether or not N-1 or less obtained by subtracting 1 from N.

If it is determined in step S92 that the variable n is equal to or less than N−1, the process proceeds to step S93, and the restoration unit 32 determines the pixel (pixel) in the nth column from the frame of the embedded image data stored in the frame memory 31. reading a column), the vector obtained by arranging each pixel (pixel value of) the elements of the n-th column (hereinafter, appropriately, to produce a) v _n of column vectors. Here, when the frame is composed of M rows of pixels, the column vector v _n (the column vector v _k described later) is an M-dimensional vector.

Thereafter, in step S94, the restoration unit 32 sets n + 1 as an initial value to a variable k for counting columns on the right side of the nth column, and proceeds to step S95. In step S95, as in step S93, the restoration unit 32 reads out the k-th column pixel from the frame of the embedded image data, and generates a column vector v _k having the k-th column pixel as an element. The process proceeds to step S96.

In step S96, recovery unit 32, using the column vectors v _n and v _k, obtains a correlation value representing the correlation between the n-th column and the k-th column.

That is, the restoration unit 32 first calculates the distance d (n, k) between the column vectors v _n and v _k according to the following equation.

d (n, k) = | v _n −v _k |
= (Σ (A (m, n) -A (m, k)) ² ) ^1/2 (1)
However, in the above equation, Σ represents the summation with m changed from 1 to M. A (i, j) represents the pixel (pixel value) in the i-th row and j-th column of the frame of the embedded image data to be processed.

Further, the restoration unit 32 uses the reciprocal 1 / d (n, k) of the distance d (n, k) between the column vectors v _n and v _k as a correlation value representing the correlation between the n-th column and the k-th column. Ask.

  After calculating the correlation value between the n-th column and the k-th column, the process proceeds to step S97, and the restoration unit 32 determines whether the variable k is equal to or less than N−1 obtained by subtracting 1 from N, which is the number of columns in the frame. Determine. When it is determined in step S97 that the variable k is N−1 or less, the process proceeds to step S98, and the restoration unit 32 increments the variable k by 1, and returns to step S95. Hereinafter, in step S97, the variable k Steps S95 to S98 are repeated until it is determined that k is not less than N-1. That is, the correlation value between the n-th column and each column of the embedded image on the right side is obtained.

  Thereafter, when it is determined in step S97 that the variable k is not N−1 or less, the process proceeds to step S99, and the restoration unit 32 obtains k that maximizes the correlation value with the n-th column, and proceeds to step S100. . Now, if k that maximizes the correlation value with the n-th column is expressed as K, for example, the restoration unit 32 stores the first frame of the embedded image data to be processed stored in the frame memory 31 in step S100. The pixel swap is performed between the (n + 1) th column and the Kth column, that is, the Kth column is replaced with the (n + 1) th column adjacent to the right of the nth column.

  Thereafter, the process proceeds to step S101, where the restoration unit 32 increments the variable n by 1, returns to step S92, and thereafter, in step S92, until it is determined that the variable n is not N−1 or less, steps S92 to S101 are performed. Repeat the process.

  Here, in the present embodiment, since the first column of the embedded image data remains the first column of the original image data, when the variable n is 1, which is the initial value, the first column is the same as the first column. The column of embedded image data having the highest correlation is subjected to pixel swap to the second column on the right side of the first column. Since the column having the highest correlation with the first column is basically the second column of the original image data because of the correlation of the image data, in this case, in the embedding process of FIG. The second column of the original image data pixel-swapped to that column is returned (restored) to the original position (column).

  When the variable n is 2, the column of the embedded image data having the highest correlation with the second column that has been pixel-swapped to the original position as described above is displayed in the third column on the right side of the second column. Pixel swapped. The column having the highest correlation with the second column is basically the third column of the original image data because of the correlation of the image data. In this case, in the embedding process of FIG. The third column of the original image data that has been subjected to pixel swapping to any column of is returned to the original position.

  Similarly, the embedded image data stored in the frame memory 31 is restored to the original image data.

  If it is determined in step S92 that the variable n is not equal to or less than N-1, that is, all the second to Nth columns constituting the embedded image data are restored to the original positions using the correlation of the image data. If the stored contents of the frame memory 31 are restored to the original image data, the process proceeds to step S102, and the restoration unit 32 reads the restored image data from the frame memory 31. . In step S102, the restoration unit 32 restores the embedded image data to the original image data based on the pixel swap method (pixel swap information) for each of the second to Nth columns of the embedded image data. The embedded data embedded in the embedded image data is restored and output.

  Thereafter, the process proceeds to step S103, where the restoration unit 32 determines whether or not a frame of embedded image data that has not yet been processed is stored in the frame memory 31, and if it is determined that the frame is stored, Returning to S91, the same processing is repeated for frames of embedded image data that are not yet processed.

  If it is determined in step S103 that no frame of embedded image data not yet processed is stored in the frame memory 31, the restoration process ends.

  As described above, the embedded image data, which is the image data in which the embedded data is embedded, is restored to the original image data and the embedded data using the correlation of the image data. Even if there is no overhead, the encoded data can be restored to the original image and the embedded data. Therefore, the restored image basically does not deteriorate in image quality due to embedding the embedded data.

  In the restoration process of FIG. 20, a correlation between the latest restored column (the first column that is not subjected to pixel swap at the time of embedding in the case of n = 1) and the column that has not been restored is obtained. Based on the correlation, a column to be pixel-swapped is detected at the position immediately to the right of the latest column that has already been restored.In addition, for example, each of a plurality of columns that have already been restored is still restored. It is also possible to detect a column to be pixel-swapped to the right of the latest restored column by calculating the correlation with a column that has not been restored.

  Further, in the above-described case, by embedding the pixels constituting the image data as the embedding target data in units of columns based on the embedding data (by performing the pixel swap of the vertical line), the embedding data is changed. Although embedding of the data to be embedded is performed, for example, image data as data to be embedded is subjected to pixel swap in units of rows (horizontal line pixel swap is performed) or arranged in the time direction. It is also possible to perform by swapping pixel columns at the same position in a predetermined number of frames.

  Further, in the embedding process by the pixel swap method, for example, it is possible to embed data to be embedded by performing pixel swap for image data as embedding target data in units of columns and further performing pixel swap in units of rows.

  Here, with respect to the embedded image data obtained by performing the pixel swap for both the column and the row, for example, when considering a restoration process for returning the column position to the original position, in the restoration process, The order of the terms to be added only changes, and the terms to be added themselves do not change. Accordingly, the distance d (n, k) obtained by the expression (1) does not change between the case where the pixel swap is performed for only the column of the image data and the case where the pixel swap is performed for both the column and the row. The embedded image data obtained by pixel swapping can be restored to the original image data and the embedded data by the restoration process of FIG. 20 in the same manner as the embedded image data obtained by pixel swapping only the columns. . That is, the embedded image data obtained by pixel swapping both the column and the row is restored to the original image data and the embedded data by performing the restoration processing of FIG. 20 independently for the column and the row, respectively. be able to.

  From the above, when embedding data is embedded in image data by performing pixel swap for both the column and row in the embedding process, it is possible to restore which pixel swap is performed first or later in the row or column. Does not affect processing. Therefore, in the embedding process, any one of the row and column pixel swaps may be performed first or later, and in the restoration process, any one of the row and column pixel swaps may be performed first or later. . It is also possible to alternately perform pixel swap of rows and columns.

  However, in the embedding process, when performing pixel swap of only the column, the method of pixel swap of the column of the embedded image data when the embedded image data is restored to the original image data depends on the restoration result of the embedded data. However, in the case where both row and column pixel swaps are performed, the pixel at the position (m, n) of the m-th row and n-th column of the embedded image data is located at which position (original image data) of the restored image (original image data) Whether the pixel swap has been performed to m ′, n ′) is a result of restoring the embedded data.

  Further, in the embedding process of FIG. 19, only the first column of the image data as the embedding target data is fixed, and in the restoration process of FIG. 20, this first column is used as a reference for restoration, so that pixel swap of the embedded image data is performed. However, the restoration criterion may not be the first column, but may be the last N-th column as long as it is set in advance by the embedding process and the restoration process, or any other arbitrary A row may be used. Furthermore, the restoration criterion does not have to be all of the pixels in one column, but may be one pixel in the extreme.

  For example, in the case where only the first column is used as a restoration criterion and the pixel swap of the embedded image data is performed using the correlation of the image data, if the pixel swap of one column is wrong, the subsequent column There is a high possibility that the pixel swap in the column (here, the column on the right side of the column in which the pixel swap is wrong) is erroneous. In this case, since the original image data cannot be restored, the embedded data cannot be restored.

  Therefore, in the embedding process, it is possible to leave a plurality of columns (for example, a plurality of columns selected at equal intervals from all the columns) as a reference for restoration (not subject to pixel swap). .

  Next, in the second method, in the embedding process, the bit plane for each bit of the bit string representing the pixel values of the plurality of pixels constituting the image data as the embedding target data is replaced in accordance with the embedded data ( Therefore, bit plane swapping (which operates in the level direction) is performed as an operation rule, and image data as embedding target data is operated according to the operation rule, so that embedded data is embedded in the image data. It is.

  That is, in the embedding process of the second method, as shown in FIG. 21, the image data as the embedding target data is blocked into image blocks composed of a plurality of pixels, and the image block is further converted into the image block. Are divided into bit planes for each bit of a bit string representing the pixel value of each pixel.

  Here, for example, assuming that 8 bits are assigned to each pixel value constituting the image data as the embedding target data, the image block includes the least significant bit of the pixel value and the second bit from the least significant bit. .., 7th bit, and most significant bit (hereinafter, appropriately referred to as first to eighth bits, respectively).

  Then, the bit plane of the image block is rearranged corresponding to the embedded data (bit-plane swapping), whereby the embedded data is embedded in the image data.

For example, as described above, when 8 bits are assigned to the pixel value constituting the image data as the embedding target data, 8 bit planes are obtained from one image block. Since there are 40320 (= 8! (! Represents factorial)) ways of rearranging the eight bit planes (how to perform bit plane swap), there is a maximum of log _{2 in} one image block. It is possible to embed embedded data of (8!) Bits (however, the fractional part is discarded). Accordingly, in the embedding process, the data to be embedded is divided for each bit number and embedded in each image block constituting the image data as the embedding target data.

  Here, in the second method, as described above, the embedded data is processed according to the operation rule that the bit plane of each image block of the image data as the embedding target data is swapped in accordance with the embedded data. Therefore, the second method is hereinafter referred to as a bit plane swap method as appropriate.

  Next, the embedding process when the embedding process by the bit plane swap method is performed in the embedding encoder 3 of FIG. 3 will be described with reference to the flowchart of FIG.

  First, in step S111, the embedding unit 22 reads embedded data for embedding in one frame image from the embedded database 2 and formats it into a unit that can be embedded in one image block ( Divide the embedded data). Further, in step S <b> 111, the embedding unit 22 stores in the frame memory 21 image data of a frame that has not yet been processed among image data frames as embedding target data stored in the embedding target database 1.

  In step S112, the embedding unit 22 divides the image of one frame stored in the frame memory 21 into, for example, image blocks of 4 × 4 pixels in the horizontal × vertical direction, and proceeds to step S113. In step S113, the embedding unit 22 divides each image block constituting the image data of one frame stored in the frame memory 21 into bit planes, and the process proceeds to step S114.

  In step S114, the embedding unit 22 sets each image block divided into bit planes not to be a target block in the raster scan order, for example, as a target block, and sets the bit plane of the target block to step 111. Bit plane swap is performed in correspondence with one unit of embedded data obtained in (1), and the image block after the bit plane swap is written to the frame memory 21 so as to be overwritten.

  Thereafter, the process proceeds to step S115, and the embedding unit 22 determines whether or not processing (processing for rearranging bit planes) has been performed on all the image blocks constituting the image of one frame stored in the frame memory 21 as a target block. If it is determined in step S115 that all the blocks of one frame have not yet been processed, the process returns to step S114, and the embedding unit 22 newly focuses on an image block that is not a target block in the raster scan order. Hereinafter, the same processing is repeated as a block.

  On the other hand, if it is determined in step S115 that all blocks of one frame have been processed as the target block, the process proceeds to step S116, where the embedding unit 22 is an image block that has been subjected to bit-plane swap corresponding to the embedded data. One frame of image data, that is, embedded image data, is read from the frame memory 21 and output, and the process ends.

  The above embedding process is repeatedly performed for each frame of image data as embedding target data stored in the embedding target database 1.

  As described above, the bit plane swap of the bit plane of each image block corresponding to the embedded data is performed, and when the embedded data is embedded in each image block, the reverse bit plane swap is performed. Thus, the original image data can be restored, and further, the embedded data can be restored based on what bit plane swap has been performed. Therefore, it is possible to embed embedded data in an image while minimizing the degradation of the image quality of the image and without increasing the amount of data.

  That is, in an image, the correlation between bits in a bit plane constituting an image block is basically higher in a bit plane of higher bits (lower in a bit plane of lower bits). Therefore, by using such image correlation, that is, here, correlation between bits constituting the bit plane, the bit plane swapped in correspondence with the embedded data can be performed without overhead. The data can be rearranged in the original order, and the embedded data can be restored by the rearrangement method. Therefore, basically, there is no deterioration in image quality due to embedding embedded data in the restored image.

  Next, the embedded image data obtained by the embedding process by the bit plane swap method is restored to the original image data and the embedded data by the restoration process by the bit plane swap method using the correlation of the image data as follows. can do.

  That is, in the restoration process using the bit plane swap method, the embedded image data for each frame is divided into the same image blocks as in the embedding process using the bit plane swap, and each image block constitutes the image block. The bit string representing the pixel value of the pixel is divided into bit planes for each bit. Then, a correlation value representing a correlation between bits in each bit plane constituting each image block is obtained.

  Specifically, for example, as shown in FIG. 23A, first, a bit plane for calculating a correlation value is extracted from the image block as a target bit plane. Now, for example, when the image block is composed of 4 × 4 pixels as described above, the target bit plane is also composed of 4 × 4 bits in the horizontal and vertical directions. For example, as shown in FIG. 23B, four small blocks having a predetermined size centered on 2 × 2 bits A, B, C, and D at the center are formed from the 4-bit target bit plane. . Here, FIG. 23B shows a small block of 3 × 3 bits configured around bit A.

  In the restoration process using the bit plane swap method, for each small block obtained from the target bit plane, the number of bits having the same value as the central bit is counted. It is calculated | required as a correlation value between these bits. Here, in the small block centered on bit A shown in FIG. 23B, the center bit (here, bit A) is 0, and its upper left, upper, upper right, right, lower right, lower, The lower left and left bits are 1, 0, 1, 0, 0, 1, 0, 0, respectively. Therefore, the same value as the central bit that is 0 is the upper, left, lower right, lower left, and left 5 bits of the central bit. The correlation value between them is 5.

  In the restoration process by the bit plane swap method, as described above, a correlation value is obtained for each small block centered on each of the bits A to D obtained from the target bit plane. Further, the correlation values for the small blocks centered on the respective bits A to D obtained in this way are added, and the added value is set as the correlation value of the target bit plane.

  Each bit plane constituting the image block is sequentially set as a target bit plane, and the correlation value is obtained as described above. When the correlation values of all the bit planes of the image block are obtained, as shown in FIG. 24, the bit plane with a high correlation value is located on the most significant bit side, and the bit plane with a low correlation value is the least significant bit. The bit planes are swapped so as to be located on the side, that is, in ascending order of correlation values. Thereby, the image block is restored. Here, in FIG. 24, a bit plane with a lower correlation value represents a bit plane with a higher correlation value.

  After an image block is restored, the sequence of bit planes that make up the restored image block (hereinafter referred to as the restored block as appropriate) is compared with the sequence of bit planes that make up the image block before restoration. Thus, when restoring the image block, it is detected how each bit plane has been bit-plane swapped, and the embedded data embedded in the restored block is restored based on the bit-plane swap method. .

  Next, with reference to the flowchart of FIG. 25, the restoration process when the restoration process by the bit plane swap method is performed in the restoration unit 6 of FIG. 4 will be described.

  In step S121, the restoration unit 32 stores the embedded image data of one frame in the frame memory 31, and proceeds to step S122. In step S122, the restoration unit 32 blocks the embedded image data of one frame stored in the frame memory 31 into an image block as in the embedding process, and proceeds to step S123. In step S123, the restoration unit 32 divides each image block into bit planes, and proceeds to step S124.

  In step S124, the restoration unit 32 sequentially sets each image block divided into bit planes as a target block, for example, in raster scan order, and the correlation value of each bit plane of the target block as described in FIG. And the process proceeds to step S125. In step S125, the restoration unit 32 performs bit-plane swap of the bit plane of the block of interest so that the correlation values of the bit planes are arranged in ascending order, thereby restoring the image blocks constituting the original image data. Further, in step S125, the restoration unit 32 writes the restored image block (restored block) in the form of overwriting the frame memory 31, and also arranges the bit planes of the restored block and the bit planes of the image block before the restoration. By comparing with the sequence of the above, it is recognized in the embedding process what bit plane swap has been performed for the restored block. Then, the restoration unit 32 restores the embedded data embedded in the restoration block based on the recognition result.

  Thereafter, the process proceeds to step S126, and the restoration unit 32 determines whether or not the processing has been performed using all image blocks of the embedded image data of one frame stored in the frame memory 31 as the target block, and all the blocks of one frame are still processed. If it is determined that processing has not been performed, the process returns to step S124, and the same processing is repeated thereafter, with the image block that has not yet been set as the target block in the raster scan order as a new target block.

  On the other hand, if it is determined in step S126 that all the image blocks of one frame stored in the frame memory 31 have been processed, the process proceeds to step S127, and the restoration unit 32 is configured by the restoration blocks stored in the frame memory 31. One frame of image data, that is, restored original image data is read out and output. Furthermore, in step S126, the restoration unit 32 outputs the embedded data restored in step S125 and ends the restoration process.

  Note that the above restoration process is repeated for each frame of embedded image data.

  As described above, bit plane swap is performed on the embedded image data in which the embedded data is embedded by performing bit plane swap of the bit plane of the image data as the embedding target data, and the bit correlation is performed using the correlation of the image. Since the planes are restored in the original image data, the embedded image data can be restored to the original image data and the embedded data without overhead. Therefore, the restored image basically does not deteriorate in image quality due to embedding the embedded data.

  Next, in the third method, in the embedding process, a line (horizontal line or vertical line or a line composed of a plurality of pixels arranged in the time direction) constituting image data as embedding target data is set as embedded data. Rotating in response to (and therefore performing a spatial or temporal operation), and operating the image data as the embedding target data according to the operation rule, Embedded data is embedded in image data.

  In other words, in the embedding process of the third method, for example, each horizontal line (a series of pixels arranged in the horizontal direction) (horizontal line) (horizontal line) of the image data as the embedding target data is equivalent to the data to be embedded. By performing line rotation shifted in the horizontal direction, embedded data is embedded in each horizontal line.

  Here, in the third method, as described above, the embedded data is embedded according to the operation rule of rotating the line of the image data as the embedded data in accordance with the embedded data. The method 3 is hereinafter referred to as a line rotation method as appropriate.

  Next, the embedding process when the embedding process by the line rotation method is performed in the embedding encoder 3 of FIG. 3 will be described with reference to the flowchart of FIG.

  From the embedding target database 1, image data as embedding target data stored therein is read out and sequentially supplied and stored in the frame memory 21.

  Then, in step S131, the embedding unit 22 reads out the horizontal lines constituting the image data stored in the frame memory 21 that have not yet been set as the target line, and proceeds to step S132. In step S132, the embedding unit 22 determines whether or not the target line is the first horizontal line (the uppermost horizontal line of the image data of one frame). If it is determined in step S132 that the target line is the first horizontal line, the process proceeds to step S133, and the embedding unit 22 writes the first horizontal line as it is in the form of overwriting the frame memory 21, and step S137. Proceed to That is, the first horizontal line is not particularly operated (line rotation), and therefore, the embedded data is not embedded.

  If it is determined in step S132 that the target line is not the first horizontal line, that is, if the target line is any horizontal line after the second horizontal line, the process proceeds to step S134, where the embedding unit 22 The embedded data to be embedded in the target line is read from the embedded database 2 and the process proceeds to step S135. In step S135, the embedding unit 22 rotates the target line in the horizontal direction by, for example, the number of pixels corresponding to the embedded data read in step S134.

  That is, for example, if the Nth horizontal line (N ≠ 1) is the target line as shown in FIG. 27A, the embedding unit 22 sets the Nth horizontal line as shown in FIG. 27B. Then, the same number of pixels as the value of the embedded data is slid (shifted) to the right or left of the horizontal direction, for example, to the right. Then, the embedded portion 22 fits the portion of the Nth horizontal line that protrudes to the right of the frame by the slide, on the left side of the Nth horizontal line, as shown in FIG. 27C.

  The embedding unit 22 rotates the attention line, and when the embedded data is embedded in the attention line, the process proceeds to step S136, and the attention line after the rotation is written in the form of overwriting the frame memory 21, step S137. Proceed to

  In step S <b> 137, the embedding unit 22 determines whether all horizontal lines of the image data stored in the frame memory 21 have been processed as a target line. If it is determined in step S137 that all the horizontal lines of the image data stored in the frame memory 21 are not yet the attention lines, the process returns to step S131 and the horizontal lines that have not yet been the attention lines (for example, until now) The horizontal line one line lower than the target line is newly set as the target line, and the same processing is repeated thereafter.

  If it is determined in step S137 that all the horizontal lines of the image data stored in the frame memory 21 are the target lines, that is, each horizontal line of the image data of one frame stored in the frame memory 21. In the case where embedded data is embedded in embedded data (excluding the first horizontal line here) and embedded image data is generated, the embedding unit 22 reads the embedded image data of one frame from the frame memory 21. Output, and the process ends.

  The above embedding process is repeated for each frame of the image data stored in the frame memory 21.

  According to the embedding process in FIG. 26, one frame of image data is set as the following embedded image data.

  That is, for example, when 10, 150, 200,... Are embedded as embedded data in the image data as shown in FIG. 28A, the first horizontal line is output as it is as shown in FIG. 28B. The second horizontal line is rotated rightward by 10 pixels having the same value as the first embedded data. Further, the third horizontal line is rotated rightward by 150 pixels having the same value as the second embedded data, and the fourth horizontal line is 200 pixels having the same value as the third embedded data. Only rotated to the right. Similarly, the fifth and subsequent horizontal lines are rotated rightward by the number of pixels corresponding to the embedded data.

  As described above, when the embedded data is embedded in each horizontal line by rotating the horizontal lines constituting the image stored in the frame memory 21 to the right by the number of pixels corresponding to the embedded data. The original image data can be restored by performing the reverse rotation (reverse rotation), and is embedded in the image data based on the rotation amount when the reverse rotation is performed. The embedded data can be restored. Therefore, it is possible to embed data to be embedded in the image data while minimizing the deterioration of the image quality of the image data and without increasing the data amount.

  That is, the horizontal line in which the embedded data is embedded is rotated from the original position by the rotation amount (number of pixels) corresponding to the embedded data. However, such a horizontal line is correlated with the image data, that is, Here, by using the correlation with the horizontal line at the original position, it is possible to return to the original position without overhead. Furthermore, by returning the horizontal line to the original position, the amount of rotation by which the horizontal line is rotated can be recognized by the embedding process, and the embedding embedded in the horizontal line is based on the rotation amount. Data can be restored. Therefore, in the restored image obtained as a result, there is basically no deterioration in image quality due to embedding data to be embedded.

  Note that when there is no horizontal line at the original position in the embedded image data, it is generally difficult to restore the image data and the embedded data using the correlation of the image as described above. It is. Therefore, here, in the embedding process of FIG. 26, the embedded data is not embedded (not rotated) in the first horizontal line of each frame, and is used as embedded image data.

  Next, the embedded image data obtained by the embedding process by the line rotation method is restored to the original image data and the embedded data by the restoration process by the line rotation method using the correlation of the image data as follows. Can do.

  That is, in the restoration process by the line rotation method, each horizontal line of the embedded image data is sequentially set as the attention line from the first horizontal line, for example, and the horizontal line on one line of the attention line is set as the reference line. Line.

  Note that the target line is the horizontal line that is about to return to the original position from now on. In this case, the horizontal line is sequentially set as the target line from the upper direction to the lower direction. Is a horizontal line that has already been returned to its original position.

  In the restoration process using the line rotation method, the correlation line represents the correlation between the target line after the rotation and the reference line while rotating the target line pixel by pixel in either the right or left direction. The value is calculated.

  Specifically, for example, as shown in FIG. 29A, without rotating the target line (rotating the target line by 0 pixels), the pixel values of the pixels constituting the target line and the reference line are set. The reciprocal of the sum of the absolute differences from the pixel values of the corresponding pixels to be configured is calculated as a correlation value between the reference line and the reference line. Further, as shown in FIG. 29B, the target line is rotated to one of the right and left by one pixel (in FIG. 29, the target line is rotated to the left, which is the reverse direction in the case described in FIG. 27). ) For the attention line after the rotation, a correlation value with the reference line is calculated. In the same manner, the attention line is sequentially rotated between 2 pixels, 3 pixels,... Until the attention line returns to the original position (original position in the embedded image data). A correlation value is calculated.

  As described above, after the correlation value for the attention line rotated by each rotation amount (here, the number of pixels constituting the horizontal line) is calculated, the correlation for the attention line is selected from each rotation amount. The rotation amount that maximizes the value is detected. Then, the detected rotation amount is determined as a rotation amount for returning the target line to the original position (this determined rotation amount is hereinafter referred to as a determined rotation amount as appropriate), and the target line is determined by the determined rotation amount. Rotating in the same direction as the case of calculating the correlation value (in the above case, the left direction) returns to the original position. That is, the attention line is restored by this.

  Further, the embedded data embedded in the target line is restored based on the determined rotation amount.

  Next, with reference to the flowchart of FIG. 30, the restoration process when the restoration process by the line rotation method is performed in the restoration unit 6 of FIG. 4 will be described.

  In the frame memory 31, the embedded image data supplied thereto is sequentially stored, for example, in units of one frame.

  Then, in step S141, the restoration unit 32 reads out the horizontal line in the upper row of the horizontal lines that are not yet set as the target line of the embedded image data stored in the frame memory 31 as the target line, Proceed to step S142. In step S142, the restoration unit 32 determines whether the target line is the first horizontal line. When it is determined in step S142 that the target line is the first horizontal line, the process proceeds to step S143, and the restoration unit 32 overwrites the frame memory 31 with the target line that is the first horizontal line as it is. Write, proceed to step S150. That is, in the embedding process of FIG. 28, as described above, since the embedded data is not embedded in the first horizontal line (and thus is not rotated), the first horizontal line is not particularly operated.

  If it is determined in step S142 that the target line is not the first horizontal line, that is, if the target line is any horizontal line after the second horizontal line, the process proceeds to step S144, and the restoring unit 32 Reads from the frame memory 31 using the horizontal line one line above the target line as a reference line, and proceeds to step S145. In step S145, the restoration unit 32 calculates a correlation value (line correlation value) between the target line and the reference line, stores the correlation value in the built-in memory, and proceeds to step S146.

  In step S146, the restoration unit 32 rotates the target line by one pixel, for example, in the left direction, and proceeds to step S147. In step S147, the restoration unit 32 determines whether each pixel of the target line after the rotation has returned to the position of each pixel of the target line stored in the frame memory 31 by rotating the target line in step S146. Determine if.

  When it is determined in step S147 that each pixel of the attention line after rotation has not returned to the position of each pixel of the attention line stored in the frame memory 31, the process returns to step S145, and A correlation value with the reference line is calculated, and thereafter the same processing is repeated.

  If it is determined in step S147 that each pixel of the target line after rotation has returned to the position of each pixel of the target line stored in the frame memory 31, the process proceeds to step S148, and the restoration unit 32 For the line, the maximum value is obtained from the correlation values for the respective rotation amounts obtained by performing the loop processing of steps S145 to S147. Further, in step S148, the restoration unit 32 detects a rotation amount that gives the maximum correlation value, and determines the rotation amount as the determined rotation amount. Then, the restoration unit 32 restores and outputs the embedded data embedded in the target line based on the determined rotation amount, that is, outputs the determined rotation amount as embedded data as it is, and proceeds to step S149. move on.

  In step S149, the restoration unit 32 reads the attention line stored in the frame memory 31, and restores the attention line by rotating leftward by the determined rotation amount (the attention line is positioned in the original image data). Back to). Further, in step S149, the restoration unit 32 writes the noticed line after rotation in the form of overwriting the frame memory 31, and proceeds to step S150.

  In step S150, the restoration unit 32 determines whether or not all horizontal lines of the embedded image data stored in the frame memory 31 have been processed as a target line. If it is determined in step S150 that all the horizontal lines of the embedded image data stored in the frame memory 31 have not yet been set as the target line, the process returns to step S141, and is one line lower than the target line until now. The horizontal line is newly set as the attention line, and the same processing is repeated thereafter.

  If it is determined in step S150 that all the horizontal lines of the embedded image data stored in the frame memory 31 are the target lines, that is, each frame of embedded image data of one frame stored in the frame memory 31 is determined. When the embedded data embedded in the horizontal line (here, excluding the first horizontal line) is restored, and the embedded image data is restored to the original image data, the restoration unit 32 performs the restoration. The processed image data is read out from the frame memory 31 and output, and the process ends.

  Note that the above restoration process is repeated for each frame of the embedded image data stored in the frame memory 31.

  As described above, for the embedded image data in which the embedded data is embedded by rotating the horizontal line, the horizontal line is rotated, and the horizontal line is returned to the original position by utilizing the correlation of the image. Therefore, the embedded image data can be restored to the original image data and the embedded data without overhead. Therefore, the restored image basically does not deteriorate in image quality due to embedding the embedded data.

  In the restoration process of FIG. 30, the sum of absolute differences between the corresponding pixels (sum of absolute differences) is used as the correlation value representing the correlation between the target line and the reference line. For example, the sum of squares of pixel differences can be used.

  Also, in the embedding process of FIG. 26, one horizontal line is rotated according to the embedding data, so that one horizontal line is embedded with a value within the range of the number of pixels constituting the horizontal line. Data can be embedded. However, even when embedding of embedded data having a value larger than the number of pixels constituting one horizontal line is performed, for example, a plurality of horizontal lines such as two horizontal lines are rotated according to the embedded data. This can be done.

  By the way, for example, in the above-described embedding process by the line rotation method of FIG. 26, the horizontal line composed of pixels arranged in the horizontal direction of the image data as the embedding target data is rotated in the horizontal direction, thereby Although embedding is performed, embedding of embedded data is performed by, for example, rotating a vertical line composed of pixels lined up in the vertical direction of image data in the vertical direction or pixels lined up in an oblique direction. It is also possible to perform such a process by rotating the configured line in the oblique direction.

  In addition, in the case of the line rotation method, by embedding embedded data by rotating either one of the horizontal line or the vertical line, and then rotating the other of the horizontal line or the vertical line, The embedded data can be further embedded.

That is, for example, as shown in FIG. 31A, in the embedding process by the line rotation method, the horizontal line of the image data as the embedding target data is rotated corresponding to the predetermined embedding data D ₁ , so that the embedding is performed. the data D ₁ embedding further, by the results vertical line of the image data as coded data C ₁ embedding is generated, to rotate in response to another of the embedded data D _2, the object to be embedded data D ₂ Can be embedded.

Now, suppose that the embedded encoded data generated by embedding the embedded data D ₂ by rotating the vertical line of the image data as the embedded encoded data C ₁ is described as embedded encoded data C _2. In the embedded encoded data C ₂ , the embedded data D ₁ and D ₂ are embedded by rotating not only the horizontal line but also the vertical line of the original image data. As described above, more embedded data can be embedded as compared with the case where only the horizontal line of the image data is rotated.

That is, for example, when the number of horizontal lines and vertical lines of image data as embedding target data is the same, embedded coding generated by rotating both the horizontal and vertical lines of image data. The data C ₂ can embed embedded data having a data amount twice as large as that of the embedded encoded data C ₁ generated by rotating only the horizontal line of the image data.

The embedded coded data C ₂ as described above, restoration process of restoring the target data and to the embedded data D ₁ and D ₂ embedded is again by using the correlation of the image, can be performed as follows .

  That is, when the horizontal line of the image data as the embedding target data is rotated, the correlation in the vertical direction that is the spatial direction orthogonal to the horizontal line in the image data is destroyed. Therefore, with respect to the embedded encoded data generated by rotating only the horizontal line of the image data, the horizontal line is restored to the original correlation (in the restoration process of FIG. Can be restored to the original image data.

  The embedded encoded data generated by rotating only the vertical lines of the image data is also changed to the original image data in the same manner as in the above-described embedded encoded data generated by rotating the horizontal lines. Can be restored.

  That is, when the vertical line of the image data as the embedding target data is rotated, the correlation in the horizontal direction, which is the spatial direction orthogonal to the vertical line, in the image data is destroyed. Therefore, the embedded encoded data generated by rotating the vertical line of the image data is restored to the original image data by rotating the vertical line so that the correlation in the horizontal direction is restored. be able to.

  From the above, when the horizontal line of the image data is rotated, the correlation in the vertical direction is destroyed, but the correlation in the horizontal direction is not affected. Further, when the vertical line of the image data is rotated, the horizontal correlation is destroyed, but the vertical correlation is not affected. The embedded encoded data generated by rotating the horizontal line of the image data can be restored to the original image data by using only the vertical correlation, and the vertical line of the image data is rotated. The embedded encoded data generated by doing so can be restored to the original image data using only the correlation in the horizontal direction.

Accordingly, as shown in FIG. 31A, the horizontal lines of the image data as an implanted target data, by rotation in response to the embedded data D _1, to generate embedded coded data C _1, the embedded coding When the embedded encoded data C ₂ is generated by rotating the vertical line of the image data as the data C ₁ corresponding to the embedded data D ₂ , the embedded encoded data C ₂ is shown in FIG. 31B. As shown in FIG. 4, the vertical line can be restored to the embedded encoded data C ₁ and the embedded data D ₂ by rotating the vertical line so as to restore the correlation in the horizontal direction. The embedded encoded data C ₁ can be restored to the embedding target data and the embedded data D ₁ by rotating the horizontal line so as to restore the correlation in the vertical direction. That is, the embedded encoded data C ₂ can be restored to the original embedding target data and the embedded data D ₁ and D ₂ .

The horizontal line rotation embedding and the vertical line rotation embedding are both embedding by a line rotation method in which data to be embedded is operated according to an operation rule of rotating a line corresponding to embedded data. However, the horizontal line rotation affects only the vertical correlation, and the vertical line rotation affects only the horizontal correlation, so the horizontal line rotation and the vertical line rotation are the same. In other words, it can be said that it is an independent operation or an orthogonal operation. Thus, the horizontal line rotation, and a vertical line rotation, because it is an independent operation, a horizontal line of image data corresponding to the embedded data D ₁ and rotation, further, burying the vertical line to be Embedded encoded data C ₂ obtained by rotation corresponding to data D ₂ can be restored to the original image data and embedded data D ₁ and D ₂ .

By the way, since the above-mentioned embedded coded data C ₂ is obtained by rotating the horizontal line and the vertical line of the image data as the embedding target data, the so-called spatial correlation of the image data is in all spatial directions. Is about to be destroyed.

Therefore, the image data as the embedded encoded data C ₂ is operated according to other operation rules as well as the line rotation method, for example, by the above-described pixel swap method or bit plane swap method. Doing embedding of the embedded data, the restoring processing according to the corresponding method, even embedded coded data C _2, it becomes difficult to recover the embedded data embedded in the embedded coded data C _2.

That is, the image data of the embedded coded data C _2, by line rotation scheme if further embedding the other of the embedded data D _3, the object to be embedded data D ₃ is the embedded coded data C ₂ horizontal lines or vertical lines, is embedded by rotation in response to the embedded data D _3.

However, the horizontal line or vertical line rotation corresponding to the embedded data D ₃ is a horizontal line rotation for embedding the embedded data D ₁ or a vertical line for embedding the embedded data D _2. since an operation of rotation and overlapping, horizontal line rotation is not able to distinguish whether it is due to the embedding of any of of the embedded data D ₁ or D _3, or vertical line rotation is Therefore, it cannot be distinguished which of the embedded data D ₂ or D ₃ is embedded, and therefore neither the embedded encoded data C ₂ nor the embedded data D ₃ can be restored. .

  In addition, in the decoding process using the pixel swap method or the bit plane swap method, the correlation between the spatially adjacent pixels is used, and the original image data and the embedded data embedded therein are used. Restoration is performed.

However, with respect to the image data as the embedded encoded data C ₂ , the correlation in the spatial direction is broken in all directions, so that such embedded encoded data C ₂ includes a pixel swap method, a bit the plane swap scheme to embed the embedded data D ₃ is no deviation of energy, for example, as target data embedding noise, and the pixel swapping scheme, the bit-plane swapping scheme, and embedding the target embedded data D ₃ Is equivalent.

  Therefore, with respect to new embedded encoded data obtained by such embedding, the spatial correlation of image data (image data having information value) cannot be used. The original embedding target data and embedded data cannot be restored by the restoration process using the bit plane swap method.

From the above, the embedded data D ₃ is embedded by the pixel swap method or the bit plane swap method with respect to the embedded encoded data C ₂ whose spatial direction correlation is broken in all directions, and a new embedded code is obtained. When the encoded data is generated, the new embedded encoded data is embedded with the embedded encoded data C ₂ by using the correlation of the image data as the embedded encoded data C _2. The embedded data D ₃ that has been stored cannot be restored.

By the way, when the embedded data D ₃ is embedded by the same line rotation method with respect to the embedded encoded data C ₂ generated by the embedding process by the line rotation method, and new embedded encoded data is generated. As described above, the operation performed when generating the embedded encoded data C ₂ and the operation performed when generating new embedded encoded data may overlap. Even if the image data as the embedding target data can be restored, it is difficult to restore all the embedded data D _{1 to} D ₃ embedded in the embedding target data.

On the other hand, when the embedded data D ₃ is embedded in the embedded encoded data C ₂ generated by the embedding process by the line rotation method, for example, by a bit plane swap method or the like different from the line rotation method. Is the same as the operation performed when generating the embedded encoded data C ₂ (here, the rotation of the horizontal line and the rotation of the vertical line) and the operation performed when generating the new embedded encoded data. There is nothing.

Therefore, in this case, if the image data as the original embedding target data can be restored, all of the embedded data D _{1 to} D ₃ embedded in the embedding target data can also be restored.

That is, if the original embedding target data can be restored from the new embedded encoded data, an operation for generating new embedded encoded data from the embedded target data is specified. Furthermore, the operation of generating new embedded encoded data from the embedding target data includes an operation of embedding the data to be embedded D ₁ and D ₂ by the line rotation method, and the embedded encoding obtained by the operation For data C ₂ , the operation includes embedding embedded data D ₃ by a bit plane swap method or the like, which is different from the line rotation method. In this case, the operation by the line rotation method for embedding the embedded data D ₁ and D ₂ and the operation by the bit plane swap method for embedding the embedded data D ₃ are orthogonal operations as described above. There is nothing to do.

Therefore, if an operation for generating embedded encoded data in which embedded data D _{1 to} D ₃ are embedded is specified from the embedded data, line rotation for embedding embedded data D ₁ and D ₂ is performed based on the operation. operation by method also, none of the operation of the operation by the bit-plane swapping scheme for embedding the embedding data D ₃ can be identified.

  Therefore, FIG. 32 shows a configuration example of the embedded encoder 3 of FIG. 2 that performs embedding by a plurality of methods such as a line rotation method and a bit plane swap method. In the figure, portions corresponding to those in FIG. 3 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, the embedded encoder 3 in FIG. 32 is basically configured in the same manner as in FIG. 3 except that a feature extraction unit 23 and an integration unit 24 are newly provided.

  The feature extraction unit 23 is supplied with one frame of image data as data to be embedded that is the same as that supplied to the frame memory 21. The feature extraction unit 23 extracts feature data representing the feature from one frame of image data as the embedding target data supplied thereto, and supplies the feature data to the integration unit 24.

  Here, the feature extraction unit 23 extracts, for example, the following feature data from one frame of image data as the embedding target data.

  That is, the feature extraction unit 23 extracts pixel values of pixels at one or more specific positions in the image data as feature data. Further, the feature extraction unit 23 calculates a difference value between pixel values at one or more specific positions of the image data and pixels adjacent to the upper, lower, left, or right thereof, and calculates the difference value as Extract as feature data. Further, the feature extraction unit 23 divides the image data into blocks of a predetermined size, obtains the activity and dynamic range of each block, and extracts the activity and dynamic range for each block as feature data. Here, as the activity of the block, for example, the sum of absolute differences (or squares of differences) of the pixel values of each pixel constituting the block and the pixels adjacent to the pixel, or the pixel constituting the block A DCT coefficient obtained by DCT conversion of a pixel value can be used that represents the degree of change in pixel value in a block, such as the sum of absolute values of AC (Alternating Current) components. As the dynamic range of the block, for example, the difference between the maximum value and the minimum value of the pixel values of the pixels constituting the block can be employed.

  In addition, the feature extraction unit 23 can extract an average value of pixel values of image data constituting one frame, a pixel value (mode value) having the highest variance and frequency, and the like as feature data. .

  Note that the feature data of the image data extracted by the feature extraction unit 23 is not limited to that described above. That is, as the feature data, in addition to the above-described ones, for example, other than the above-described ones can be used as long as they represent the features of the image data such as the contour and the center of gravity of the object displayed in the image data. Is possible. Also, the feature extraction unit 23 can extract two or more types of feature data.

  The integration unit 24 receives the feature data from the feature extraction unit 23, and stores the image data stored in the frame memory 21, that is, the image data from which the feature data is extracted (in addition to the image data itself as the embedding target data, Image data as embedded encoded data obtained by embedding embedded data in image data) and feature data are integrated.

  Here, the method of integrating the image data and the feature data in the integration unit 24 is not particularly limited. That is, the image data and the feature data can be integrated by simply adding the feature data or the image data to the image data or the feature data, or by time division multiplexing, frequency division multiplexing, or frequency spreading using different spreading codes. It is also possible to integrate by such as. Furthermore, the image data and the feature data can be integrated by embedding the feature data in the image data.

  Next, the embedding process by the embedded encoder 3 in FIG. 32 will be described with reference to the flowchart in FIG.

  From the embedding target database 1, image data as embedding target data stored therein is read in units of frames and supplied to the frame memory 21 and the feature extraction unit 23. The frame memory 21 stores one frame of image data as the embedding target data supplied from the embedding target database 1.

  The feature extraction unit 23 receives one frame of image data as the embedding target data supplied from the embedding target database 1. In step S 161, the feature extraction unit 23 extracts feature data from the image data and supplies the feature data to the integration unit 24. The process proceeds to step S162.

  In step S162, the integration unit 24 integrates the feature data from the feature extraction unit 23 and the image data as the embedding target data stored in the frame memory 21, and the embedding unit 22 receives the target data from the embedded database 2. The embedded data is read out and embedded in the image data as the embedding target data stored in the frame memory 21. Thereby, the image data as the embedding target data stored in the frame memory 21 is embedded encoded data in which the feature data is integrated, and in step S162, the embedded encoded data in which the feature data is further integrated. The data is read from the frame memory 21 and the process is terminated.

  Note that the processing in FIG. 33 is repeated every time one frame of image data as data to be embedded is newly stored in the frame memory 21.

  Next, FIG. 34 shows the embedded encoded data integrated with the feature data output from the embedded encoder 3 (the frame memory 21) of FIG. 3 shows a configuration example of the restorer 6 in FIG. 2 that restores data. In the figure, portions corresponding to those in FIG. 4 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, the restorer 6 in FIG. 34 is basically configured in the same manner as in FIG. 4 except that a feature separation unit 33, a feature extraction unit 34, and a determination control unit 35 are newly provided. .

  The feature separation unit 33 separates the feature data from the embedded encoded data stored in the frame memory 31 and supplies the feature data to the determination control unit 35. That is, embedded encoded data in which feature data is integrated is supplied to the frame memory 31, and the frame memory 31 temporarily stores embedded encoded data in which the feature data is integrated. The unit 33 separates the feature data from the embedded encoded data integrated with the feature data stored in the frame memory 31 and supplies the feature data to the determination control unit 35.

  Here, the feature data separated from the embedded encoded data in which the feature data is integrated represents the feature of the image data extracted from the original image data (original image) as the embedding target data. In order to distinguish from temporary restoration feature data described later, hereinafter, it will be referred to as original feature data as appropriate.

  The feature extraction unit 34 extracts the feature data from the image data as the embedding target data temporarily restored by the restoration unit 32 (hereinafter referred to as temporary restoration image data as appropriate) in the same manner as in the feature extraction unit 23 of FIG. The extracted feature data is supplied to the determination control unit 35 as temporary restoration feature data.

  That is, in the embodiment of FIG. 34, the restoration unit 32 embeds the embedded encoded data stored in the frame memory 32 when the embedded encoder 3 embeds the embedded data in the embedded target data. By operating according to the operation rule corresponding to the above operation, image data (temporary restoration image data) as temporary embedding target data and temporary embedding data are restored. Then, the feature extraction unit 34 extracts the feature data from the temporarily restored image data obtained by the restoration unit 32, and supplies the feature data to the determination control unit 35 as temporary restoration feature data.

  The determination control unit 35 determines the coincidence between the original feature data supplied from the feature separation unit 33 and the temporary restoration feature data supplied from the feature extraction unit 34, and the restoration unit 32 restores based on the determination result. Control processing.

  In other words, if the temporarily restored image data matches the image data as the original embedding target data (if correctly restored), the temporarily restored feature data that is the feature data of the temporarily restored image data is Should match the feature data. Therefore, if the original feature data and the temporary restoration feature data match, it can be said that the temporary restoration image data is accurately restored to the image data as the original embedding target data.

  Therefore, the determination control unit 35 determines whether or not the original feature data and the temporary restoration feature data match, and when they do not match, controls the restoration processing in the restoration unit 32. Accordingly, the restoration unit 32 corresponds to the operation of the embedding target data performed when the embedded encoder 3 embeds the embedded data in the embedding target data for the embedded encoded data stored in the frame memory 32. By performing another operation according to the operation rule, the temporary restored image data and the temporary embedded data are newly restored.

  On the other hand, if the original feature data and the temporarily restored feature data match, the determination control unit 35 uses the temporarily restored image data from which the temporarily restored feature data is obtained as image data as the original embedding target data. In addition, the restoration unit 32 is controlled so that the temporarily restored image data is overwritten in the frame memory 31 on the assumption that the restoration has been correctly performed.

  Next, with reference to the flowchart of FIG. 35, the restoration process by the decompressor 6 of FIG. 34 will be described.

  The embedded encoded data in which the feature data (original feature data) is integrated is supplied to the frame memory 31 and temporarily stored.

  In step S 171, the feature separation unit 33 separates the original feature data from the embedded encoded data stored in the frame memory 31 and supplies the original feature data to the determination control unit 35.

  Further, in step S171, the restoration unit 32 performs a predetermined operation according to the operation rule of the embedding process in the embedding unit 22 in FIG. 32 on the embedded encoded data stored in the frame memory 31, thereby Temporary restoration image data as temporary embedding target data and temporary embedding data are restored.

  Thereafter, the process proceeds to step S172, where the feature extraction unit 34 extracts feature data from the temporarily restored image data obtained by the restoration unit 32, supplies the feature data to the determination control unit 35 as temporary restoration feature data, and proceeds to step S173. .

  In step S173, the determination control unit 35 determines whether the original feature data matches the temporary restoration feature data. If it is determined in step S173 that the original feature data and the temporary restoration feature data do not match, the process proceeds to step S174, and the decision control unit 35 performs another operation on the embedded encoded data. 32, and proceeds to step S175.

  In step S175, the restoration unit 32 follows the operation rules of the embedding process in the embedded encoder 3 in FIG. 32 with respect to the embedded encoded data stored in the frame memory 32 according to the control from the determination control unit 35. By performing other operations, the temporary restored image data and the temporary embedded data are newly restored. Then, the process returns to step S172, and the processes in steps S173 to S175 are repeated until it is determined in step S173 that the original feature data and the temporary restoration feature data match.

  If it is determined in step S173 that the original feature data and the temporary restoration feature data match, the determination control unit 35 controls the restoration unit 32 to overwrite the temporary restoration image data in the frame memory 31. The process proceeds to step S176. In step S176, the restoration unit 32 writes the temporary restoration image data in the form of overwriting the frame memory 31, and outputs the temporary embedded data restored together with the temporary restoration image data as an accurate restoration result. The restoration process ends.

  Note that the processing in FIG. 35 is repeated every time one frame of image data as embedded encoded data is newly stored in the frame memory 31.

  Next, as an integration method of feature data (original feature data), for example, an embedding process in the embedding encoder 3 and a restoring process in the decompressor 6 when embedding is employed will be described.

  In this case, as shown in FIG. 36A, the embedded encoder 3 extracts feature data from the image data as the embedding target data, and the feature data and the embedded data are embedded in the image data as the embedding target data. . In other words, the embedded encoder 3 performs an embedding process for manipulating the image data as the embedding target data in accordance with the feature data and the embedded data, and generates embedded encoded data.

  On the other hand, in the decompressor 6, as shown in FIG. 36B, the embedded encoded data is operated according to the same operation rule as that of the embedding process in the embedded encoder 3, and the image data as the embedding target data, The feature data and the embedded data embedded in the image data are restored.

  More specifically, in the embedded encoder 3, for example, as shown in FIG. 37A, feature data is extracted from the image data as the embedding target data, and the image data as the embedding target data is converted into, for example, a line rotation method. Thus, an embedding process is performed to operate corresponding to the feature data. Thus, the feature data is embedded in the image data as the embedding target data, and embedded encoded data is generated. Here, the embedded encoded data obtained by performing the initial embedding process on the embedding target data is hereinafter referred to as first embedded encoded data as appropriate.

  The embedded encoder 3 further performs an operation orthogonal to the line rotation method on the image data as the first embedded encoded data obtained by embedding the feature data in the embedding target data. For example, bit plane swap According to the method, an embedding process is performed to operate corresponding to the embedded data. Thereby, the embedded data is embedded in the image data as the first embedded encoded data (embedding target data in which the feature data is embedded), and embedded encoded data is generated. Here, the embedded encoded data obtained by performing the embedding process on the first embedded encoded data, that is, the image data as the embedding target data is different from the first embedding process. The embedded encoded data obtained by performing the second embedding process is hereinafter referred to as second embedded encoded data as appropriate.

  On the other hand, in the decompressor 6, as shown in FIG. 37B, the second embedded coded data is operated as a restoration process by the same bit plane swap method as the second embedded process in the embedded encoder 3. Thus, the temporary first embedded encoded data and the temporary embedded data are restored.

  Here, in the first restoration process performed by the decompressor 6, by operating all patterns, the provisional first embedded encoded data and provisional embedded data for each operation pattern are restored. The That is, when the first restoration process is performed by the bit plane swap method as described above, the restoration unit 6 sequentially generates all the rearranged bit planes of the image block, The image data composed of the arranged bit planes is sequentially output as temporary first embedded encoded data.

  When the decompressor 6 obtains temporary first embedded encoded data by a certain operation by the bit plane swap method, the first embedded pad in the embedded encoder 3 is obtained for the temporary first embedded encoded data. An operation as a restoration process by the same line rotation method as that of the process is performed, and thereby, provisional embedding target data (temporary restoration image data) and provisional feature data (original feature data) are restored.

  Here, in the second restoration process performed by the restoration unit 6, unlike the first restoration process, only the most probable restoration result among the restoration results obtained by performing the operation of all patterns is output. The That is, when the second restoration process is performed by the line rotation method as described above, the restorer 6 temporarily converts the image data obtained by rotating each line to the position where the correlation is the largest. Is output as data to be embedded.

  Then, the decompressor 6 extracts feature data (temporary restoration feature data) from the provisional embedding target data and compares it with the provisional original feature data, thereby matching the provisional original feature data with the provisional restoration feature data. Determine sex.

  When the temporary original feature data and the temporary restored feature data do not match, the decompressor 6 has restored the temporary first embedded encoded data to the original first embedded encoded data accurately. If not, the second embedded coded data is subjected to another pattern operation as a restoration process (first restoration process) by the bit plane swap method, thereby newly providing the temporary first embedded code. Data and temporary embedded data are restored.

  Further, the restoring unit 6 performs a restoration process (second restoration process) on the new temporary first embedded coded data by the same line rotation method as the first embedding process in the embedded encoder 3. By performing an operation, the temporary embedding target data (temporary restored image data) and the temporary feature data (original feature data) are restored.

  Then, the decompressor 6 extracts feature data (temporary restoration feature data) from the provisional embedding target data and compares it with the provisional original feature data, thereby matching the provisional original feature data with the provisional restoration feature data. Determine sex.

  If the provisional original feature data and the provisional restoration feature data do not match, the restoration unit 6 repeats the same process as described above.

  On the other hand, if the provisional original feature data and the provisional restoration feature data match, the restoration unit 6 uses the provisional embedding target data obtained at that time and the provisional embedded data as an accurate restoration result. And the process ends.

  Next, FIG. 38 shows a configuration example of the embedded encoder 3 in FIG. 2 that performs the embedding process described in FIG. 37A.

  The image data as the embedding target data stored in the embedding target database 1 (FIG. 2) is supplied to the feature extraction unit 41 and the embedder 42 in units of frames, for example, and the embedded database 2 ( The embedded data stored in FIG. 2) is supplied to the embedder 43.

  Similar to the feature extraction unit 23 described with reference to FIG. 32, the feature extraction unit 41 extracts feature data from the image data for each frame as the embedding target data supplied thereto and supplies the feature data to the embedder 42.

  The embedder 42 operates one frame of image data as embedding target data supplied from the embedding target database 1 (FIG. 2) in accordance with the line rotation method corresponding to the feature data supplied from the feature extraction unit 41. Thus, the first embedded encoded data in which the feature data is embedded in the image data as the embedding target data is generated and supplied to the embedder 43.

  Here, when the embedding unit 2 performs the embedding process by the line rotation method as described above, the feature extraction unit 41 does not extract the feature data for each frame of image data as the embedding target data. Instead, it is possible to extract feature data for each horizontal line or vertical line that is a unit operated by the line rotation method.

  The embedder 43 operates one frame of image data as the first embedded encoded data supplied from the embedder 42 in accordance with the embedded data supplied from the embedded database 2 by the bit plane swap method. Thus, the second embedded encoded data in which the embedded data is embedded in the image data is generated and output as the first embedded encoded data.

  Next, the embedding process by the embedded encoder 3 in FIG. 38 will be described with reference to the flowchart in FIG.

  First, in step S181, one frame of image data as embedding target data stored in the embedding target database 1 (FIG. 2) is supplied to the feature extraction unit 41 and the embedder 42, and the embedded database 2 The embedded data stored in (FIG. 2) is supplied to the embedder 43, and the process proceeds to step S182.

  In step S182, the feature extraction unit 41 extracts feature data from one frame of image data as embedding target data supplied thereto, supplies the feature data to the embedder 42, and proceeds to step S183.

  In step S183, the embedder 42 performs line rotation on one frame of image data as the embedding target data supplied from the embedding target database 1 (FIG. 2) corresponding to the feature data supplied from the feature extraction unit 41. By operating according to the method, the feature data is embedded in the image data as the embedding target data. The first embedded encoded data obtained by this embedding is supplied from the embedders 42 to 43, and the process proceeds to step S184.

  In step S184, the embedder 43 converts one frame of image data as the first embedded encoded data supplied from the embedder 42 into a bit plane corresponding to the embedded data supplied from the embedded database 2. The operation is performed according to the swap method, whereby the embedded data is embedded in the image data as the first embedded encoded data. The second embedded encoded data obtained by this embedding is output as final embedded data, and the process ends.

  Note that the processing in FIG. 39 is repeated for each frame of image data as embedding target data stored in the embedding target database 1.

  Next, FIG. 40 shows a configuration example of the restorer 6 of FIG. 2 that performs the restoration process described in FIG. 37B.

  Image data as second embedded encoded data obtained by the embedded encoder 6 as described with reference to FIGS. 38 and 39 is supplied to the restoration unit 51.

  The restoration unit 51 operates the second embedded encoded data supplied thereto by the same bit-plane swap method as that in the embedder 43 in FIG. 38, and thereby the provisional first embedded encoded data Then, the temporary embedded data is restored. Further, the restoration unit 51 supplies the temporary first embedded encoded data to the restoration unit 52 and also supplies the temporary embedded data to the embedded data buffer 55.

  In addition, the restoration unit 51 is information (hereinafter, how the bit plane is replaced) indicating how to operate (operation pattern) when the second embedded encoded data is operated by the bit plane swap method. The operation information is generated as appropriate, and the operation information is supplied to the operation information buffer 58.

  Note that the restoration unit 51 switches the operation information stored in the operation information buffer 58 to the switch 59 in accordance with the control of the determination control unit 56, which will be described later, for the operation of the second embedded encoded data by the bit plane swap method. While referring through.

  That is, when receiving the second embedded encoded data, the restoration unit 51 first performs a predetermined operation by the bit plane swap method on the second embedded encoded data, thereby obtaining a bit plane. Are rearranged. Then, the restoration unit 51 supplies operation information (here, for example, how to rearrange bit planes) corresponding to the operation to the operation information buffer 58 to be stored.

  After that, when receiving a reprocessing instruction signal indicating a request for another restoration process from the determination control unit 56, the restoration unit 51 refers to the operation information stored in the operation information buffer 58 via the switch 59. Thus, the operation by the bit plane swap method performed on the second embedded encoded data so far is recognized, and the operation by the bit plane swap method other than the recognized operation is performed on the second embedded encoded data. . As a result, the restoration unit 51 newly restores temporary first embedded encoded data different from the temporary first embedded encoded data restored so far.

  The restoration unit 52 operates the temporary first embedded encoded data supplied from the restoration unit 51 by the same line rotation method as that in the embedder 42 in FIG. The temporary original feature data is restored. Further, the restoration unit 52 supplies the temporary embedding target data to the embedding target data buffer 53 and the feature extraction unit 54, and supplies the temporary original feature data to the determination control unit 56.

  The embedding target data buffer 53 stores the temporary restoration image data as the temporary embedding target data supplied from the restoration unit 52 in the form of overwriting sequentially.

  The feature extraction unit 54 is configured in the same manner as the feature extraction unit 23 of FIG. 32, extracts feature data from temporary restoration image data as temporary embedding target data supplied from the restoration unit 52, and serves as temporary restoration feature data. And supplied to the determination control unit 56.

  The embedded data buffer 55 stores the embedded data supplied from the restoration unit 51 in the form of sequentially overwriting.

  Similar to the determination control unit 35 in FIG. 34, the determination control unit 56 determines the coincidence between the temporary original feature data supplied from the restoration unit 52 and the temporary restoration feature data supplied from the feature extraction unit 54, Based on the determination result, the restoration process and the like in the restoration unit 51 are controlled.

  That is, the determination control unit 56 controls the restoration unit 51, the switch 57, and the switches 59 to 61 based on the coincidence between the temporary original feature data and the temporary restoration feature data.

  The switch 57 selects the terminal 57A or 57B according to the control of the determination control unit 56. The operation information buffer 58 temporarily stores the operation information supplied from the restoration unit 51.

  The switches 59, 60, 61 are turned on / off under the control of the determination control unit 56. When the switch 59 is turned on, the restoration unit 51 can refer to the operation information stored in the operation information buffer 58. Further, when the switch 60 is turned on, the embedding target data stored in the embedding target data buffer 53 can be output. Further, when the switch 61 is turned on, the embedded data stored in the embedded data buffer 55 can be output.

  Here, the switches 59 to 61 are turned off by default unless otherwise controlled by the determination control unit 56.

  Next, with reference to the flowchart of FIG. 41, the restoring process by the restoring unit 6 of FIG. 40 will be described.

  First, in step S191, one frame of image data as the second embedded encoded data is supplied to the restoration unit 51, and the restoration unit 51 receives the second embedded encoded data. Further, in step S191, the embedding target data buffer 53, the embedded data buffer 55, and the operation information buffer 58 clear the stored contents, and the process proceeds to step S192.

  In step S192, the restoration unit 51 operates the second embedded encoded data supplied thereto by the same bit-plane swap method as that in the embedder 43 in FIG. Temporary first embedded encoded data, which is image data in which planes are arranged, and temporary embedded data are restored. Then, the restoration unit 51 supplies the provisional first embedded encoded data to the restoration unit 52 and also supplies the provisional embedded data to the embedded data buffer 55 to be stored in an overwritten form. Further, in step S192, the restoration unit 51 generates operation information indicating how to operate the second embedded encoded data when the second embedded encoded data is operated by the bit-plane swap method, and the operation information is used for the operation information. The data is supplied to and stored in the buffer 58, and the process proceeds to step S193.

  In step S193, the restoration unit 52 operates the temporary first embedded encoded data supplied from the restoration unit 51 by the same line rotation method as that in the embedder 42 in FIG. Then, the temporary embedding target data obtained by rotating the vertical line to the position where the correlation is highest and the original original feature data are restored. Then, the restoration unit 52 supplies the temporary embedding target data to the embedding target data buffer 53 and stores it in an overwritten form. Further, the restoration unit 52 supplies the temporary embedding target data to the feature extraction unit 54 and also supplies the temporary original feature data to the determination control unit 56, and the process proceeds to step S194.

  In step S194, the feature extraction unit 54 extracts feature data from the temporary restoration image data as temporary embedding target data supplied from the restoration unit 52, and supplies the extracted feature data to the determination control unit 56 as temporary restoration feature data. The process proceeds to step S195.

  In step S195, the determination control unit 56 compares the temporary original feature data supplied from the restoration unit 52 with the temporary restoration feature data supplied from the feature extraction unit 54, and proceeds to step S196. In step S196, the determination control unit 56 determines the coincidence based on the comparison result between the temporary original feature data and the temporary restored feature data in step S195.

  If it is determined in step S196 that the temporary original feature data and the temporary restoration feature data do not match, the process proceeds to step S197, and the determination control unit 56 sets the restoration unit 51, the switch 57, and the switches 59 to 61. Control.

  That is, the determination control unit 56 causes the switch 57 to select the terminal 57A and generates and outputs a reprocessing instruction signal. Thereby, the reprocessing instruction signal output from the determination control unit 56 is supplied to the restoration unit 51 and the switch 59 via the switch 57 and the terminal 57A.

  When the switch 59 receives the reprocessing instruction signal, the switch 59 changes from the default off state to the on state, whereby the restoration unit 51 is in a state in which the operation information buffer 58 can be referred to via the switch 59. Become.

  Further, when receiving the reprocessing instruction signal, the restoration unit 51 refers to the operation information stored in the operation information buffer 58 via the switch 59 and has not yet performed the second embedded encoded data. Recognize how to do no bitplane swap. Then, the restoration unit 51 selects one of the recognized bit plane swap methods, and returns to step S192.

  In step S192, the restoration unit 51 operates the second embedded encoded data in accordance with the bit-plane swap method selected in step S197, so that the second embedded encoded data has been processed so far. Another temporary first embedded encoded data and temporary embedded data are newly restored. In step S193, the restoration unit 52 operates the new temporary first embedded encoded data restored by the restoration unit 51 according to the line rotation method. The temporary embedding target data and the temporary original feature data are newly restored, and the same processing is repeated thereafter.

  On the other hand, if it is determined in step S196 that the temporary original feature data and the temporary restoration feature data match, that is, the temporary embedding target data has been accurately restored to the original embedding target data. If the original original feature data is also accurately restored to the original feature data obtained from the original embedding target data, and if the temporary original feature data matches the temporary restored feature data, the process proceeds to step S198, and the judgment control is performed. The unit 56 controls the switches 57, 60 and 61.

  That is, the determination control unit 56 causes the switch 57 to select the terminal 57B and generates and outputs an output instruction signal. As a result, the output instruction signal output from the determination control unit 56 is supplied to the switches 60 and 61 via the switch 57 and the terminal 57B.

  When receiving the output instruction signal, the switches 60 and 61 temporarily change from the off state to the on state, and the process proceeds to step S199, where the storage contents of the embedding target data buffer 53 are read out via the switch 60 and embedded. The stored contents of the target data buffer 55 are read out via the switch 61, and the process is terminated.

  That is, in the embedding target data buffer 53, every time the restoration unit 52 outputs new temporary embedding target data, the new temporary embedding target data is stored in an overwritten form. Also, in the embedded data buffer 55, every time the restoration unit 51 outputs new temporary embedded data, the new temporary embedded data is stored in an overwritten form.

  On the other hand, when the temporary original feature data and the temporary restoration feature data match, as described above, the temporary embedding target data output from the restoration unit 52 is accurately restored as the original embedding target data. In addition, the temporary embedded data output from the restoration unit 51 is also accurately restored as the original embedded data.

  Accordingly, when the temporary original feature data and the temporary restoration feature data match, the embedding target data buffer 53 stores the embedding data that has been correctly restored, and the embedded data buffer 55 stores the embedding target data. Also, the embedded data that has been accurately restored is stored. As a result, in step S199, the correctly restored embedding target data is read from the embedding target data buffer 53 via the switch 60, and the embedding target data buffer 55 is accurately output via the switch 61. The data to be embedded restored in (1) is read out.

  Note that the processing in FIG. 41 is repeatedly performed each time the second embedded encoded data is newly supplied.

  Next, in the above-described case, as described with reference to FIG. 37A, only the feature data extracted from the image data is embedded in the initial embedding process for the image data as the embedding target data. In the embedding process, it is possible to embed embedded data as arbitrary data in addition to feature data.

  That is, for example, if the embedded data to be embedded in the first embedding process is referred to as first embedded data and the embedded data to be embedded in the second embedding process is referred to as second embedded data, In the encoder 3, for example, as shown in FIG. 42A, feature data is extracted from image data as embedding target data, and the first embedded data corresponds to the feature data in accordance with a predetermined operation rule. By the operation, the feature data is embedded in the first embedded data.

  Specifically, for example, if the first embedded data is image data, the embedded encoder 3 converts the image data as the first embedded data into feature data by, for example, the line rotation method. The embedding process is performed in response to the above. Thereby, the feature data is embedded in the image data as the first embedded data, and embedded encoded data is generated. Here, the embedded encoded data obtained by embedding the feature data in the first embedded data is hereinafter referred to as feature embedded data as appropriate.

  Thereafter, the embedding encoder 3 performs an embedding process (first embedding process for the embedding target data) for manipulating the image data as the embedding target data in accordance with the feature embedding data, for example, by a line rotation method. Thereby, the feature embedding data is embedded in the image data as the embedding target data, and the first embedded encoded data is generated.

  In the embedded encoder 3, as described in FIG. 37A, the image data as the first embedded encoded data is subjected to an operation that does not overlap with the operation by the line rotation method in the initial embedding process. An embedding process (second embedding process for the embedding target data) is performed in accordance with the second embedding data by the plane swap method. As a result, the second embedded data is embedded in the image data as the first embedded encoded data (embedding target data in which the feature embedded data is embedded), and the second embedded encoded data is generated.

  Here, both the embedding of the feature data into the first embedded data and the embedding of the feature embedded data into the embedding target data are performed by the line rotation method. However, the feature for the first embedded data is A method other than the line rotation method (for example, the above-described pixel swap method) can be employed for embedding data and embedding feature embedding data in data to be embedded. Also, the feature data embedding method for the first embedded data and the feature embedding data embedding method for the embedding target data need not be the same method, but different methods (for example, a pixel swap method and a line). It is possible to employ a rotation method.

  On the other hand, in the decompressor 6, as shown in FIG. 42B, the same bits as the second embedding process in the embedded encoder 3 are performed on the second embedded encoded data, as in the case described in FIG. 37B. An operation as a restoration process by the plane swap method is performed, and thereby, the provisional first embedded encoded data and the provisional second embedded data are restored.

  Further, the decompressor 6 performs an operation as a restoration process by the same line rotation method as the first embedding process in the embedded encoder 3 on the temporary first embedded encoded data, thereby The embedding target data and the temporary feature embedding data are restored.

  Thereafter, the decompressor 6 uses the same method (here, as described above) for the temporary feature embedded data, in which the embedded encoder 3 embeds the feature data in the image data as the first embedded data. In addition, an operation as a restoration process by a line rotation method) is performed, thereby restoring the temporary first embedded data and the temporary original feature data.

  Then, the decompressor 6 extracts feature data (temporary restoration feature data) from the provisional embedding target data and compares it with the provisional original feature data, thereby matching the provisional original feature data with the provisional restoration feature data. Determine sex.

  When the temporary original feature data and the temporary restored feature data do not match, the decompressor 6 has restored the temporary first embedded encoded data to the original first embedded encoded data accurately. If not, the second embedded encoded data is subjected to another operation as a restoration process by the bit plane swap method, so that the temporary first embedded encoded data and the temporary second Restore embedded data.

  Furthermore, the decompressor 6 performs an operation as a restoration process by the same line rotation method as the first embedding process in the embedded encoder 3 on the new temporary first embedded encoded data, The temporary embedding target data and the temporary feature embedding data are newly restored. Further, the restorer 6 performs an operation as a restoration process by the line rotation method on the temporary feature embedded data, thereby newly adding the temporary first embedded data and the temporary original data. Restore feature data.

  Then, the decompressor 6 extracts feature data (temporary restoration feature data) from the new provisional embedding target data and compares it with the new provisional original feature data, so that the provisional original feature data and the provisional restoration are obtained. A match with feature data is determined.

  If the provisional original feature data and the provisional restoration feature data do not match, the restoration unit 6 repeats the same process as described above.

  On the other hand, when the temporary original feature data and the temporary restoration feature data match, the decompressor 6 obtains the temporary embedding target data obtained at that time and the temporary first and second embedded data. Is regarded as an accurate restoration result, and the process is terminated.

  Next, FIG. 43 shows a configuration example of the embedded encoder 3 in FIG. 2 that performs the embedding process described in FIG. 42A. In the figure, portions corresponding to those in FIG. 38 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, the embedded encoder 3 in FIG. 43 is basically configured in the same manner as in FIG. 38 except that an embedded unit 44 is newly provided.

  In the embodiment of FIG. 43, the embedding target data is supplied to the feature extraction unit 41 and the embedder 42. The first embedded data is supplied to the embedder 44, and the second embedded data is supplied to the embedder 43.

  The feature data output from the feature extraction unit 41 is supplied to the embedder 44, and the embedder 44 receives the first embedded data (here, the image data as described above). Then, an embedding process that operates according to the feature data is performed by the line rotation method, thereby generating feature embedding data in which the feature data is embedded in the first embedded data, and supplying it to the embedder 42.

  Next, the embedding process by the embedded encoder 3 in FIG. 43 will be described with reference to the flowchart in FIG.

  First, in step S201, one frame of image data as the embedding target data stored in the embedding target database 1 (FIG. 2) is supplied to the feature extraction unit 41 and the embedder 42. Further, in step S201, some of the embedded data stored in the embedded database 2 (FIG. 2) is supplied as the first embedded data to the embedder 44 and other embedded data. Is supplied to the embedder 43 as the second embedded data, and the process proceeds to step S202.

  In step S202, the feature extraction unit 41 extracts feature data from one frame of image data as embedding target data supplied thereto, supplies the feature data to the embedder 44, and proceeds to step S203.

  In step S <b> 203, the embedder 44 operates the first embedded data supplied from the embedded database 2 (FIG. 2) in accordance with the line rotation method corresponding to the feature data supplied from the feature extraction unit 41. . As a result, the embedder 44 generates feature embedding data in which the feature data is embedded in the first embedded data, supplies it to the embedder 42, and proceeds to step S204.

  In step S <b> 204, the embedder 42 performs line rotation on one frame of image data as embedment data supplied from the embedment target database 1 (FIG. 2) corresponding to the feature embedding data supplied from the embedder 44. By operating according to the method, the feature embedding data is embedded in the image data as the embedding target data. The first embedded encoded data obtained by this embedding is supplied from the embedders 42 to 43, and the process proceeds to step S205.

  In step S205, the embedder 43 corresponds to the second embedded data supplied from the embedded database 2 with one frame of image data as the first embedded encoded data supplied from the embedder 42. The second embedded data is embedded in the image data as the first embedded encoded data by operating by the bit plane swap method. In step S206, the embedder 43 outputs the second embedded encoded data obtained by this embedding as final embedded data, and ends the process.

  Note that the processing in FIG. 44 is repeated for each frame of image data as embedding target data stored in the embedding target database 1.

  Next, FIG. 45 shows a configuration example of the restorer 6 in FIG. 2 that performs the restoration process described in FIG. 42B. In the figure, portions corresponding to those in FIG. 40 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, the restorer 6 in FIG. 45 is basically configured in the same manner as in FIG. 40 except that a switch 62, an embedded data buffer 63, and a restoration unit 64 are newly provided.

  In the embodiment of FIG. 45, the restoration unit 51 operates the second embedded encoded data supplied thereto by the same bit-plane swap method as that in the embedder 43 of FIG. The first embedded encoded data and the temporary second embedded data are restored. Further, the restoration unit 51 supplies the temporary first embedded encoded data to the restoration unit 52 and also supplies the temporary second embedded data to the embedded data buffer 55.

  The restoration unit 52 operates the provisional first embedded encoded data supplied from the restoration unit 51 by the same line rotation method as that in the embedder 42 in FIG. The temporary feature embedding data is restored. Further, the restoration unit 52 supplies the temporary embedding target data to the embedding target data buffer 53 and the feature extraction unit 54, and supplies the temporary feature embedding data to the restoration unit 64.

  The embedded data buffer 55 stores the first embedded data supplied from the restoration unit 51 in an overwritten form.

  The determination control unit 56 determines the coincidence between the temporary original feature data supplied from the restoration unit 64 and the temporary restoration feature data supplied from the feature extraction unit 54, and based on the determination result, the restoration unit 51 Control restoration processing and others.

  That is, the determination control unit 56 controls the restoration unit 51, the switch 57, and the switches 59 to 62 based on the coincidence between the temporary original feature data and the temporary restoration feature data.

  The switch 62 is turned on / off under the control of the determination control unit 56. When the switch 62 is turned on, the embedded data stored in the embedded data buffer 63 can be output. Further, the switch 62 is in an off state by default, unless otherwise controlled by the determination control unit 56, like the switches 59 to 61 described in FIG.

  The embedded data buffer 63 stores the temporary first embedded data output from the restoration unit 64 in an overwritten form.

  The restoration unit 64 operates the feature embedding data supplied from the restoration unit 52 by the same line rotation method as that in the embedder 44 in FIG. 43, and thereby the provisional first embedded data and the provisional data Restore original feature data. Further, the restoration unit 64 supplies the temporary first embedded data to the embedded data buffer 63 and supplies the temporary original feature data to the determination control unit 56.

  Next, with reference to the flowchart of FIG. 46, the restoring process by the restoring unit 6 of FIG. 45 will be described.

  First, in step S211, one frame of image data as the second embedded encoded data is supplied to the restoration unit 51, and the restoration unit 51 receives the second embedded encoded data. In step S211, the embedded data buffer 53, the embedded data buffer 55, the operation information buffer 58, and the embedded data buffer 63 clear the stored contents, and the process proceeds to step S212.

  In step S212, the restoration unit 51 operates the second embedded encoded data supplied thereto by the bit plane swap method, whereby the provisional first embedded encoded data and the provisional first Restore the embedded data. Then, the restoration unit 51 supplies the provisional first embedded encoded data to the restoration unit 52 and also supplies the provisional first embedded data to the embedded data buffer 55 to be overwritten. Remember. Further, in step S212, the restoration unit 51 generates operation information indicating how to operate the second embedded encoded data when the second embedded encoded data is operated by the bit-plane swap method, and the operation information is used for the operation information. The data is supplied to and stored in the buffer 58, and the process proceeds to step S213.

  In step S213, the restoration unit 52 operates the provisional first embedded encoded data supplied from the restoration unit 51 by the line rotation method, thereby obtaining the provisional embedding target data and the provisional feature embedding data. Restore. Then, the restoration unit 52 supplies the temporary embedding target data to the embedding target data buffer 53 and stores it in an overwritten form. Further, the restoration unit 52 supplies the temporary embedding target data to the feature extraction unit 54 and also supplies the temporary feature embedding data to the restoration unit 64, and the process proceeds to step S214.

  In step S214, the restoration unit 64 operates the temporary feature embedding data supplied from the restoration unit 52 by the line rotation method, thereby restoring the temporary first embedded data and the temporary original feature data. To do. Then, the restoration unit 64 supplies the temporary first embedded data to the embedded data buffer 63 and stores it in an overwritten form. Further, the restoration unit 64 supplies temporary original feature data to the determination control unit 56.

  In step S214, the feature extraction unit 54 extracts feature data from temporary restoration image data as temporary embedding target data supplied from the restoration unit 52 in parallel with the above-described processing performed by the restoration unit 64. Then, the temporary restoration feature data is supplied to the determination control unit 56, and the process proceeds to step S215.

  In step S215, the determination control unit 56 compares the temporary original feature data supplied from the restoration unit 64 with the temporary restoration feature data supplied from the feature extraction unit 54, and proceeds to step S216. In step S216, the determination control unit 56 determines the coincidence based on the comparison result between the temporary original feature data and the temporary restored feature data in step S215.

  If it is determined in step S216 that the temporary original feature data and the temporary restoration feature data do not match, the process proceeds to step S217, and the determination control unit 56 sets the restoration unit 51, the switch 57, and the switches 59 to 62. Control.

  That is, the determination control unit 56 causes the switch 57 to select the terminal 57A and generates and outputs a reprocessing instruction signal. Thereby, the reprocessing instruction signal output from the determination control unit 56 is supplied to the restoration unit 51 and the switch 59 via the switch 57 and the terminal 57A.

  When the switch 59 receives the reprocessing instruction signal, the switch 59 changes from the default off state to the on state, whereby the restoration unit 51 is in a state in which the operation information buffer 58 can be referred to via the switch 59. Become.

  Further, when receiving the reprocessing instruction signal, the restoration unit 51 refers to the operation information stored in the operation information buffer 58 via the switch 59 and has not yet performed the second embedded encoded data. Recognize how to do no bitplane swap. Then, the restoration unit 51 selects one of the recognized bit plane swap methods, and returns to step S212.

  In step S212, the restoration unit 51 operates the second embedded encoded data in accordance with the bit-plane swap method selected in step S217, and thus the second embedded encoded data has been processed so far. Another temporary first embedded encoded data and temporary first embedded data are newly restored. In step S213, the restoration unit 52 manipulates the new temporary first embedded encoded data restored by the restoration unit 51 by the line rotation method. The temporary embedding target data and the temporary feature embedding data are newly restored, and the process proceeds to step S214.

  In step S214, the restoration unit 64 operates the new temporary feature embedding data supplied from the restoration unit 52 by the line rotation method, whereby the new temporary first embedded data and the temporary original data Restore feature data. Further, in step S214, the feature extraction unit 54 extracts feature data from temporary restored image data as new temporary embedding target data supplied from the restoration unit 52, and thereafter the same processing is repeated.

  On the other hand, if it is determined in step S216 that the temporary original feature data and the temporary restoration feature data match, that is, the temporary embedding target data has been accurately restored to the original embedding target data. If the original original feature data is also restored to the original feature data obtained from the original embedding target data, and the temporary original feature data matches the temporary restored feature data, the process proceeds to step S218, and the determination control is performed. The unit 56 controls the switches 57, 60, 61, and 62.

  That is, the determination control unit 56 causes the switch 57 to select the terminal 57B and generates and outputs an output instruction signal. Thus, the output instruction signal output from the determination control unit 56 is supplied to the switches 60 to 62 via the switch 57 and the terminal 57B.

  Upon receiving the output instruction signal, the switches 60 to 62 temporarily change from the off state to the on state, and the process proceeds to step S219, where the storage contents of the embedding target data buffer 53 are transferred to the embedding target data via the switch 60. The storage contents of the buffer 55 are read out via the switch 61 and the storage contents of the data buffer 63 to be embedded via the switch 62, respectively, and the process ends.

  That is, in the embedding target data buffer 53, every time the restoration unit 52 outputs new temporary embedding target data, the new temporary embedding target data is stored in an overwritten form. Also, in the embedded data buffer 55, every time the restoration unit 51 outputs new temporary second embedded data, the new temporary second embedded data is stored in an overwritten form. . Furthermore, in the embodiment of FIG. 45, every time the restoration unit 64 outputs new temporary first embedded data in the embedded data buffer 63, the new temporary first embedded data is stored. Is stored in an overwritten form.

  On the other hand, when the temporary original feature data and the temporary restoration feature data match, as described above, the temporary embedding target data output from the restoration unit 52 is accurately restored as the original embedding target data. Furthermore, both the provisional second embedded data output from the restoration unit 51 and the provisional first embedded data output from the restoration unit 64 are accurately restored as the original embedded data.

  Therefore, when the temporary original feature data and the temporary restored feature data match, the embedding target data buffer 53 stores the embedding data that has been correctly restored, and the embedded data buffer 55 63 also stores the second embedded data and the first embedded data that have been correctly restored. As a result, in step S219, the embedding target data accurately restored is read from the embedding target data buffer 53 via the switch 60, and the switches 61 and 62 are switched from the embedding target data buffers 55 and 63. Thus, the correctly restored second and first embedded data are respectively read out.

  As described above, since the embedded data is embedded in the embedding target data and the feature data of the embedding target data is integrated, the integrated feature data (original feature data) and the restored embedding Based on the coincidence with the feature data (temporary restoration feature data) obtained from the target data, it is possible to recognize whether the embedding target data has been accurately restored to the original. As a result, even if the first embedded data is embedded in the embedding target data, and the second embedded data is embedded in the second target data, the embedding target data and the first and second target data are embedded. The embedded data can be restored to the original. That is, a lot of data can be embedded and restored.

  In the embedded encoder 3 of FIG. 43, the embedding units 42 and 43 perform the embedding process by a fixed method. However, the embedding units 42 and 43 have some predetermined methods (for example, It is possible to select an arbitrary method from among the pixel swap method, the line rotation method, the bit plane swap method, and the like described above, and perform the embedding process by the selected method.

  In this case, in the restoration device 6 of FIG. 45, when there is no coincidence of the feature data in the determination control unit 56, the above-described several predetermined methods are sequentially selected, and the restoration process by the selected method is performed. By adopting control of the restoration units 51 and 52 such that the data to be embedded and the embedded data can be restored.

  Next, in the embodiment shown in FIG. 37, the feature data is embedded by the line rotation method in the first embedding process, and the embedded data is converted by the bit plane swap method different from the line rotation method in the second embedding process. Although embedding is performed, the feature data and the data to be embedded can be embedded by embedding processing using the same method.

  That is, in this case, as shown in FIG. 47A, the embedded encoder 3 operates the image data as the data to be embedded in accordance with the feature data and the embedded data, for example, by the line rotation method. Data. Specifically, the embedded encoder 3 rotates the horizontal line of the image data as the data to be embedded corresponding to the feature data, and further rotates the vertical line corresponding to the embedded data. Thus, embedded encoded data in which the feature data and the embedded data are embedded in the embedding target data is generated.

  On the other hand, in this case, in the decompressor 6, as shown in FIG. 47B, the horizontal and vertical lines of the image data as the embedded encoded data are rotated as described with reference to FIG. Thus, the embedding target data, the original feature data, and the embedded data are restored.

  Furthermore, the decompressor 6 extracts feature data from the restored embedding target data. Then, the decompressor 6 determines the matching between the extracted feature data and the restored original feature data. Based on the determination result, the decompressor 6 converts the embedded encoded data into the embedding target data, the original feature data, and the encoded data. Controls the restoration process to restore the embedded data.

  In the embodiment of FIG. 47, the feature data and the embedded data are embedded in the data to be embedded by the line rotation method, and as described with reference to FIG. 31, the embedded encoded data obtained by the embedding is In principle, by using the correlation of the image data as the embedding target data, it is accurately restored to the embedding target data, the original feature data, and the embedded data. Therefore, basically, it is not necessary to control the restoration process based on the determination result of the matching of the feature data.

  However, if any error (disturbance) occurs in the embedded encoded data obtained by the embedded encoder 3, the embedded encoded data to be restored by the decompressor 6 is generated by the embedded encoder 3. The embedded encoded data may be different from the embedded encoded data, and the embedded encoded data can be obtained only by using the correlation of the image data as the embedded target data. It may be difficult to accurately restore data and embedded data.

  Therefore, in the embodiment of FIG. 47B, as described above, the decompressor 6 determines the coincidence between the feature data extracted from the restored embedding target data and the restored original feature data, and the determination result is Based on this, the restoration processing for restoring the embedded encoded data to the embedding target data, the original feature data, and the embedded data is controlled, and even if there is an error in the embedded encoded data, Such embedded encoded data can be accurately restored to embedding target data, original feature data, and embedded data.

  That is, FIG. 48 shows a configuration example of the embedded encoder 3 in FIG. 2 that performs the embedding process described in FIG. 47A. In the figure, portions corresponding to those in FIG. 38 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, the embedded encoder 3 in FIG. 48 is basically configured in the same manner as in FIG. 38 except that the embedded unit 43 is not provided.

  In the embodiment of FIG. 48, the embedded data supplied from the embedded database (FIG. 2) is supplied to the embedder 42 together with the feature data output from the feature extraction unit 41. Then, the embedder 42 rotates the horizontal line of the image data as the embedding target data supplied from the embedding target database 1 (FIG. 2) corresponding to the feature data, and further, the vertical line is replaced with the embedded data. , The embedded encoded data in which the feature data and the embedded data are embedded in the data to be embedded is generated and output.

  Next, the embedding process by the embedded encoder 3 in FIG. 48 will be described with reference to the flowchart in FIG.

  First, in step S221, one frame of image data as embedding target data stored in the embedding target database 1 (FIG. 2) is supplied to the feature extraction unit 41 and the embedder 42, and the embedded database 2 The embedded data stored in (FIG. 2) is supplied to the embedder 42, and the process proceeds to step S222.

  In step S222, the feature extraction unit 41 extracts feature data from one frame of image data as embedding target data supplied thereto, supplies the feature data to the embedder 42, and proceeds to step S223.

  In step S223, the embedder 42 uses one frame of image data as the embedding target data supplied from the embedding target database 1 (FIG. 2), the feature data supplied from the feature extraction unit 41, and the embedded database 2 ( Corresponding to the embedded data supplied from FIG. 2), operation is performed by the line rotation method, whereby the feature data and the embedded data are embedded in the image data as the embedded data, and the process proceeds to step S224. In step S224, the embedder 42 outputs the embedded encoded data and ends the process.

  Note that the processing in FIG. 49 is repeatedly performed for each frame of image data as embedding target data stored in the embedding target database 1.

  Next, FIG. 50 shows a configuration example of the restoration device 6 of FIG. 2 that performs the restoration processing described in FIG. 47B. In the figure, portions corresponding to those in FIG. 40 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, the restorer 6 of FIG. 50 is basically the same as the case of FIG. 40 except that a value change unit 71 or a change information buffer 72 is provided instead of the restore unit 51 or the operation information buffer 58, respectively. It is configured similarly.

  The image data as embedded encoded data obtained by the embedded encoder 3 in FIG. 48 is supplied to the value changing unit 71.

  The value changing unit 71 changes only a minute value by manipulating the pixel value of each pixel constituting the image data as the embedded encoded data supplied thereto, and the embedded encoded data in which the pixel value is changed (Hereinafter referred to as change embedded data as appropriate) is supplied to the restoration unit 52. Here, to what extent the value changing unit 71 changes the pixel value is, for example, the noise that can be superimposed on the embedded encoded data between the embedded encoder 3 and the decompressor 6. Determined based on maximum level.

  Further, the value changing unit 71 generates information indicating how the pixel value of each pixel constituting the image data as the embedded encoded data has been changed (hereinafter referred to as change information as appropriate), and the change. The information is supplied to the change information buffer 72.

  The value changing unit 71 changes the pixel value of each pixel constituting the image data as the embedded encoded data according to the control of the determination control unit 56, and changes the change information stored in the change information buffer 72 as follows. This is done with reference to the switch 59.

  That is, when the value changing unit 71 receives certain embedded encoded data, first, for example, without changing the pixel value of the image data as the embedded encoded data (add 0 to the pixel value). As it is, it is output as change embedded data. Then, the value changing unit 71 generates change information corresponding to the change in the pixel value of the image data as the embedded encoded data, and supplies the change information to the change information buffer 72 for storage. In this case, change information indicating that the change of the pixel value is 0 (no change of the pixel value) is generated for each pixel constituting the image data as the embedded encoded data.

  After that, when the value change unit 71 receives a reprocessing instruction signal indicating a request for another restoration process from the determination control unit 56, the value change unit 71 refers to the change information stored in the change information buffer 72 via the switch 59. Thus, the change of the pixel value that has been applied to the embedded encoded data is recognized, and changes other than the recognized change are applied to the embedded encoded data. As a result, the value changing unit 71 obtains image data as change embedded data composed of pixel values of a pattern not obtained so far.

  The restoration unit 52 operates the image data as the change embedded data supplied from the value change unit 71 by the same line rotation method as that in the embedder 42 in FIG. Original feature data and temporary embedded data are restored.

  That is, the restoration unit 52 rotates the horizontal line of the image data as the change embedding data to a position where the correlation becomes maximum using the correlation of the image data as the embedding target data, and the vertical line After all, by using the correlation of the image data as the embedding target data, the temporary embedding target data is restored by rotating to the position where the correlation becomes maximum. Further, the restoration unit 52 restores the temporary original feature data based on the horizontal line rotation method of the image data as the change embedded data, and the temporary embedding target based on the vertical line rotation method. Restore data.

  The restoration unit 52 supplies the temporary embedding target data to the embedding target data buffer 53 and the feature extraction unit 54, and supplies the temporary embedding data to the embedded data buffer 55. Further, the restoration unit 52 supplies temporary original feature data to the determination control unit 56.

  The change information buffer 72 temporarily stores change information supplied from the value changing unit 71.

  Next, with reference to the flowchart of FIG. 51, the restoration process by the decompressor 6 of FIG. 50 will be described.

  First, in step S231, one frame of image data as embedded encoded data is supplied to the value changing unit 71, and the value changing unit 71 receives the embedded encoded data. Further, in step S231, the embedding target data buffer 53, the embedded data buffer 55, and the change information buffer 72 clear the stored contents, and the process proceeds to step S232.

  In step S232, the value changing unit 71 changes the pixel value of the pixels constituting the image data as the embedded encoded data supplied thereto by a minute value, thereby obtaining the changed embedded data, and the restoring unit 52. Further, in step S232, the value changing unit 71 generates change information indicating how the pixel value of the image data as the embedded encoded data has been changed, and supplies the change information to the change information buffer 72. And the process proceeds to step S233.

  In step S233, the restoration unit 52 operates the change embedded data supplied from the value change unit 71 by the same line rotation method as that in the embedder 42 in FIG. Original feature data and temporary embedded data are restored. Then, the restoration unit 52 supplies the temporary embedding target data to the embedding target data buffer 53 and stores the temporary embedding target data in the form of overwriting, and also supplies the temporary embedding data to the embedded data buffer 55 for overwriting. To remember. Further, the restoration unit 52 supplies the temporary embedding target data to the feature extraction unit 54 and also supplies the temporary original feature data to the determination control unit 56, and the process proceeds to step S234.

  In step S234, the feature extraction unit 54 extracts feature data from the temporary restoration image data as temporary embedding target data supplied from the restoration unit 52, and supplies the extracted feature data to the determination control unit 56 as temporary restoration feature data. The process proceeds to step S235.

  In step S235, the determination control unit 56 compares the temporary original feature data supplied from the restoration unit 52 with the temporary restoration feature data supplied from the feature extraction unit 54, and proceeds to step S236. In step S236, the determination control unit 56 determines the coincidence based on the comparison result between the temporary original feature data and the temporary restored feature data in step S235.

  In step S236, when it is determined that the temporary original feature data and the temporary restoration feature data do not match, the process proceeds to step S237, and the determination control unit 56, the value change unit 71, the switch 57, and the switches 59 to 61. To control.

  That is, the determination control unit 56 causes the switch 57 to select the terminal 57A and generates and outputs a reprocessing instruction signal. Accordingly, the reprocessing instruction signal output from the determination control unit 56 is supplied to the value changing unit 71 and the switch 59 via the switch 57 and the terminal 57A.

  When the switch 59 receives the reprocessing instruction signal, the switch 59 changes from the default off state to the on state, whereby the value changing unit 71 can refer to the change information buffer 72 via the switch 59. It becomes.

  Further, when the value change unit 71 receives the reprocessing instruction signal, the value change unit 71 refers to the change information stored in the change information buffer 72 via the switch 59 and still does not process the image data as the embedded encoded data. Recognize how to change pixel values that have not been performed. Then, the value changing unit 71 selects one of the recognized pixel value changing methods, and returns to Step S232.

  In step S232, the value changing unit 71 changes the pixel value of the image data as the embedded encoded data in accordance with the method of changing the pixel value selected in step S237. Change embedded data is newly obtained as image data having a pixel value pattern different from the previous one. In step S233, the restoration unit 52 operates the new change embedding data obtained by the value change unit 71 by the line rotation method. As a result, another temporary embedding target data different from the previous one is also obtained. The temporary original feature data and the temporary embedded data are newly restored, and the same processing is repeated thereafter.

  On the other hand, if it is determined in step S236 that the temporary original feature data and the temporary restoration feature data match, that is, the temporary embedding target data has been accurately restored to the original embedding target data. If the original original feature data is also accurately restored to the original feature data obtained from the original embedding target data, and if the temporary original feature data matches the temporary restored feature data, the process proceeds to step S238, and determination control is performed. The unit 56 controls the switches 57, 60 and 61.

  That is, the determination control unit 56 causes the switch 57 to select the terminal 57B and generates and outputs an output instruction signal. As a result, the output instruction signal output from the determination control unit 56 is supplied to the switches 60 and 61 via the switch 57 and the terminal 57B.

  When receiving the output instruction signal, the switches 60 and 61 temporarily change from the off state to the on state, and the process proceeds to step S239, where the stored contents of the embedding target data buffer 53 are read out via the switch 60 and embedded. The stored contents of the target data buffer 55 are read out via the switch 61, and the process is terminated.

  That is, in the embedding target data buffer 53, every time the restoration unit 52 outputs new temporary embedding target data, the new temporary embedding target data is stored in an overwritten form. Also, in the embedded data buffer 55, every time the restoration unit 52 outputs new temporary embedded data, the new temporary embedded data is stored in an overwritten form.

  On the other hand, when the temporary original feature data and the temporary restoration feature data match, as described above, the temporary embedding target data output from the restoration unit 52 and the temporary embedding data are respectively the original data It is accurately restored as the embedding target data and the original embedded data.

  Accordingly, when the temporary original feature data and the temporary restoration feature data match, the embedding target data buffer 53 stores the embedding data that has been correctly restored, and the embedded data buffer 55 stores the embedding target data. Also, the embedded data that has been accurately restored is stored. As a result, in step S239, the correctly restored embedding target data is read from the embedding target data buffer 53 via the switch 60, and the embedding target data buffer 55 is accurately output via the switch 61. The data to be embedded restored in (1) is read out.

  Note that the process of FIG. 51 is repeatedly performed each time the embedded encoded data is newly supplied.

  As described above, the decompressor 6 determines the matching between the feature data extracted from the restored embedding target data and the restored original feature data, and based on the determination result, the embedding coding in the restoration process is performed. Since the change of the pixel value of the image data as data is controlled, an error occurs in the embedded encoded data, and the pixel value of the image data as the embedded encoded data has changed. The data to be embedded and the embedded data can be accurately restored from such embedded encoded data. Therefore, in this case, robustness against errors can be improved.

  In the above case, the feature data and the embedded data are embedded in the image data as the embedding target data. However, only the feature data is embedded in the image data as the embedding target data. Is possible.

  Next, the series of processes described above can be performed by hardware or software. When a series of processing is performed by software, a program constituting the software is installed in a general-purpose computer or the like.

  Thus, FIG. 52 shows a configuration example of an embodiment of a computer in which a program for executing the above-described series of processing is installed.

  The program can be recorded in advance in a hard disk 105 or a ROM 103 as a recording medium built in the computer.

  Alternatively, the program is stored temporarily on a removable recording medium 111 such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), a MO (Magneto Optical) disk, a DVD (Digital Versatile Disc), a magnetic disk, or a semiconductor memory. It can be stored permanently (recorded). Such a removable recording medium 111 can be provided as so-called package software.

  The program is installed in the computer from the removable recording medium 111 as described above, or transferred from the download site to the computer wirelessly via a digital satellite broadcasting artificial satellite, LAN (Local Area Network), The program can be transferred to a computer via a network such as the Internet, and the computer can receive the program transferred in this way by the communication unit 108 and install it in the built-in hard disk 105.

  The computer includes a CPU (Central Processing Unit) 102. An input / output interface 110 is connected to the CPU 102 via the bus 101, and the CPU 102 operates an input unit 107 including a keyboard, a mouse, a microphone, and the like by the user via the input / output interface 110. When a command is input as a result, the program stored in a ROM (Read Only Memory) 103 is executed accordingly. Alternatively, the CPU 102 also transfers from a program stored in the hard disk 105, a program transferred from a satellite or a network, received by the communication unit 108 and installed in the hard disk 105, or a removable recording medium 111 attached to the drive 109. The program read and installed in the hard disk 105 is loaded into a RAM (Random Access Memory) 104 and executed. Thus, the CPU 102 performs processing according to the above-described flowchart or processing performed by the configuration of the above-described block diagram. Then, the CPU 102 outputs the processing result from the output unit 106 configured with an LCD (Liquid Crystal Display), a speaker, or the like via the input / output interface 110, or from the communication unit 108 as necessary. Transmission and further recording on the hard disk 105 are performed.

  Here, in this specification, the processing steps for describing a program for causing a computer to perform various types of processing do not necessarily have to be processed in time series according to the order described in the flowchart, but in parallel or individually. This includes processing to be executed (for example, parallel processing or processing by an object).

  Further, the program may be processed by a single computer, or may be processed in a distributed manner by a plurality of computers. Furthermore, the program may be transferred to a remote computer and executed.

  The embedded data is not particularly limited. For example, image data, audio data, text, a computer program, a control signal, and other data can be used as the embedded data.

  In the present embodiment, image data is used as the embedding target data, and the embedded data and feature data are embedded in the image data. However, as the embedding target data, for example, audio data or the like is used. It is also possible to adopt. That is, in this case, for example, it is possible to embed embedded data and feature data in audio by dividing time-series audio data into appropriate frames and replacing the audio data of each frame according to the embedded data. It is. In this case, linear prediction coefficients, cepstrum coefficients, and the like can be used as the feature data of the speech data as the embedding target data.

  Furthermore, in this embodiment, image data is set as data to be embedded, and an embedding process for embedding embedded data is performed on the image data. However, the process performed on the image data is limited to the embedding process. It is not a thing. In other words, it is possible to perform some processing on the image data and output the image data integrated with the feature data. The processing applied to the image data is extremely irreversible, and can be restored based on the processed image data.

  In other words, in the present invention, when the processed image data is restored, the temporary image data is restored by performing some kind of restoration processing on the processed image data. Then, the restoration process is controlled so that the feature data extracted from the temporary image data matches the feature data integrated with the processed image data. As a result, even when irreversible processing is performed on the image data, the original image data can be restored. In this case, if the process performed on the image data is regarded as encryption, the image data is subjected to strong encryption, and the secrecy can be improved.

It is a figure explaining the outline | summary of the embedding process and restoration process which have been proposed previously. It is a block diagram which shows the structural example of one Embodiment of an embedded encoding / restoration system. 3 is a block diagram illustrating a configuration example of an embedded encoder 3. FIG. 3 is a block diagram illustrating a configuration example of a restorer 6. FIG. It is a figure which shows the image data as embedding target data. It is a figure for demonstrating embedding / restoration using correlation. It is a figure for demonstrating embedding / restoration using correlation. It is a flowchart explaining an embedding process. It is a flowchart explaining a restoration process. It is a figure explaining the embedding / restoration using continuity. It is a flowchart explaining an embedding process. It is a figure explaining an embedding process and a decompression | restoration process. It is a flowchart explaining a restoration process. It is a figure for demonstrating embedding / restoration using similarity. It is a figure for demonstrating embedding / restoration using similarity. It is a flowchart explaining an embedding process. It is a figure explaining an embedding process and a decompression | restoration process. It is a flowchart explaining a restoration process. It is a flowchart for demonstrating an embedding process. It is a flow chart for explaining restoration processing. It is a figure for demonstrating the embedding process by a bit plane swap system. It is a flowchart for demonstrating an embedding process. It is a figure for demonstrating the calculation method of the correlation of a bit plane. It is a figure for demonstrating the restoration process by a bit plane swap system. It is a flow chart for explaining restoration processing. It is a flowchart for demonstrating an embedding process. It is a figure for demonstrating rotation. It is a figure for demonstrating the result of an embedding process. It is a figure for demonstrating the restoration process by a line rotation system. It is a flow chart for explaining restoration processing. It is a figure explaining the embedding process and restoration process which rotate both a horizontal line and a vertical line. 3 is a block diagram illustrating a configuration example of an embedded encoder 3. FIG. 5 is a flowchart for explaining embedding processing by an embedded encoder 3; 3 is a block diagram illustrating a configuration example of a restorer 6. FIG. It is a flowchart explaining the decompression | restoration process by the decompression | restoration device 6. FIG. It is a figure explaining the outline | summary of the embedding process which embeds feature data, and the decompression | restoration process using the feature data. It is a figure explaining the embedding process by the embedding encoder 6, and the decompression | restoration process by the decompression | restoration device 6. FIG. 3 is a block diagram illustrating a configuration example of an embedded encoder 3. FIG. 5 is a flowchart for explaining embedding processing by an embedded encoder 3; 3 is a block diagram illustrating a configuration example of a restorer 6. FIG. It is a flowchart explaining the decompression | restoration process by the decompression | restoration device 6. FIG. It is a figure explaining the embedding process by the embedding encoder 6, and the decompression | restoration process by the decompression | restoration device 6. FIG. 3 is a block diagram illustrating a configuration example of an embedded encoder 3. FIG. 5 is a flowchart for explaining embedding processing by an embedded encoder 3; 3 is a block diagram illustrating a configuration example of a restorer 6. FIG. It is a flowchart explaining the decompression | restoration process by the decompressor 6. FIG. It is a figure explaining the embedding process by the embedding encoder 6, and the decompression | restoration process by the decompression | restoration device 6. FIG. 3 is a block diagram illustrating a configuration example of an embedded encoder 3. FIG. 5 is a flowchart for explaining embedding processing by an embedded encoder 3; 3 is a block diagram illustrating a configuration example of a restorer 6. FIG. It is a flowchart explaining the decompression | restoration process by the decompression | restoration device 6. FIG. It is a block diagram which shows the structural example of one Embodiment of the computer to which this invention is applied.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Embedded database, 2 Embedded database, 3 Embedded encoder, 4 Recording medium, 5 Transmission medium, 6 Restorer, 11 Encoding apparatus, 12 Decoding apparatus, 21 Frame memory, 22 Embedding part, 23 Feature extraction part, 24 integration unit, 31 frame memory, 32 restoration unit, 33 feature separation unit, 34 feature extraction unit, 35 determination control unit, 41 feature extraction unit, 42, 43, 44 embedder, 51, 52 restoration unit, 53 embedding target data Buffer, 54 feature extraction unit, 55 embedded data buffer, 56 determination control unit, 57 switch, 57A and 57B terminals, 58 operation information buffer, 59 to 62 switch, 63 embedded data buffer, 64 restoration unit, 71 Value change part, 72 Change information buffer, 101 bus, 102 CPU, 103 ROM, 104 RAM, 105 hard disk, 106 output unit, 107 input unit, 108 communication unit, 109 drive, 110 input / output interface, 111 removable recording medium

Claims (9)

Feature data extraction means for extracting feature data representing the feature from input data;
In accordance with a predetermined operation rule for performing operation so that the information embedded with other information is restored to the original information using the energy bias of the information, the feature data is added to other arbitrary data. A data processing apparatus comprising: embedding means for embedding the other arbitrary data in which the first feature data is embedded in the input data, and further embedding the arbitrary data. A feature data extraction step for extracting feature data representing the feature from the input data;
In accordance with a predetermined operation rule for performing operation so that the information embedded with other information is restored to the original information using the energy bias of the information, the feature data is added to other arbitrary data. A data processing method comprising: embedding, embedding the other arbitrary data in which the first feature data is embedded in the input data, and further embedding the arbitrary data. A feature data extraction step for extracting feature data representing the feature from the input data;
In accordance with a predetermined operation rule for performing an operation so that the information embedded with other information is restored to the original information by using the energy bias of the information, the first arbitrary A program for embedding feature data, embedding the other arbitrary data in which the feature data is embedded in the input data, and further embedding the arbitrary data. A data processing device for restoring processing data obtained by processing the input data, which is transmitted together with first feature data representing the characteristics of the input data,
A restoration unit that restores the input data by performing a predetermined restoration process on the processing data, and outputs the restored input data as restoration data;
Feature data extraction means for extracting second feature data representing the feature from the restored data;
Control means for determining the coincidence of the first and second feature data and controlling the restoration processing in the restoration means based on the determination result;
The processing data is stored in other arbitrary data in accordance with a predetermined operation rule for performing operation so that the information embedded with other information is restored to the original information by using the energy bias of the information. The first feature data is embedded, the other arbitrary data in which the first feature data is embedded is embedded in the input data, and further, arbitrary data is embedded,
The restoration means includes
The input data in which the other arbitrary data in which the first feature data is embedded is embedded from the processing data, and the arbitrary data is restored,
The input data in which the other arbitrary data embedded with the first feature data is embedded is restored to the other arbitrary data in which the first feature data is embedded, and the input data,
Furthermore, the other arbitrary data in which the first feature data is embedded is restored to the first feature data and the other arbitrary data. The data processing apparatus according to claim 4, wherein the control unit controls the restoration process so that the first and second feature data match. The restoration means restores the input data by performing a restoration process for operating the processing data according to a predetermined operation rule,
The data processing apparatus according to claim 4, wherein the control unit controls an operation on the processing data in the restoration processing. The restoration means restores the input data by changing a value of the processing data and performing a restoration process of operating the processing data after the change according to a predetermined operation rule,
The data processing apparatus according to claim 4, wherein the control unit controls a change in the value of the processing data in the restoration processing. A data processing method for restoring processing data obtained by processing the input data, which is transmitted together with first feature data representing characteristics of the input data,
A restoration step of restoring the input data by performing a predetermined restoration process on the processed data, and outputting the restored input data as restored data;
A feature data extracting step of extracting second feature data representing the feature from the restored data;
A control step of determining matching between the first and second feature data, and controlling a restoration process in the restoration step based on a result of the determination,
The processing data is stored in other arbitrary data in accordance with a predetermined operation rule for performing operation so that the information embedded with other information is restored to the original information by using the energy bias of the information. The first feature data is embedded, the other arbitrary data in which the first feature data is embedded is embedded in the input data, and further, arbitrary data is embedded,
In the restoration step,
The input data in which the other arbitrary data in which the first feature data is embedded is embedded from the processing data, and the arbitrary data is restored,
The input data in which the other arbitrary data embedded with the first feature data is embedded is restored to the other arbitrary data in which the first feature data is embedded, and the input data,
Furthermore, the other arbitrary data in which the first feature data is embedded is restored to the first feature data and the other arbitrary data. A program for causing a computer to perform data processing for restoring processing data obtained by processing the input data, which is transmitted together with first feature data representing characteristics of the input data,
A restoration step of restoring the input data by performing a predetermined restoration process on the processed data, and outputting the restored input data as restored data;
A feature data extracting step of extracting second feature data representing the feature from the restored data;
A control step of determining matching between the first and second feature data, and controlling a restoration process in the restoration step based on a result of the determination,
The processing data is stored in other arbitrary data in accordance with a predetermined operation rule for performing operation so that the information embedded with other information is restored to the original information by using the energy bias of the information. The first feature data is embedded, the other arbitrary data in which the first feature data is embedded is embedded in the input data, and further, arbitrary data is embedded,
In the restoration step,
The input data in which the other arbitrary data in which the first feature data is embedded is embedded from the processing data, and the arbitrary data is restored,
The input data in which the other arbitrary data embedded with the first feature data is embedded is restored to the other arbitrary data in which the first feature data is embedded, and the input data,
Further, a program for restoring the other arbitrary data in which the first feature data is embedded into the first feature data and the other arbitrary data.

Images