JP2001218006A - Picture processor, picture processing method and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To shorten time required for the judgment processing of a specified picture with respect to the other picture at the time of judging the specified picture having first information and second information. SOLUTION: A first information extraction step for extracting first information from the picture and a judgment step for judging whether second information is to be extracted from the picture in accordance with a first information extraction result are included for solving the subject.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus and method for detecting a specific image for which printing is not permitted, and a storage medium storing the method.

[0002]

2. Description of the Related Art In recent years, with the rapid development and spread of computers and their networks, various types of information such as character data, image data, and voice data have been digitized. Digital information is not degraded due to aging and can be stored forever in perfect condition, but can be easily copied, and copyright protection is a major issue. Therefore, security technology for copyright protection is rapidly gaining importance.

[0003] One technique for protecting copyrights is "digital watermarking". Digital watermarking is a technology that embeds the name of the copyright holder or the ID of the purchaser in a form that cannot be perceived by humans in digital image data, audio data, character data, etc., and tracks unauthorized use by illegal copying. is there.

[0004] Furthermore, in addition to the purpose of tracking unauthorized use by illegal copying, digital watermarking can be used for the purpose of detecting unauthorized printing of specific images such as banknotes and securities. This is to detect a specific image by embedding a digital watermark in a specific image in advance and extracting the digital watermark during printing.

[0005]

Heretofore, the above-described apparatus for extracting a digital watermark has the following features: A similar process was being executed. Therefore, unnecessary processing is performed on an image in which a digital watermark is not embedded, which causes the digital watermark extraction processing time to be increased.

Further, when a specific image in which a digital watermark is embedded is detected at the time of printing, unnecessary processing is performed on most non-specific images, and the printing time is lengthened.

[0007]

To solve the above problems, the present invention provides a first information extracting step of extracting first information from an image, and a second information extracting step according to the first information extracting result. And judging whether or not to extract from the image.

[0008]

DESCRIPTION OF THE PREFERRED EMBODIMENTS (First Embodiment) [1. Digital Watermark Embedding Apparatus] Hereinafter, an outline of a digital watermark embedding apparatus according to the present embodiment will be described with reference to the drawings.

FIG. 1 shows a digital watermark embedding device according to the present embodiment. As shown in FIG. 1, the digital watermark embedding device includes a color component extracting unit 0101, a registration signal embedding unit 0102, a pattern arrangement determining unit 0110, an embedding position determining unit 0103, an additional information embedding unit 0104, a color component synthesizing unit 0105, JPE
G compression encoding means 0106, memory 0107, JPEG
It comprises a decompression decoding means 0108 and a printer 0109.

The digital watermark embedding device stores image data.
I is entered. This is multi-valued image data to which a plurality of predetermined bits are assigned per pixel.

In the present embodiment, it is possible to cope with whether the input image data I is grayscale image data or color image data. Gray-scale image data is composed of one type of element per pixel, and color image data is composed of three types of element per pixel. In the present embodiment, the three types of elements are a red component (R), a green component (G), and a blue component (B). However, the present invention is applicable to other combinations of color components.

The image data I input to the digital watermark embedding device is first input to the color component extracting means 0101.

If the input image data I is color image data, only the blue component is separated from the color image data by the color component extraction means 0101 and output to the registration signal embedding means 0102 at the subsequent stage.

On the other hand, the other color components are output to the subsequent color component synthesizing means 0105. That is, here, only the color component in which the digital watermark information is to be embedded is separated and sent to the digital watermark processing system.

In the present embodiment, digital watermark information is embedded in the blue component. This is because the blue component is the least insensitive to human vision among the red, blue, and green components. Therefore, embedding the digital watermark information in the blue component has the effect of making it harder for human eyes to perceive image quality degradation due to the digital watermark information than embedding the digital watermark information in other color components.

If the input image data I is gray scale image data, the color component extracting means 01
Reference numeral 01 temporarily converts grayscale image data into pseudo color image data.

The pseudo color image data referred to here is color image data composed of three types of elements per pixel. In the present embodiment, three types of elements (R, G, B) are used. Are all equal.

The gray scale image data is converted into the above-described pseudo color image data, and the blue component (B) in the color image data is extracted and output to the registration signal embedding means 0102.

On the other hand, the other color components are output to the subsequent color component synthesizing means 0105. In this way, as in the case of the color image data described above, the electronic watermark information is embedded not in all the color components but only in the blue component.

In the following description, the case where the image data I is color image data and the case where the image data I is grayscale image data are described as little as possible.
That is, description will be made so that color image data and pseudo color image data are not distinguished.

Next, the registration signal embedding means 0102 will be described. Here, the registration signal is a signal required to execute geometric correction as preprocessing for extracting digital watermark information.

Registration signal embedding means 01
In 02, the image data of the blue component obtained by the color component extracting means 0101 is input. The registration signal embedding unit 0102 embeds a registration signal in image data using a kind of digital watermarking technique. That is, in image data in which a registration signal is embedded, human vision cannot perceive the registration signal. Details such as the method of embedding the registration signal will be described later.

Registration signal embedding means 01
02 outputs image data in which a registration signal is embedded.

The pattern arrangement determining means 0110 converts the image data in which the digital watermark information is embedded into the image data by the printer 0109.
And the resolution of the image represented by the input image data so that the digital watermark information (additional information) can be sufficiently extracted (detected) even when the density gradation changes to the area gradation. The pattern arrangement in which the digital watermark information (additional information) is embedded is determined based on the output resolution from.
The method of determining the pattern arrangement will be described later.

The resolution of an image is defined as the number of pixels per inch of the image (bitmap image) when the image is to be printed in a predetermined size. Therefore, when a certain image is to be printed in a predetermined size, the image having a larger number of pixels has a higher image resolution. Pixel / inch is used as the unit indicating the resolution of the image.

The output resolution of the printer indicates the number of dots printed by the printer per inch on a print medium. A printer with a larger number of dots printed per inch has a higher output resolution.

The pattern arrangement determining means 0110 outputs the pattern arrangement selected from the plurality to the embedding position determining means 0103 together with the input image data.

The next embedding position determining means 0103 determines the embedding position of the additional information Inf in the image data in which the registration signal is embedded.

The embedding position determining means 0103 outputs, to the additional information embedding means 0104, control data representing the position where the additional information Inf is embedded in the image, together with the input image data and pattern arrangement.

The additional information embedding means 0104 inputs additional information Inf (a plurality of bit information) in addition to the image data, the pattern arrangement, and the control data. This additional information I
The nf is embedded at the determined embedding position in the image data of the blue component by using a digital watermark technique.
Embedding of the additional information Inf using the digital watermark technique will also be described later.

Image data in which the additional information Inf is embedded is output from the additional information embedding unit 0104, and is input to the color component synthesizing unit 0105.

The color component synthesizing unit 0105 uses the blue component processed up to the preceding stage (additional information embedding unit 0104) and the red and green components directly input from the color component extracting unit 0101 to generate a normal color component. Synthesize into the form of image data.

The color image data obtained by the color component synthesizing means 0105 is subsequently converted into JPEG compression-encoding means 0106.
Is output to The JPEG compression encoding unit 0106 converts the input color image data composed of red, blue, and green components into color image data composed of luminance and color difference components, and performs JPEG compression encoding.

The JPEG compressed data compressed by the JPEG compression encoding means 0106 is temporarily stored in the memory 0107. The data is read from this memory in accordance with the timing of transmission to an external device or printing, and output to the JPEG decompression / decoding means 0108 at the subsequent stage. J
The PEG decompression / decoding means 0108 decompresses the JPEG compressed data and outputs it as color image data.

The color image data wI output from the JPEG expansion / decoding means 0108 is input to the printer 0109. The printer 0109 converts the input color image data into CMYK color components, further performs halftone processing and the like, and then outputs the print data pwI on a print medium such as paper.

It is to be noted that the printed matter pwI may be subjected to geometric editing such as rotation, or an attack such as copying by a copier, by hands other than the user of the apparatus.

A printed matter having a possibility that the printed matter is deformed is designated as pwI '. The printed matter pwI ′ is digitized again by using a scanner 2001 shown in FIG.

The overall flow of each means described above is
This will be described with reference to the flowchart shown in FIG.

First, in step 3102, the image data I is input to the color component extracting means 0101. This includes a step of reading a photograph or a printed matter with a scanner or the like and generating image data. Further, the blue component is separated and used for inputting a registration signal in the subsequent stage.

Next, a registration signal is generated in step 3103, and the registration signal is embedded in step 3104. This step 31
The registration signal embedding process in 04
Registration signal embedding means 01 in FIG.
02 corresponds to the process executed inside, and will be described later in detail.

In step 3111, the pattern arrangement is determined by the pattern arrangement determining means 0110. Printer 01
09 according to the output resolution and the resolution of the image.
Determine the pattern sequence to be used for embedding Inf.

Further, a mask is created in step 3105, and the created mask is input to step 3106, and defines the relationship between the embedding bit information and the embedding position. In step 3107, the pattern arrangement determined in step 3111 is input, and the mask is expanded to an enlarged mask with reference to this pattern arrangement. The detailed description of the mask / pattern arrangement correspondence means will be described later.

In step 3108, additional information Inf is embedded in the image data in which the registration signal is embedded in steps 3103 and 3104. This additional information embedding process repeatedly embeds additional information Inf in the entire image in macroblock units. This processing will be described later in detail with reference to FIG. Here, a macroblock indicates a minimum embedding unit, and one complete additional information is added to an image area corresponding to the macroblock.
All information of Inf is embedded.

In step 3109, the image data in which the additional information Inf is embedded is JPEG-compressed and stored in the memory 0107, and after JPEG decompression and decoding, the image data is output from the printer 0109 as a printed matter pwI.

[2 Digital Watermark Extraction Apparatus] Next, an outline of a digital watermark extraction apparatus according to the present embodiment will be described.

FIG. 2 shows a digital watermark extracting apparatus according to the present embodiment. As shown in FIG. 2, the digital watermark extracting apparatus includes a scanner 0201, a color component extracting unit 0202, a registration unit 0203, and an additional information extracting unit 0203.
204.

First, the printed matter pwI ′ is set on the document table of the digital watermark extracting apparatus, and the scanner 0201 sets the printed matter pwI ′.
By scanning I ', digitized image data wI' is generated. As described above, the above printed matter
pwI 'has the possibility to be different from the printed matter pwI in FIG.

The image data wI 'has been attacked to cause various geometric distortions on the image data wI. The attack includes scaling, rotation, printing & scanning, and the like, but in the case of the present embodiment, an attack involving at least one printing & scanning is performed.

Therefore, it is ideal that the contents of the image data wI ′ and wI are the same, but in fact, the contents of the two image data are often significantly different.

The color component extracting means 0202 outputs the image data w
After inputting I ′ and extracting the blue component, the image data of the blue component is output to the registration means 0203 at the subsequent stage. In the image data wI ′, the red component and the green component other than the blue component are unnecessary and are discarded here.

The registration means 0203 stores the blue component image data obtained by the color component extraction means 0202.
wI ₁ 'is input. And this blue component image data wI
Image data wI corrected for geometric distortion using ₁ '
Generates ₂ '.

As described above, the image data wI 'may have a different scale from the image data wI, whereas the image data wI ₂ ' always has the same scale as the image data wI. The reason and the image data wI ₂ ′
Details of the processing for making the scale the same as wI will be described later.

The additional information extracting unit 0204 can extract the additional information Inf embedded in the image data wI ₂ ′ by performing a predetermined process according to the embedding method of the additional information embedding unit 0103. And outputs the extracted additional information Inf.

The overall flow of each means described above is
This will be described with reference to the flowchart of FIG. First, in step 3202, image data wI 'is input. This image data wI ′ is obtained by scanning image data expected to be a printed matter pwI ′ with a scanner. In general, image data wI 'is significantly different from image data wI.

Further, only the blue component of the image data wI 'is extracted and used for the next step. Next, in step 3203, the input blue component image data wI ₁ ′
Is corrected. This scaling process is
This is a process executed inside the registration unit 0203 of FIG. 2, and will be described later in detail.

In the next step 3211, step 32
03 using the scaling rate output from
The pattern arrangement used for embedding Inf is determined.

In step 3204, the offset of the input blue component image data wI ₁ ′ is corrected.

Next, an extraction process using the first pattern array is performed in step 3206, and an extraction process using the second pattern array is performed in step 3205, from the image data wI ₂ ′ whose scale and offset have already been corrected. The embedded additional information Inf is extracted.

In the statistical test step 3207, the likelihood of the extracted additional information Inf is calculated and determined, and if it is determined that the additional information Inf is not correct, the process proceeds to step 32.
Returning to step 02, an image in which the additional information Inf is considered to be embedded is input again. On the other hand, sufficiently accurate additional information In
If it is determined to be f, the additional information Inf is extracted by the comparison processing in step 3208. In step 3210, the information indicating the certainty is displayed as a reliability index D described later.

The above-described pattern arrangement determination processing, offset adjustment processing, extraction processing using the first pattern arrangement,
The extraction processing using the second pattern array, the statistical test processing, and the comparison processing are performed by the additional information extraction unit 02 in FIG.
03, which will be described later in detail.

[3 Detailed Description of Each Part] Next, each part will be described in detail.

First, the registration means 0203 on the digital watermark extraction side and the processing called registration executed in step 3203 will be described.

The registration process is a pre-process for extracting the additional information Inf from the image data wI ′ input to the digital watermark extracting device when extracting the digital watermark information. In general, the term “registration processing” has a meaning including not only the scale alignment processing but also the alignment processing. However, in the present embodiment,
The positioning process uses the position information embedded as a part of the additional information Inf.
4 will be described.

In the following, first, what kind of change the image data that has been subjected to the printing process will be considered. And
Consider a registration process for such a change, and consider a registration process for a printing system.

Consider this embodiment in which image data wI is printed by a YMCK ink jet printer, and the printed matter is scanned by a scanner.

At this time, if the output resolution of the printer is different from the input resolution of the scanner, the original color image data wI and the image data w
The scale of I 'will be different. Therefore, there is little possibility that the digital watermark information can be accurately extracted as it is from the obtained image data wI ′. Therefore, it is necessary to provide a means for correcting the difference between these scales.

In this embodiment, since both the input resolution and the output resolution are known, the scale ratio can be calculated from these ratios. For example, when the output resolution is 600 dpi and the input resolution is 300 dpi, the scale ratio between the image before printing and the image after scanning is twice. Therefore, according to the calculated scale ratio, the image data wI ′
Is scaled. As a result, the image data wI
And the image size represented by the image data wI ′ can be set to the same scale.

However, the output and input resolutions are not always known in all cases. If both resolutions are not known, the above method cannot be used. In this case,
In addition to the means for correcting the difference between the scales, means for further knowing the scale ratio is required.

An image obtained by subjecting the image data wI to printing-related processing becomes an image as shown in FIG. 3 after being input by scanning with a scanner. In FIG. 3, 0
The entire image 301 is an image represented by the image data wI ′. The image data 0301 includes an original image 0302 represented by the image data wI and a white margin portion 0303. Such a margin becomes inaccurate if the user cuts it out with a mouse or the like. The position adjustment processing for the position shift due to the scanning is performed in the offset adjustment processing in the additional information extraction processing 0204.

The above points are considered to always occur in the image representing the image data wI ′ obtained through the printing system, and the image data wI is subjected to the printing system processing. If so, these need to be resolved.

As described above, a case has been described in which image data is obtained at least once through a printing process before extracting a digital watermark. However, such a situation may occur due to artificial editing.

In the following, assuming that the input / output resolution ratio is unknown, a description will be given of the registration signal embedding means and the registration means provided to solve the problem of the difference in scale.

[3-1 Registration Signal Embedding Processing] First, the registration signal embedding means 0
Details of step 102 (step 3104) will be described.

Registration signal embedding means 01
02 is located before the additional information embedding means 0104. This means 0102 embeds a registration signal referred to for registration of the image data wI 'in the registration means of FIG. 2 in the original image data in advance. This registration signal is embedded in image data (in this embodiment, the blue component of the color image data) that is hardly visible to human eyes as digital watermark information.

FIG. 4 shows the internal configuration of the registration signal embedding means 0102. The registration signal embedding unit 0102 is a block division unit 04 of FIG.
01, Fourier transform means 0402, adder means 0403,
Inverse Fourier transform means 0404, block synthesis means 040
5 is comprised. Hereinafter, details of each means will be described.

The block dividing means 0401 divides the input image data into a plurality of non-overlapping blocks. The size of this block is 2 in this embodiment.
To the power of. Actually, other sizes can be applied. However, when the size of the block is a power of two, high-speed processing can be performed in the Fourier transform unit 0402 coupled after the block dividing unit 0401.

The blocks divided by the block dividing means 0401 are divided into two sets I ₁ and I _2.
I ₁ is input to the subsequent Fourier transform means 0402, and I ₂ is input to the subsequent block synthesis means 0405. In the present embodiment, as I ₁ , _one of the blocks obtained by the block dividing means 0401 is selected as the one located closest to the center of the image data I, and all the remaining blocks are I ₂ Is selected as

This is because the present embodiment can be realized by using at least one block, and the processing time can be shortened when the number of blocks is small. However, the invention is not limited thereto, including the scope may select two or more blocks as I _1.

Further, information on what size the block is divided into and which block is selected as a registration signal embedding target needs to be shared between the digital watermark embedding device and the digital watermark extracting device.

A part I ₁ of the image data obtained by the division by the block dividing means 0401 is input to the Fourier transform means 0402.

[0081] Then the Fourier transform unit 0402 performs a Fourier transform to the image data I ₁ input. The original data form of the input image data I ₁ whereas called a spatial region, called the data form after being Fourier transformed frequency domain. The Fourier transform is applied to all input blocks. In the present embodiment, since the size of the input block is a power of 2, fast Fourier transform is used to speed up the processing.

The fast Fourier transform means that the Fourier transform is n
× n times the amount of computation required, but (n / 2) log
_{2 This} is a conversion algorithm that can be executed with an operation amount of (n). Here, n is a positive integer. The fast Fourier transform and the Fourier transform differ only in the speed of obtaining the operation result, and the same result is obtained from both. Therefore, in this embodiment, the fast Fourier transform and the Fourier transform will not be separately described.

The frequency domain image data obtained by the Fourier transform is represented by an amplitude spectrum and a phase spectrum. Of these, only the amplitude spectrum is input to the adding means 0403. On the other hand, the phase spectrum is input to the inverse Fourier transform unit 0404.

Next, the adding means 0403 will be described. A signal r called a registration signal is separately input to the addition means 0403 together with the amplitude spectrum. FIG. 5 shows an example of the registration signal.
The impulse signal as shown in FIG.

In FIG. 5, the two obtained by Fourier transform
The amplitude spectrum of the dimensional spatial frequency components is shown. The center is a low frequency component, and the periphery is a high frequency component.
Reference numeral 0501 denotes an amplitude spectrum of a signal component included in an original image component. In a signal corresponding to a natural image such as a photograph, many large signals are concentrated in a low band. On the other hand, there is almost no signal in the high frequency range.

Although the present embodiment is described on the assumption that a series of processing is performed on a natural image, the present invention is not limited to this, and a document image, a CG image, and the like may be similarly processed. However, the present embodiment is particularly effective when processing a natural image having a relatively large intermediate density.

FIG. 5 shows a signal 0501 originally possessed by a natural image.
To the horizontal and vertical Nyquist frequency components of the frequency domain signal, impulse signals 0502, 0503, 0504,
This is an example of the present embodiment with the addition of 0505. As in this example, the registration signal is preferably an impulse signal. This is because it is easy to extract only a registration signal in a digital watermark extraction device described later.

Although an impulse signal is added to the Nyquist frequency component of the input signal in FIG. 5, the present invention is not limited to this. That is, it is sufficient that the registration signal is not removed even when the image in which the digital watermark information is embedded is attacked. As described above, a lossy compression method such as JPEG compression has an effect like a low-pass filter. Therefore, even if the impulse signal is embedded in the high-frequency component to be compressed,
It may be removed by the expansion process.

On the other hand, embedding an impulse in a low-frequency component has a disadvantage that it is more likely to be perceived as noise from human visual characteristics than embedding in a high-frequency component. Therefore, in the present embodiment, the impulse signal is embedded at an intermediate-level frequency that is higher than the first frequency that is difficult for human eyes to recognize and lower than the second frequency that is not easily removed by the irreversible compression / decompression processing. Shall be. The registration signal is added to the addition means 0
It is added to each block (one block in this embodiment) input to 403.

The adding means 0403 outputs a signal obtained by adding the registration signal to the amplitude spectrum of the image data in the frequency domain to the inverse Fourier transform means 0404.

The inverse Fourier transform means 0404 performs an inverse Fourier transform on the input frequency domain image data. This inverse Fourier transform is applied to all input blocks. As in the case of the above-described Fourier transform means 0402, the size of the input block is a power of 2, so that the fast Fourier transform is used to speed up the processing. The signal in the frequency domain input to the inverse Fourier transform means 0404 is converted into a signal in the spatial domain by inverse Fourier transform and output.

The image data of the spatial domain output from the inverse Fourier transform means 0404 is combined with the block combining means 0405.
Is input to

The block combining means 0405 performs a process reverse to the division performed by the block dividing means 0405. As a result of the processing of the block combining unit 0405, the image data (blue component) is reconstructed and output.

The details of the registration signal embedding means 0102 shown in FIG. 1 have been described above.

In FIG. 4, the method of embedding the registration signal in the Fourier transform domain has been described. On the other hand,
A method of embedding a registration signal in a spatial domain is also conceivable. This method will be described with reference to FIG.

FIG. 29 is composed of block dividing means 2901, adding means 2902, block combining means 2903, and inverse Fourier transform means 2904.

The block dividing means 2901 and the block synthesizing means 2903 correspond to the block dividing means 04 in FIG.
01 and the same operation as the block synthesis means 0405. The image data input to the registration signal embedding unit 0102 is first input to the block division unit 2901 and is divided. The block obtained here is input to the adding means 2902. On the other hand, the registration signal r is input to the inverse Fourier transform means 2904, and is converted into a signal r 'by inverse Fourier transform processing. here,
The registration signal r is a signal in the frequency domain as in the case shown in FIG. The adding means 2902 includes
The block from the block dividing means 2901 and the signal r 'from the inverse Fourier transform means 2904 are input and added respectively. The signal output from the adding means 2902 is input to the block synthesizing means 2903, and the image data (blue component)
Is reconstructed and output.

The means configuration of FIG. 29 performs the same processing as that of the means configuration of FIG. 4 in the spatial domain. Compared with the means configuration of FIG. 4, high-speed processing can be performed because no Fourier transform means is required.

Further, in FIG. 29, the signal r 'is a signal independent of the input image data I. Therefore, the calculation of the signal r ′, that is, the processing of the inverse Fourier transform unit 2904 does not need to be executed every time the input image data I is input, and
It is possible to generate r '. In this case, the inverse Fourier transform means 2904 can be eliminated from the means configuration of FIG. 29, and the registration signal can be embedded at a higher speed. The registration processing with reference to the registration signal will be described later.

{Patchwork Method} In the present embodiment, a principle called a patchwork method is used for embedding the additional information Inf. Therefore, the principle of the patchwork method will be described first.

In the patchwork method, embedding of the additional information Inf is realized by causing a statistical bias to an image.

This will be described with reference to FIG. In FIG. 30, reference numerals 3001 and 3002 denote subsets of pixels, respectively.
Reference numeral 3003 denotes the entire image. Two subsets A 3001 and B 3002 are selected from the entire image 3003.

In the method of selecting the two subsets, the embedding of the additional information Inf by the patchwork method according to the present embodiment can be executed if they do not overlap each other. However, the size and selection method of these two subsets are determined by the tolerance of the additional information Inf embedded by the patchwork method, that is, the additional information Inf when the image data wI is attacked.
Has a great effect on the strength to prevent loss. This will be described later.

Now, the subsets A and B are A = {a ₁ ,
a ₂ ,..., a _N }, and B = {b ₁ , b ₂ ,..., b _N }. Each element a _i , b _i of the subset A and the subset B is a pixel value or a set of pixel values. In the present embodiment, it corresponds to a part of the blue component in the color image data.

Here, the following index d is defined.

D = 1 / NΣ (a _i −b _i )

This indicates the expected value of the difference between the pixel values of the two sets.

When an appropriate subset A and subset B are selected for a general natural image and an index d is defined, there is a property that d ≒ 0. Hereinafter, d is referred to as a reliability distance.

On the other hand, as an operation of embedding each bit constituting the additional information Inf, an operation of a ′ _i = a _i + c b ′ _i = b _i −c is performed. This is an operation of adding a value c to all the elements of the subset A and subtracting c for all the elements of the subset B.

Here, as in the case described above, the additional information In
The subset A and the subset B are selected from the image in which f is embedded, and the index d is calculated.

Then, d = 1 / NΣ (a ′ _i −b ′ _i ) = 1 / NΣ {(a _i + c) − (b _i −c)} = 1 / NΣ (a _i −b _i ) + 2c = 2c, and does not become zero.

That is, when a certain image is given, by calculating the reliability distance d for the image, if d ≒ 0, the additional information Inf is not embedded, while d is a fixed amount from 0. If the values are distant from each other, it can be determined that the additional information Inf is embedded.

The above is the basic concept of the patchwork method.

By applying the principle of the patchwork method, in this embodiment, a plurality of bits of information are embedded. In this method, the method of selecting the subset A and the subset B is also defined by the pattern arrangement.

In the above-described method, the embedding of the additional information Inf is realized by adding or subtracting the elements of the pattern arrangement with respect to the predetermined elements of the original image.

FIG. 9 shows an example of a simple pattern arrangement. FIG. 9 is a pattern array showing the amount of change in pixel value from the original image when referring to 8 × 8 pixels to embed one bit. As shown in FIG. 9, the pattern array includes an array element having a positive value, an array element having a negative value, and an array element having a value of 0.

In the pattern shown in FIG. 9, the position indicated by the + c array element indicates the position at which the pixel value at the corresponding position is increased by c, and is a position corresponding to the subset A described above. On the other hand, the position indicated by the -c array element indicates a position at which the pixel value of the corresponding position is reduced by c, and is a position corresponding to the subset B described above. The position indicated by 0 indicates a position other than the subsets A and B described above.

In this embodiment, the number of array elements having a positive value is equal to the number of array elements having a negative value so as not to change the overall density of the image. That is, the sum of all array elements in one pattern array is 0. Note that this condition is indispensable for the operation of extracting the additional information Inf described later.

Using the pattern arrangement as described above, additional information
An operation of embedding each bit information constituting Inf is performed.

In the present embodiment, a plurality of bits of information, that is, additional information Inf, are embedded by arranging the pattern of FIG. 9 a plurality of times in different areas of the original image data to increase / decrease the pixel value. In other words, in different regions of one image, not only the combination of the subsets A and B, but also the subsets A 'and B', the subsets A "and B",
By assuming a plurality of combinations,..., The additional information Inf including a plurality of bits is embedded.

In this embodiment, when the original image data is large, the additional information Inf is repeatedly embedded. This is due to the fact that the patchwork method uses statistical properties, and therefore requires a sufficient number of statistical properties to appear.

In this embodiment, when embedding a plurality of bits, a relative position at which the pattern arrangement is used is determined in advance by the mutual bits in order to avoid overlapping the areas where the pixel values are changed using the pattern arrangement. I do. That is, additional information
The relationship between the position of the pattern array for embedding the information of the first bit constituting Inf and the position of the pattern array for embedding the information of the second bit is appropriately determined.

For example, if the additional information Inf is composed of 16 bits, the positional relationship of the 8 × 8 pixel pattern arrangement of each of the 1st to 16th bits indicates the image quality on an area having a size larger than 32 × 32 pixels. It is given relatively so that deterioration is reduced.

Further, when the image data is large, the additional information Inf (each bit information constituting it) is embedded as many times as possible. The purpose of this is to enable each bit of the additional information Inf to be correctly extracted. In particular, in the present embodiment, since the statistical measurement is performed using the fact that the same additional information Inf is repeatedly embedded, the above-described repetition is important.

The selection of the embedding position as described above
This is executed by the embedding position determining means 0103 in FIG.

Next, how to determine the subset A and the subset B will be described.

[3-2 Pattern Arrangement Determining Means] In the patchwork method, the way of deciding the subset A and the subset B largely depends on the attack resistance of the additional information Inf and the image quality of the image in which the additional information Inf is embedded.

In this embodiment, the image data wI in which the additional information Inf is embedded in FIG. 1 and subjected to JPEG compression and decompression are printed out by a printer, input by the scanner 0201 in FIG. wI '
Becomes Various attacks including printing and scanning have been applied during the process of obtaining the image data wI and wI ′.

The following describes how the additional information Inf embedded by the patchwork method can be made resistant to printing attacks.

In the patchwork method, the shape of the pattern array and the magnitude of the element values are parameters that determine the trade-off between the embedding strength of the additional information Inf and the image quality of the image data wI. Therefore, whether or not the additional information Inf can be extracted after the above-described attack is performed can be optimized by operating this parameter. This is explained in more detail.

[0131] In this embodiment, fixed by matrix exemplified in FIG. 9 the basic positional relationship between the elements b _i of the elements of the subset A a _i and subset B in patchwork method is shown.

The elements a _i and b _i are not limited to one pixel value, but may be a set of a plurality of pixel values.

A plurality of the pattern arrays are assigned so as not to overlap in the image, and each assigned pixel in the image is changed based on the value of the element of the pattern array.

If a subset of pixels whose image is changed to a positive value (+ c) of the pattern array is A and a subset of pixels whose image is changed to a negative value (−c) of the pattern array is B, a patchwork It can be seen that the principle of the law is applied.

In the following description, a group of pixels having a positive value (+ c) in the pattern array (corresponding to the position of the element a _i of the subset) has a positive patch and a negative value (−c). A group of pixels (corresponding to the position of the element b _i of the subset) is called a negative patch.

Hereinafter, a positive patch and a negative patch are sometimes used without distinction. In this case, the patch refers to either a positive patch or a negative patch.

As the size of each patch of the pattern arrangement, an example of which is shown in FIG. 9, increases, the value of the reliability distance d in the patchwork method increases, so that the tolerance of the additional information Inf increases and the additional information Inf increases. The image after embedding is greatly degraded in image quality from the original image.

On the other hand, when the value of each element of the pattern array becomes smaller, the tolerance of the additional information Inf becomes weaker, and the image quality after embedding the additional information Inf is not so deteriorated from the original image.

As described above, the size of the pattern arrangement shown in FIG. 9 and the patch elements (±
Optimizing the magnitude of the value of c) is very important for the durability and image quality of the image data wI.

First, consider the size of the patch. When the size of the patch is increased, the additional information Inf embedded by the patchwork method becomes stronger, while when the size of the patch is reduced, the additional information Inf embedded by the patchwork method becomes weaker. This is due to the lossy compression and the fact that the printing system has the effect of a low-pass filter as a whole process. When the size of the patch increases, the signal weighted to embed the additional information Inf is embedded as a signal of a low frequency component, while when the size of the patch decreases, the signal weighted to embed the additional information Inf becomes It will be embedded as a high frequency component signal.

The additional information Inf embedded as a high-frequency component signal may be erased by being subjected to a printing-related process, to a process like a low-pass filter, and being erased. On the other hand, there is a high possibility that the additional information Inf embedded as a signal of the low frequency component can be extracted without being erased even if a printing process is performed.

As described above, in order for the additional information Inf to have resistance to attack, it is desirable that the size of the patch be large. However, increasing the size of the patch is equivalent to adding a low frequency component signal to the original image,
This leads to an increase in image quality deterioration in the image data wI. This is because human visual characteristics have VTF characteristics as shown by 1301 in FIG. 13 in FIG.
As can be seen from 301, human visual characteristics are relatively sensitive to low frequency component noise, but relatively insensitive to high frequency component noise. Therefore, the size of the patch is determined by the additional information In embedded by the patchwork method.
It is desirable to optimize to determine the intensity of f and the image quality in the image data wI.

Next, the value (± c) of the patch will be considered.

The value of each element (± c) constituting the patch is called “depth”. If you increase the depth of the patch,
The additional information Inf embedded by the patchwork method has a higher resistance, whereas if the depth of the patch is reduced, the additional information Inf embedded by the patchwork method has a lower resistance.

[0145] The depth of the patch is closely related to the reliability distance d used for extracting the additional information Inf. The reliability distance d is a calculated value for extracting the additional information Inf, which will be described in detail in the extraction process. Generally, when the depth of the patch is increased, the reliability distance d increases. That is, it is easy to extract the additional information Inf. On the other hand, when the depth of the patch is reduced, the reliability distance d is reduced, and the extraction is difficult.

As described above, the depth of the patch is determined by the additional information Inf.
Are important parameters for determining the image quality of the image in which the additional information Inf is embedded, and the intensity of the additional information Inf is preferably optimized. By always using an optimized patch size and depth, it is possible to embed additional information Inf that has resistance to attacks such as irreversible compression or printing and reduces image quality degradation. is there.

Next, the specific depth and size of the patch used in this embodiment will be described.

To simplify the explanation, the processing in the printing system will be simplified. As an example of printing-related processing, consider gradation conversion by halftone processing.

The halftone process is a change in the method of expressing a gradation as described above. Before and after halftone processing, human vision perceives gradation in the same manner. However, input means such as scanners do not have the vague perception of humans and do not necessarily "perceive" the gradation before and after.

That is, the scanner alone does not recognize whether or not the gradation expressed by the area gradation actually has the gradation information expressed by the original density gradation. Next, let us consider what kind of halftone processing can be performed next to express the gradation expressed by the density gradation by the area gradation.

First, the relationship between the density gradation and the area gradation by halftone processing will be considered.

FIG. 43 shows an example of the relationship between a 4 × 4 dither matrix and the gradations that can be expressed by the matrix. FIG.
In, the matrix is a gray scale expressed by area gray scales, and the gray scales expressed by the matrix are indicated by numbers below.

A 4 × 4 matrix has 16 pixels. By turning on / off these 16 pixels, 4 × 4 +
1 = 17 gradations can be expressed.

Generally, halftone processed mx
The n dots can represent (m × n + 1) gradations.

This will be described with reference to the example shown in FIG. In FIG. 44, it is assumed that the pixel 4403 is represented by a density gradation having a dynamic range of 0 to 16, and the value is 8. Four pixels having the same value as this pixel are arranged vertically and horizontally to generate a block 4402 having a size of 4 × 4. Halftone processing is performed on this block using an appropriate dither matrix having a size of 4 × 4,
The binary data 4403 is generated. The binarized data is transmitted to the printer and output. Thereafter, the image is input again at the same resolution as the output resolution of the printer by an image input device such as a scanner. At this time, assuming that the output resolution of the dots of the printer and the input resolution at which the scanner reads the pixels have a 1: 1 relationship, the pixels output by the printer and input by the scanner are equal to the binary data 4403. . The image data thus generated is the binary data 4404. Binary data 4404
, The binary data is scaled to 1 / (4 × 4) by a method using an appropriate interpolation process, and multi-value data 4405 is generated. This multi-level data has a value of 8. If the resolution is not high enough for the scanner to determine the binary data 4403 as the binary data 4404, the binary data 4403 is optically converted to the multi-value data 4405.
Is converted to

As described above, when the gradation information expressed by the density gradation using FIG. 44 is converted into the area gradation and then expressed again by the density gradation, the state where the gradation information is correctly propagated is shown. explained. Generally, halftone processing is performed using an area gradation such that one pixel is represented by m × n pixels.
The gradation information is propagated by performing an interpolation process such that the binary data of × n pixels becomes one pixel.

In this embodiment, the size and depth of the patch used for embedding the additional information Inf are determined by the above-described relationship between the area gradation and the density gradation in order to provide resistance to attacks including printing and scanning. Consider the design. In the present embodiment, a case is considered where images of various sizes are output to a predetermined size by a printer.

FIG. 45 shows two images 4501 and 4504 having different image resolutions, each having the same size 4.
Reference numerals 503 and 4506 denote output by a printer. FIG. 45A shows a case where the resolution of the image is low.
(B) is a series of processing when the resolution of the image is high.

First, an image 4501 and an image 4504
Is subjected to enlargement processing so that one pixel corresponds to one dot. At this time, interpolation such as the nearest neighbor method is used as the enlargement method. Note that the nearest neighbor method is a method of performing enlargement by copying the same pixel value to neighboring pixels. (If the image has a very high resolution, reduction (thinning out) may be considered.) As a result, the image 4501 is enlarged to the image 4502 and the image 4504 is enlarged to the image 4505. Thereafter, the printed matter (print image data) such as 4503 and 4506 is represented by dots by halftone processing.

In the actual processing of the printer, CMYK conversion processing, color matching, and the like are performed, but are omitted here for the sake of simplicity.

As shown in FIG. 45, the lower the resolution of the image,
It can be seen that pixels can be represented by many dots, and that the higher the resolution of an image, the more pixels must be represented by fewer dots.

Next, it will be shown that the influence of the embedding by the patch is transmitted even when the density gradation is converted to the area gradation. Here, in order to make the explanation easy to understand, the influence by the resolution of the image is considered.

In FIG. 46, reference numerals 4601 and 4605 denote image areas (subset A) operated by a positive patch when the additional information Inf is embedded in a certain image, and before the halftone processing.

In FIG. 46, reference numerals 4603 and 4607 denote image areas (subset B) operated by negative patches when embedding the additional information Inf in a certain image, and before the halftone processing.

In FIG. 46, reference numerals 4601 and 4603 indicate the cases where the additional information Inf is not embedded by the patch, and reference numerals 4605 and 4607 indicate the cases where the additional information Inf is embedded.

At this time, 4601, 4603, and 460
5, 4607 correspond to one dot per pixel immediately before halftone processing is performed.

By the halftone processing, 460 in FIG.
The images shown at 1, 4603, 4605, 4607 are
4602, 4604, 4606, and 4608 are represented by dots as area gradation.

When the additional information Inf is not embedded, the difference between the number of ink dots in 4602 and the number of ink dots in 4604 can be generally said to be almost the same. When the image is large and the average value of the difference between the ink dots is obtained for each patch, the difference is substantially zero.

On the other hand, when the additional information Inf is embedded, the difference between the number of ink dots in 4606 and the number of ink dots in 4608 is considered to appear.

Even when the additional information Inf is expressed by area gradation, it is possible to control the increase or decrease of the ink dots by designing the patch. It can be said that the patchwork method can be resistant to printing and scanning attacks.

From FIG. 46, it is intuitively imagined that the number of ink dots increases with an increase in the area in which the patches are embedded, and that the number of ink dots increases with an increase in the depth of the patches. it can.

FIG. 4 shows the relationship between the number of patches and the number of ink dots.
7 will be described. FIG. 47 is a diagram showing changes in ink dots depending on the size and depth of a patch.

In FIG. 47, the horizontal axis represents the coefficient value of the dither matrix for performing the halftone processing on the subset A or the subset B in which one pixel is enlarged to one dot, and the vertical axis represents the coefficient value of the dither matrix. Shows the appearance frequency of. At the same time, for the sake of simplicity, the horizontal axis shows the average of the pixel values of the subset A or the subset B in which one pixel is enlarged to one dot, which is subjected to halftone processing.

As shown in FIG. 47, in general, when the coefficient value of the dither matrix corresponds to a large subset A or a subset B, it is considered that the values have almost no bias and the appearance frequencies are almost equal. Good.

Assuming that the average 4703 of the pixel values before embedding has changed to the average 4704 of the pixel values after embedding due to the embedding of the additional information Inf, the dither matrix binarization process allows only the diagonal line portion 4702 to form the ink dots. It can be seen that the number increases.

That is, it can be seen that the depth of the patch and the increased number of ink dots are in a proportional relationship.

When the size of the patch is increased, the frequency of appearance of the coefficient value of the dither matrix further increases, so that the area 4702 of the hatched portion increases in the frequency of appearance, and the patch depth and the ink dot It can be seen that the number of increase is also in a proportional relationship.

In consideration of the above properties, (1) the embedding depth is proportional to the number of dots on the printed matter in the entire image. (2) The size of the patch is proportional to the number of dots on the printed matter.

That is, if the difference in the number of dots in the entire image between the area where the positive patch of the entire image is embedded and the area where the negative patch is embedded in the entire image is Δβ = 2 × α × PA × C + γ (Equation 47-1).

Α is a proportional coefficient, γ is a margin, C is an embedding depth, and PA is an area corresponding to one dot per pixel of the entire positive or negative patch image. Where α, Δβ,
γ is a number determined by experiment.

The principle of (Equation 47-1) is applicable not only to halftone processing using a dither matrix, but also to the error diffusion method since the above (1) and (2) hold.

(Equation 47-1) does not consider the resolution of an image, the output resolution of a printer, and the input resolution of a scanner. Hereinafter, a case where the resolution of an image, the output resolution of a printer, and the input resolution of a scanner have changed will be considered.

In this embodiment, the input resolution of the scanner is 600 pp which is a sufficient resolution in the flatbed scanner to hold as much information as possible.
Fix to i.

Next, the output resolution of the printer and the resolution of the image will be considered.

As described above with reference to FIG. 45, when printing an image, the number of dots expressing the density gradation of one pixel is as follows.
Determined by the resolution of the image. An example is shown below.

For example, in FIG. 45, the image 4501 is 1
It is assumed that the image has 000 × 800 pixels.

When this image is output by a printer having an output resolution of 1200 dpi in both the main scanning direction and the sub-scanning direction so that the long side falls within 5 inches, one pixel becomes one dot before the halftone processing. Enlargement processing is performed, and the number of output dots on the long side is 1200 dpi × 5 = 600.
0 dot. Therefore, image 4501 is 6000 × 4
The image is enlarged to an image 4502 having 800 pixels. When reproducing the gradations of the images 4503 to 4501 subjected to the halftone processing, one pixel is represented by 6 × 6 dots.

On the other hand, the image 4504 is 3000 × 2400
It is assumed that the image has pixels.

When this image is output by a printer having the same resolution of 1200 dpi so that the long side falls within 5 inches, the image is similarly converted into 6 dots so that one pixel becomes one dot.
An image 4506 that has been enlarged to an image 4505 having 000 × 4800 pixels, and then subjected to halftone processing
become. One pixel of the image 4504 is represented by 2 × 2 dots.

Since the density of the ink dots is considered to be fixed, when one pixel is represented by 5 × 5 dots, the dynamic range of the density gradation that can be represented by one pixel is large. On the other hand, when represented by 2 × 2 dots,
The dynamic range of density gradation that can be expressed by one pixel is small.

As will be described in detail in the additional information extracting process, in the patchwork method, at the time of detection, in each pattern array unit, (sum of pixel values of a region where a positive patch is embedded) − (pixel value of a region where a negative patch is embedded) Is calculated, and the average value of each pattern array unit is obtained for the entire image. This average value is referred to as a reliability distance d. The greater the reliability distance d, the more reliable additional information can be extracted.

FIG. 48 schematically shows the difference between the positive and negative patch areas in the pattern arrangement unit. FIG.
48A shows a case where the image resolution is low, 4801 is a positive patch area, 4802 is a negative patch area, and FIG. 48B is a case where the image resolution is high, 4803 is a positive patch area.
Reference numeral 4 denotes a negative patch area.

Since each ink dot has a fixed density, each pixel 4801 and 4802 is composed of many ink dots, and the reliability distance d has a dynamic range up to a large value. On the other hand, in 4803 and 4804, since one pixel is represented by a small number of ink dots, the reliability distance d does not have a dynamic range up to a large value.

In general, when one pixel is composed of a small number of dots (when the image resolution is high), a large reliability distance d cannot be obtained because the dynamic range of the gradation of one pixel is small. There may be cases where the additional information Inf cannot be extracted.

Therefore, when the resolution of the image is high, the area of the patch is increased or the embedding depth (± c)
Need to be larger.

In general, when outputting at a high resolution, positional displacement is a serious problem. Therefore, it is preferable to increase the area of the patch.

Assuming that the number of dots required to detect additional information per pattern array is Δβp, the number of pixels of a positive or negative patch is P, the embedding depth C of the patch and the number m × n of dots representing one pixel. Is expressed by (Expression 47-1), Δβp = 2 × α ′ × P × (m × n) × (C / 255) + γ ′ (Expression 47-2).

Here, (m × n) × (C / 255) indicates that even if the embedding depth C is changed to a maximum of 255 gradations, the maximum number of dots per pixel is only m × n. I have.

Here, α ′ is a proportional coefficient, and γ ′ is a margin.

The number of dots for reproducing one pixel, m × n
Is obtained by the following equation: m × n = (printer output resolution in the main scanning direction / image resolution) × (printer output resolution / image resolution in the sub-scanning direction), and m × n becomes smaller as the image resolution becomes higher. Become.

Therefore, when Δβp, α ′, and γ ′ are obtained by experiments, the embedding depth per pattern array, the patch size (the size of the pattern array), and the embedding depth required for detecting the additional information Inf are determined. It can be obtained from the output resolution of the printer and the resolution of the image.

Based on the above considerations, a method of changing the embedding depth (± c) and the size of the patch according to the resolution of the image is proposed.

Hereinafter, the device will be specifically described.

The output resolution of the printer and the resolution of the image are input to the pattern arrangement determining means 0110 of FIG. 1, and a pattern arrangement suitable for extracting the additional information Inf is output.

As an example, if the output resolution of the printer is 12
Consider a case where the image is printed at 00 dpi and the long side of the image is printed with a size of about 6 inches. 300p for the image
pi to 600 ppi (long side 1800 pixels to 360
Assume that there is an image of (0 pixel).

The pattern array used for embedding the additional information Inf is set to a resolution of 500 p according to the resolution of the image.
In the case of less than pi, the pattern array 49 in FIG.
01, and when the resolution of the image is 500 ppi or more, the pattern array 4903 in FIG. 49 is used.

It is assumed that the embedding depth is appropriately determined using (Equation 47-2).

The pattern arrangement is appropriately determined in this way by the pattern arrangement determining means 0110, and the pattern arrangement determining means 0110 is provided in the subsequent embedding position determining means 0103.
The embedding position is determined based on the size of the pattern array input from. Further, the additional information embedding unit 0104 embeds the additional information Inf into the image according to the embedding position of the pattern array input from the embedding position determining unit 0103.

On the other hand, the additional information Inf cannot be extracted unless the pattern arrangement used for embedding is known.
Therefore, the resolution of the image is determined from the scaling ratio output from the registration unit 0203 using the pattern arrangement determination unit 2001 described later.

When the output resolution of the printer is fixed and the resolution of the image is known from the scaling ratio, the pattern arrangement determining means 2001 can determine the pattern arrangement used for embedding.

Therefore, even when the patch or pattern arrangement is made variable depending on the resolution of the image, additional information In can be obtained by using information obtained from the registration signal.
Extraction of f is possible.

[3-3 Embedding Position Determination Processing] FIG. 11 shows the internal configuration of the embedding position determination means 0103.

The mask creating means 1101 shown in FIG. 11 creates a mask for defining the embedding position of each bit information constituting the additional information Inf. The mask is a matrix including position information that defines a relative arrangement method of a pattern arrangement (see FIG. 9) corresponding to each bit information.

An example of a mask is shown at 1701 in FIG.
Coefficient values are assigned inside the mask, and each coefficient value has an equal frequency of appearance in the mask.
If this mask is used, it is possible to embed additional information Inf consisting of up to 16 bits.

Next, the mask reference means 1102 reads the mask created by the mask creation means 1101, associates each coefficient value in the mask with information on the bit number of each bit information, and stores each bit information. The arrangement method of the pattern array for embedding is determined.

Further, mask / pattern arrangement correspondence means 110
3 develops array elements (for example, 8 × 8 size) of each pattern array input from 0110 in the preceding stage at the position of each coefficient value in the mask. That is, each coefficient value (1 cell) of the mask shown by 1701 in FIG.
It is made × 8 times and can be referred to as the embedding position of each pattern array.

The additional information embedding means 0104 described below
Each bit information is embedded using the pattern size with reference to the embedded head coordinates 1702 in FIG.

In this embodiment, the mask forming means 1
Each time image data (blue component) is input to 101, the mask is created. Therefore, when inputting image data of a large size, the same additional information is repeated several times.
You will embed Inf.

In the above method, when extracting the additional information Inf from the image, the configuration of the mask (array of coefficient values) plays a key role. That is, there is an effect that only the key owner can extract information.

Note that the present invention also includes the case where a mask created in advance is stored in the internal storage means of the mask creating means 1101 and is called up when necessary, without creating a mask in real time. In this case, it is possible to shift to the subsequent processing at high speed.

Next, details of each processing executed in the embedding position determining means 0103 will be described.

[3-3-1 Mask Creation Means] First,
The mask creation unit 1101 will be described.

In embedding the additional information Inf using the patchwork method, when information is embedded by applying a large operation to the pixel value in order to strengthen attack resistance (for example, when the value of c in the pattern array is set large). In the image represented by the original image data, the deterioration of the image quality is relatively inconspicuous at the so-called edge portion where the pixel value changes rapidly,
In a flat portion where the change in pixel value is small, the portion where the pixel value is manipulated becomes noticeable as noise.

FIG. 13 shows the spatial frequency characteristics perceived by the human eye. The horizontal axis shows the spatial frequency (Radial Frequency), and the vertical axis shows the visual response value. FIG. 13 shows that when the pixel values are manipulated and information is embedded, image quality degradation is conspicuous in a low frequency region where human eyes can perceive sensitively.

For this reason, in the present embodiment, the pattern corresponding to each bit is arranged in consideration of the characteristics of the blue noise mask and the cone mask which are usually used for the binarization processing of the multi-valued image.

Next, the characteristics of the blue noise mask and the cone mask will be briefly described.

First, the characteristics of the blue noise mask will be described.

The blue noise mask has such a characteristic that it becomes a blue noise pattern even if it is binarized at any threshold value. This blue noise pattern is a pattern showing a frequency characteristic in which the spatial frequency is biased toward a high frequency region.

FIG. 37 shows a part of a certain blue noise mask.

In FIG. 14, reference numeral 1401 denotes a threshold value of 10 and 2
FIG. 3 shows a schematic diagram of a spatial frequency characteristic of a digitized blue noise mask.

The horizontal axis of 1401 is the Radial Frequency, which indicates the distance from the origin (DC component) when the blue noise mask is Fourier transformed. The vertical axis is Power sp
ectrum, and is a value obtained by averaging the sum of squares of the amplitude components at the distance indicated by the radial frequency on the horizontal axis. In the figure, a two-dimensional frequency characteristic of an image is converted into a one-dimensional graph to make it easier to understand visually.

Comparing with FIG. 13, it can be understood that the blue noise mask is hardly perceived by human eyes because the high frequency component is biased. Therefore, in the case of an ink-jet printer or the like, when expressing the gradation of a multi-valued image by the area gradation using dots, the spatial frequency component is biased toward a high frequency by using a blue noise mask, so that it is noticeable to human eyes. It is known that the area gradation can be expressed without any area.

Next, an example of the process of generating a blue noise mask will be described below.

1. Generate white noise 2. Perform low-pass filtering on the binary image P _gl of gray level g (the initial value is a white noise mask) to generate a multi-valued image P ′ _gl 3. Gray level g (initial Value: 127) is compared with the low-pass filtered image P ′ _gl (multi-valued),
Invert the black and white pixels of the binary image P _g to obtain a binary image P _{gl + 1} 4. Repeat steps 2 and 3 until the error is minimized.
The binary image P _gl (the initial value is a white noise mask) is gradually changed to a binary image P _g (a blue noise mask) of a gray level g (an initial value: 127) 5. The gray level g + 1 (g The binary black (white) point of -1) is given to a random position, and a few operations are repeated, and P
_{get g + 1} (P _g-1 )

By repeating the above operation, a blue noise mask is generated for all gradations, and a dither matrix is generated.

For example, in a 32 × 32 blue noise mask, four points increase (decrease) for each gradation.

However, at this time, since the black (white) bit determined by the previous gray level g cannot be inverted in order to have 256 gray levels, the limiting condition becomes severe at low or high gray levels, and randomness with poor uniformity is obtained. There is a disadvantage that only a pattern can be obtained.

FIG. 12 shows an appearance frequency distribution (histogram) 1201 of each coefficient constituting the blue noise mask.
In FIG. 12, all the values (coefficients) of 0 to 255 exist in the mask in the same number.

A technique in which the above-mentioned blue noise mask is used for binarizing a multi-valued image is well known.
Opt.Soc.Am A / Vol.9, No.11 / November 1992 Digital ha
lftoning technique using a blue-noise mask Tehopha
no Mitsa, Kevin J. Parker "and the like.

Next, the characteristics of the cone mask will be described.

When each coefficient included in this cone mask is binarized, a periodic mask is formed on the spatial frequency domain representing the obtained binary information as shown by 1402 in FIG. Or that a quasi-periodic peak occurs
One feature. However, it is designed so that a peak does not stand in a low frequency region.

FIG. 38 shows a part of a coefficient array of a certain cone mask.

When the cone mask is binarized at any threshold value, an appropriate distance is maintained between dots, so that a peak does not occur in a low frequency region.

FIG. 14 shows a schematic diagram 1402 of the spatial frequency characteristic when binarization is performed with the threshold value 10 of the cone mask. Similar to the spatial frequency characteristic of the blue noise mask of 1401, the characteristic of 1402 shows that the low frequency component is small.

In the case of a cone mask, a peak is generated from a frequency higher than the low-frequency band of the blue noise mask regardless of whether the threshold is low or high.
The number of portions densely arranged at the embedding position is smaller than that of the blue noise mask. Therefore, there is an advantage that the embedding noise generated when the additional information Inf is embedded is less noticeable than the blue noise.

The frequency of use of the coefficients constituting the cone mask is also the same as that of the blue noise mask.
An appearance frequency distribution (histogram) indicated by 01 is obtained.

Therefore, in association with the coefficients of this mask,
If a pattern corresponding to each bit information constituting the additional information Inf is embedded in the image data, the same number of patterns corresponding to each bit information can be arranged in this image data, and as a result, Information Inf can be embedded in a well-balanced manner.

In this embodiment, a cone mask is used as the embedding reference mask from the above advantages.

[3-3-2 Mask Reference Means] The mask (cone mask) created by the mask creation means 1101 is
It is input to the mask reference means 1102.

The mask reference means 1102 determines the embedding position by associating the embedding position of the N-bit information to be embedded in the image with the mask number (pixel value).

A method for determining an embedding position performed by the mask reference means 1102 will be described.

Although the above-described cone mask is used in this embodiment, in order to make the explanation easy to understand, FIG.
A description will be given using a 4 × 4 mask shown in 1501 of FIG.

The mask shown in FIG. 15 has 4 × 4 coefficients, and the coefficient values from 0 to 15 are arranged one by one. The embedding position of the additional information Inf is referred to using the 4 × 4 mask. In the case of the mask used in this description, the additional information Inf consisting of a maximum of 16 bits can be embedded. Hereinafter, a case where the additional information Inf of 8 bits is embedded will be described.

First, the structure of additional information Inf will be described with reference to FIG. Additional information Inf as in the figure, consists of a start bit Inf ₁ and use information Inf _2.

The start bit Inf ₁ recognizes that the position where the actual additional information Inf is embedded is shifted from the ideal position, and the digital watermark (additional information)
In order to correct the extraction start position of Inf), it is used by the offset matching means 2003 included in the digital watermark extraction device side. Details will be described later.

Further, the usage information Inf ₂ contains the original additional information,
That is, the information is actually used as additional information of the image data I. This information includes, for example, the ID of the apparatus shown in FIG. 1 or the ID of the user if the purpose is to track the cause in the event of unauthorized use of the image data wI. If the printed matter of the image data wI is to be prohibited from being copied, control information indicating that copying is prohibited is included.

In this embodiment, the start bit is 5 bits, and a bit string “11111” is used. However, the present invention is not limited to this, and it is also possible to use a bit number other than 5 bits in the additional information Inf as a start bit, and similarly, it is also possible to use a bit string other than “11111”. However, the bit number and bit sequence of the start bit need to be shared between the digital watermark embedding device and the digital watermark extracting device.

A simple example of embedding 8-bit additional information Inf of 5 start bits and 3 use information using the cone mask composed of 4 × 4 coefficients as described above will be described.

However, the present invention is not limited to this. For example, the present invention can be applied to a case where a total of 69 bits of additional information Inf including a start bit of 5 bits and use information of 64 bits is embedded using a 32 × 32 cone mask.

In the additional information Inf, the start bit is 5 bits “11111” and the usage information is 3 bits “010”. The first has bit information of 1st, the 2nd has 1, the 3rd has 1, the 4th has 1, the 5th has 1, the 6th has 0, the 7th has 1, and the 8th has 0 bit information.

A pattern corresponding to each of these bits (see FIG. 9)
Are assigned to positions corresponding to the respective coefficients of the cone mask, and each pixel value of the original image data is changed by ± c according to this positional relationship. Thereby, for the original image data of the size corresponding to one cone mask,
One piece of additional information Inf is embedded.

In the present embodiment, a certain threshold value is determined based on the minimum number of bits required to embed the additional information Inf, and the threshold value is determined at a position in the cone mask where a coefficient less than this threshold value is located. , And embeds corresponding bit information. As a result, 1 regardless of the number of bits of the additional information Inf
One piece of additional information Inf is embedded in one cone mask.

It should be noted that the present invention is not limited to the above-described method, and the corresponding bit information may be embedded at a position where a coefficient equal to or greater than a certain threshold value is arranged, and the threshold value may be determined based on this. good.

Next, in the present embodiment, the ratio of the number of coefficients equal to or less than the threshold value used for embedding to the number of coefficients of the entire mask will be referred to as the embedding filling rate.

In order to correctly embed the 8-bit additional information Inf an integer number of times, the threshold for determining which coefficient to use as the embedding reference position in the mask 1501 in FIG. 15 needs to be 8 or 16. The optimum threshold value is determined in consideration of the effect on tolerance and image quality.

Here, when the threshold value of the mask is 8, the filling factor is 50%. That is, 50% of the original image data to be compared with the mask is subjected to the processing using the pattern arrangement of FIG.

An example of the correspondence between each bit information and the coefficients in the mask is shown in Table 1.

[0268]

[Table 1]

Here, S1 to S5 are bit information (start bits) for alignment used in the offset alignment processing device. 1 to 3 are 3-bit usage information.

According to the correspondence in the correspondence table 1, 160 in FIG.
Each bit information is a pattern at the pixel position of the input image data corresponding to the position of the coefficient (0 to 7) represented by 1 (FIG. 9).
(See). The correspondence between the order of the bit information to be embedded and the coefficient values in the mask is a part of the key information, and it is not possible to extract each bit information without knowing this correspondence. In this embodiment, in order to simplify the description, coefficient values from 0 to a threshold value are sequentially set as shown in Table 1.
It is assumed that S1 to S5 correspond to three bits of usage information.

Next, the filling rate in the case of actually embedding using a cone mask of 32 × 32 size will be described a little. Note that the processing procedure is the same as when the mask 1501 is used.

First, a threshold value necessary for correctly embedding the additional information Inf an integer number of times is determined in consideration of the deterioration of the image quality at the time of embedding.

Further, in order that each bit constituting the additional information Inf is embedded with the same number of repetitions, the number of coefficients equal to or smaller than the threshold is divided by the number N of bits constituting the additional information Inf.
Determine how many times each bit can be embedded with one mask size.

For example, in the original image data corresponding to the coefficient values of 0 to 255, the start bit 5
When embedding 69-bit additional information Inf consisting of bits and usage information of 64 bits, the threshold value is set to 137, for example.

In this case, the number of effective coefficient values in the mask is 138. Since the number of bits required to represent one piece of additional information Inf is 69, each bit information can be embedded 138/69 = 2 times in one mask size.

When the embedding position is determined by using the cone mask, embedding is performed for all points having a coefficient value equal to or less than a certain threshold value because the low frequency component of the spatial frequency has no peak. This is to take advantage of the characteristics of the mask.

When the embedding position is determined as described above, and the embedding filling rate is 50% and the embedding information amount is 69 bits, each bit information constituting the additional information Inf and each coefficient value constituting the cone mask are obtained. Table 2 shows the relationship
It becomes like.

[0278]

[Table 2]

Here, S1 to S5 are start bits,
This is bit information for alignment used in the offset alignment processing device. 1 to 64 are usage information.

However, the present invention is not limited to this correspondence.
If the bit information is sequentially embedded in all the positions of the coefficients from to the threshold (or from the threshold to 255) using the pattern of FIG. 9, the correspondence between each bit information and each coefficient value is different. There may be.

In the case of a 32 × 32 cone mask, 1
Four positions each having the same coefficient exist in one mask.

In the case where each bit information is embedded in the original image data based on the above-mentioned correspondence table 2 for all coefficients, 32 × 3
In the case of a cone mask having a large size such as 2,64 × 64, each bit information constituting the additional information Inf is embedded approximately the same number of times. Also, the same bit information is spread and embedded in the original image data.

In the patchwork method, the embedding position is conventionally randomly selected, but in the present embodiment, the same effect can be obtained by referring to the cone mask, and the image quality is less deteriorated.

As a result, the mask reference means 1102 sets the coordinates (x, y) of the embedding position corresponding to each bit information.
Get.

When the information is represented by an array S [bit] [num] = (x, y), in the case of the correspondence table 1, the bits are start bits S1 to S1.
S5 and 1 to 3 bits of usage information. Num is the order assigned to each coefficient that appears repeatedly in the cone mask. (x, y) contains the relative coordinates in the mask.

The above operation is performed by the mask reference means 1102.

[3-3-3 Mask / Pattern Arrangement Corresponding Means] The embedding position of each bit information in the cone mask obtained by the mask reference means 1102 is input to the mask / pattern arrangement corresponding means 1103.

Since the embedding position determined by the mask reference means 1102 is the position of each bit information pattern (for 8 × 8 pixels), the addition area (+ c) and the subtraction area shown in FIG. It is necessary to assign (-c) and other (0). For this reason, an operation of developing an 8 × 8 size pattern array corresponding to FIG. 9 at all positions of the cone mask referred to by the mask reference unit 1102 is performed by the mask / pattern array correspondence unit 1103.

Specifically, the coordinates of the array S [bit] [num] = (x, y) obtained by the mask reference means 1102 are multiplied by the horizontal size of the pattern array, and y An operation of multiplying the coordinates by the vertical size of the pattern array is performed. As a result, the coordinates 1701 in the mask in FIG. 17 are the start coordinates 1702 in which one pixel in the mask is enlarged into one pattern array.
Becomes

Using the pattern arrangement shown in FIG. 19 from the start coordinates, the area 1 having the size of the pattern arrangement
703 can be embedded without overlapping.

The coordinates (x, y) change to coordinates (x ', y'), but the bits and num of the array S [bit] [num] do not change.

Accordingly, it is possible to embed a plurality of pieces of bit information by setting the additional information Inf corresponding to the bits of the array S [bit] [num] to (x ', y') as the head position where the pattern array is embedded.

Note that the mask / pattern arrangement correspondence means 11
A large mask in which each coefficient of the cone mask is developed (enlarged) into an 8 × 8 pattern array by 03 is referred to as an enlarged mask.

The size of the enlarged mask is (32 × 8)
× (32 × 8) size, which is the additional information In
This is the minimum image unit (macroblock) necessary to embed at least one f.

In this embodiment, the size of the pattern arrangement is not limited to 8 × 8, as described in the pattern arrangement determining means 0110. In the present invention, the pattern arrangement and the patch size are determined by the pattern arrangement
At 103, selection (optimization) is made according to the image resolution or the output resolution of the printer. Even when the size of the pattern array is not 8 × 8, the mask / pattern array correspondence unit 1103 can determine the embedding position of the additional information Inf by using the same unit.

The above is the mask / pattern arrangement correspondence means 11
03.

In general, a small mask has a smaller degree of freedom in the dot arrangement position at the time of creation than a large mask, and it is difficult to create a mask having desired characteristics such as a cone mask. For example, when the additional information Inf is embedded by repeatedly assigning a small mask to the entire image data, the spatial frequency of the small mask appears in the entire image data.

On the other hand, since the complete additional information Inf is extracted from one mask, it is possible to extract additional information Inf from partial image data wI ′ by setting the size of the mask large. Probability) will be smaller. Therefore, it is necessary to determine the size of the mask in consideration of the balance between the cutout resistance and the image quality deterioration.

The above is the description of the embedding position determining means 010 in FIG.
3 is the process performed.

[3-4 Additional Information Embedding Processing] The additional information embedding means 010 shown in FIG. 1 is referred by referring to the embedding position of each bit information in the image data determined as described above.
No. 4 actually embeds the additional information Inf.

FIG. 10 shows the flow of the operation of the process of repeatedly embedding the additional information Inf.

In the method shown in FIG. 10, a plurality of macroblocks that can be allocated to the entire image are allocated, and the bit information of the first bit is repeatedly embedded in all the macroblocks. The third bit is repeatedly embedded. This is a procedure in which if there is bit information that has not been subjected to the embedding processing, the embedding processing of 1001 to 1003 is performed on all the unprocessed macroblocks.

However, the present invention is not limited to this order, and the internal and external relations of the two loop processes may be reversed. That is,
If there is an unprocessed macroblock, the procedure may be changed to a procedure for embedding all the bit information that has not been embedded yet.

Specifically, when the additional information Inf is embedded, when the bit information to be embedded is “1”, the pattern arrangement shown in FIG. 9 is added. If the bit to be embedded is "0", the pattern arrangement in FIG. 9 is reduced, that is, the value obtained by inverting the sign of FIG. 9 is added.

The above addition / subtraction processing is realized by switching control of the switching means 1001 in FIG. 10 according to the bit information to be embedded. That is, when the bit information to be embedded is "1", it is connected to the adding means 1002, and when the bit information is "0", it is connected to the subtracting means 1003. These processes 1001 to 1003 are performed with reference to the bit information and the information of the pattern arrangement.

FIG. 19 shows how one of the bit information is embedded. FIG. 9 shows an example in which the bit information to be embedded is "1", that is, a pattern array is added.

In the example shown in FIG. 19, I (x, y) is the original image,
P (x, y) is an 8 × 8 pattern array. Each coefficient constituting the 8 × 8 pattern array is superimposed on the original image data (blue component) having the same size as the pattern array,
The value at the same position is added or subtracted. As a result, I ′ (x, y) is calculated and output to the color component synthesizing unit 0105 of FIG. 1 as blue component image data in which bit information is embedded.

The above-described addition / subtraction process using the 8 × 8 pattern array is repeatedly performed for all embedding positions (positions to which pattern arrays for embedding each bit information are allocated) determined in the above-mentioned correspondence table 2. .

In this embodiment, as described in the description of the pattern arrangement determining means 0110, the size of the pattern arrangement is not limited to 8 × 8. In the present invention, the pattern arrangement and the patch size are selected and determined (optimized) by the pattern arrangement determining unit 0110 according to the image resolution or the output resolution of the printer. Even when the size of the pattern array is not 8 × 8, the additional information embedding unit 0104 can embed the additional information Inf by using a similar unit.

Next, FIG. 18 shows how the internal loop processing of FIG. 10 is performed.

In FIG. 18, in order to repeatedly embed each bit information, the entire image data 1801 (1803) is
The macro block 1802 is repeatedly allocated and embedded from upper left to lower right in raster order (1001 to 1 in FIG. 10).
003).

[0312] The above operation is performed by the additional information embedding means 0104, and the additional information Inf is embedded in the entire image.

By the above processing, the additional information Inf is embedded in the image data. If each pixel of the image data in which the additional information Inf is embedded is expressed by a sufficiently small number of dots, the size of the pattern array becomes sufficiently small. Only human eyes can perceive it. Therefore, the spatial frequency characteristics of the cone mask are also maintained, and are hardly visible to human eyes.

[3-5 Compression and Decompression of File] In this embodiment, after the additional information is embedded by the additional information embedding means 0104, the file is compressed, stored in the memory, and expanded.

Here, a description will be given of a digital watermark embedding design method in consideration of compression of an image in which a digital watermark is embedded.

[3-5-1 JPEG Compression Coding] FIG.
FIG. 9 is a diagram showing visual chromaticity spatial frequency characteristics. Each curve uses a spatial sine wave pattern consisting of white-black (monochromatic) and a red-green or yellow-blue color pair of opposite brightness, and changes the period and contrast of the spatial sine wave pattern. The above pattern when it is made to occur can be obtained by measuring a limit that can be recognized by human eyes.

In FIG. 39, white-black (light / dark information)
, The sensitivity becomes maximum at about 3 [cycle / deg], but the chromaticity (red-green and yellow-blue) becomes maximum at about 0.3 [cycle / deg].

From this, it can be seen that the light and dark information is sensitive to the discrimination of a fine portion such as the resolution of an image, and that the chromaticity affects the appearance of a spatially wide (low spatial frequency) portion.

It can also be seen that the yellow-blue pattern is not involved in discriminating spatial information finer than the red-green pattern.

As described above, in the method of embedding digital watermark information by directly modulating a grayscale image having only a luminance component, the deterioration of image quality is more conspicuous than the method of embedding digital watermark information in the color components of color image data. You can see that. Also, it can be said that the method of embedding digital watermark information in the blue component (B) for color image data of RGB is the least noticeable to human eyes.

When a change is made to the color component in order to embed digital watermark information in the color component, the color is noticeable as color unevenness when viewed by a human in a spatially wide area (in a state where the spatial frequency is low). When viewed by a human in a narrow area (in a state where the spatial frequency is high), it becomes less noticeable than when embedding digital watermark information in luminance.

In this embodiment, a grayscale image having only one type of element per pixel is converted into color image data having a plurality of elements per pixel, and then subjected to digital watermark information (additional information Inf). Etc.)
This has the effect of preventing the image quality from deteriorating as compared with the case where the digital watermark information is embedded with the normal gray scale.

A comparison between a case where digital watermark information is embedded in grayscale image data and a case where digital watermark information is embedded only in one of a plurality of types of elements constituting color image data indicates that image output at high resolution is possible. In the case of performing the operation (for example, in the case where the value of one pixel is expressed by gradation with a small number of ink dots), the latter can maintain better image quality.

However, there is a demerit that the output color image data (file size) is about three times as large as the original image data when simply considered.

Therefore, in the present embodiment, in order to minimize the file size, the JPEG compression encoding means 01
At 06, JPEG compression encoding is further performed on the image data in which the digital watermark information is embedded.

In general, JPEG compression coding is a technique for reducing the amount of data by utilizing components of human visual perception and removing components that are insensitive to human visual perception. On the other hand, the digital watermark technology is a technology for embedding information in a component to which human vision is not sensitive. Therefore, JPEG compression encoding and digital watermarking technology are difficult technologies to coexist, and JPEG compression encoding is considered to be a type of attack on digital watermarking information.

[0327] A method for imparting resistance to JPEG compression encoding will be briefly described below.

The pattern arrangement as shown in FIG. 9 used in the present embodiment is set so that additional information already embedded in the color image data is not lost by sub-sampling to the color difference components and quantization processing. ing.

First, the JPEG compression encoding method will be briefly described.

[0330] The color image data input to the JPEG compression encoding apparatus 0106 includes luminance (Y) and chrominance (Cr, Cb).
Is converted to When color image data composed of the original red component (R), green component (G), and blue component (B) is input, Y = 0.29900 × R + 0.58700 × G + 0.1
1400 x B Cr = 0.50000 x R-0.41869 x G-
0.08131 × B Cb = −0.16874 × R−0.33126 × G +
Using the formula of 0.50000 × B, the original color image data is format-converted into another color image data composed of luminance (Y) and color difference (Cr, Cb).

Image data decomposed into a luminance component and a chrominance component is 8 × 8 shown in FIG. 40 in raster order from the upper left of the image.
It is divided into blocks of pixels. In JPEG compression encoding, the compression encoding process is repeated for each 8 × 8 block.

Next, the sampling process of the color components in the JPEG compression encoding will be described.

In the JPEG compression encoding, sampling of the color difference component is performed for each 8 × 8 pixel in accordance with the sampling ratio of the sampling option.

FIG. 41 shows a state of sampling of image data. Hereinafter, a procedure of 4: 2: 2 sampling in JPEG compression encoding will be described.

Reference numeral 4101 denotes a luminance component of 4 × 4 pixels. Since there is a lot of visually important information for the luminance component, the thinning process is not performed and 4 × 4 pixels are output as they are.

4103 is a color difference component of 4 × 4 pixels (Cr,
Cb). Since the chromaticity component is not so visually sensitive, one pixel is thinned out for two pixels in the horizontal or vertical direction. As a result, the color difference components (Cr, Cb) of 4 × 4 pixels are converted into 4 × 2 pixels 4104. By performing the above sampling, the color difference components of 8 × 8 pixels are reduced to 8 × 4 pixels.

Therefore, as a result of 4: 2: 2 sampling, the luminance component Y, chrominance component Cr, and chrominance component Cb for 8 × 8 pixels are 8 × 8 pixels, 8 × 4 pixels, and 8 × 4 pixels, respectively. become. DCT (Discrete Cosine Transform) operation, quantization, zigzag scan, Huffman coding, and the like are performed on each of the sampled pixels by a known procedure.

In addition, by utilizing the fact that human visual characteristics are not so sensitive to high-frequency components, the number of quantization steps of DCT coefficients into high-frequency components is reduced, thereby efficiently compressing them. In addition, quantization is performed so that the number of quantization steps for the chrominance component is smaller than that for the luminance as a whole.

A pattern arrangement having resistance to the above-described compression encoding processing will be considered.

FIG. 42 shows the pattern arrangement of FIG. 9 described above again. In FIG. 42, an area 4201 having a positive element of + c is a positive patch, and an area 42 having a negative element of -c.
02 is referred to as a negative patch. At this time, in each patch, in the minimum coding unit 4001 composed of 8 × 8 pixels shown in FIG. 40, JPEG compression resistance can be enhanced by biasing information to low frequency components. However,
The present invention is not limited to this, and various settings may be made with the minimum coding unit being 16 × 16 pixels.

Also, 4: 1: 1 (the color difference component is thinned out every other pixel in the vertical and horizontal directions), 4: 2: 2 (the color difference component is
(Or thinning out every other pixel in the horizontal direction) When sampling is performed, each patch is vertically and /
Alternatively, if the size has a width that is an integral multiple of two pixels in the horizontal direction, resistance to sampling can be enhanced.

That is, (1) Each patch uses a minimum coding unit (8 × 8 pixels) which is biased toward a low frequency.

(2) The size of each patch is 2 × N (N is an integer) pixels in the vertical and / or horizontal direction according to the sampling method.

In each area (8 × 8 pixels) to be subjected to JPEG compression encoding, in order for each patch to have a low-frequency component, a position on an image to which a pattern array is assigned,
And each size of the pattern array (8 × 8 pixels in FIG. 9)
Is preferably synchronized with each region to be encoded.

That is, (3) the size of the pattern array and the embedding position are synchronized with the unit size subjected to JPEG compression encoding.

In consideration of the above conditions, if the additional information Inf is embedded using a pattern arrangement as shown in FIG. 9, for example, the digital watermark information (additional information Inf) can be obtained even after JPEG compression encoding. It can be left in image data and can be said to be resistant to JPEG compression encoding.

In the present invention, the color component extracting means 0101 directly converts a gray scale (monochromatic) image into Y (luminance).
The category includes a case where modulation is performed in which the information is converted into Cr and Cb (color difference) components and the additional information Inf or the like is embedded as an electronic watermark only in the Cb component. In this case, the conversion into the luminance and chrominance components need not be performed by the JPEG compression encoding means, and the number of processing steps is reduced.

The color component extracting means 0101 converts Y (yellow), M (magenta), C (cyan), K
The category also includes a case where modulation is performed by directly converting to a (black) component and embedding the additional information Inf or the like as a digital watermark only in the Y component among the components. In this case, the step of converting the color components immediately before the printing unit can be omitted.

That is, the present invention relates to a case where the additional information Inf or the like is embedded in a part of all components constituting one pixel, not limited to the blue component, the Cb component, and the Y component. Include in category.

[3-5-2 Memory storage] The above JPEG
The encoded data obtained by the compression encoding is temporarily stored in a memory. The encoded data is read out from the memory to the JPEG decompression decoding unit 0108 in accordance with the timing of transmission to an external device or the timing of printing by a printer connected to the subsequent stage of the apparatus in FIG.

It should be noted that, as in the present embodiment, the grayscale image data is once converted to color image data, the blue component is modulated, further converted to color image data consisting of luminance and color difference components, and JPEG-compressed. In the case of coded data, the original grayscale image is directly converted into color image data consisting of luminance and chrominance components and compared with the amount of coded data when JPEG compression coding is performed. Although an increase occurs, there is an effect that it does not lead to a large increase in the memory capacity.

That is, assuming that JPEG compression encoding is performed after the digital watermark information is embedded in the original image data, the digital watermark on the grayscale image data as in the present embodiment is considered. The method of embedding information (additional information Inf, etc.) can improve the image quality without increasing the overall data amount so much as compared with the method of embedding digital watermark information by directly modulating normal grayscale image data. There is an advantage that it can be measured.

[3-5-3 JPEG Decompression Decoding] JP
The EG decompression decoding unit 0108 reads the encoded data from the memory 0107 in accordance with the timing of transmission to an external device or printing by the printer 0109, and decodes the color image data using the reverse procedure of the above-described compression method. Become

[3-6 Registration Processing] Next, the registration means 0203 in FIG. 2 provided in the digital watermark extraction device will be described in detail.

The registration means 0203 is a means located before the additional information extraction means 0204.
This is a pre-process of the additional information Inf extraction process. The registration means 0203 includes a preceding color component extraction means 0202.
The image of the blue component extracted by is input.

In the registration means 0203, the image data wI output from the digital watermark embedding device and
The difference in scale of the image data wI ′ input to the electronic watermark extraction device is corrected.

FIG. 7 shows details of the registration means 0203. As shown in FIG. 7, the registration unit 0203 includes a block division unit 0701, a Fourier transform unit 0702, an impulse extraction unit 0703, a scaling ratio calculation unit 0704, and a scaling unit 0705.

[0358] The block dividing means 0701 performs the same block dividing processing as the above-described registration signal embedding means 0102 (block dividing means 0401). By this processing, generally, the registration signal embedding unit 0102 (the block division unit 040)
It is difficult to extract the same block as in 1). This is because the size of the image data wI in which the digital watermark information is embedded is changed and the position of the image data wI is further deviated by performing a printing process.

However, there is no problem even if the extraction of this block differs to some extent. This is because the registration signal is embedded in the amplitude spectrum of the image data in the digital watermark embedding device. The amplitude spectrum has a property that it is not affected by a positional shift in the spatial domain of the image data. Therefore, in each of the digital watermark embedding device and the digital watermark extracting device,
The blocks divided by each block dividing means are
There is no problem even if some displacement occurs in the spatial region.

The block dividing means 0701 outputs the image data obtained by dividing the block to the Fourier transform means 0702. The Fourier transform unit 0702 converts the image data in the spatial domain into image data in the frequency domain, as in the case of the registration signal embedding unit 0102 described above. The Fourier-transformed image data in the frequency domain is represented by an amplitude spectrum and a phase spectrum. Of these, only the amplitude spectrum is the impulse extraction means 0703
Is input to On the other hand, the phase spectrum is discarded.

The image data converted to the frequency domain is input to the impulse extracting means 0703. The impulse extracting means 0703 extracts only an impulse signal from the image data converted into the frequency domain. That is, 0502, 0503, FIG. 5 already embedded in the image data.
0504 and 0505 are extracted.

This can be performed by using a known image processing technique. For example, it can be realized by performing threshold processing on the image data converted into the frequency domain. This example is shown in FIG.
It is shown in (a). FIG. 8A shows how the amplitude spectrum 0801 input to the impulse extraction means 0703 is subjected to threshold processing using a threshold 0802. For the sake of explanation, the converted image data in FIG. 8 is represented in one dimension. By selecting an appropriate threshold 0802, an impulse signal can be extracted. However, the original image data having the same size as the impulse signal existing in the low band is also extracted at the same time.

FIG. 8 (b) shows a system of the present embodiment which has solved this problem. Image data 0 converted to frequency domain
801 is subjected to a secondary differentiation process. This is equivalent to applying a Laplacian filter or the like. Reference numeral 0803 denotes a result obtained by performing second differentiation on the image data 0801 converted into the frequency domain. By selecting an appropriate threshold 0804 for this data 0803 and performing threshold processing, an impulse signal can be extracted.

The principle of the extraction of the impulse signal will be described in more detail with reference to FIG. In this figure, the processing on the registration signal embedding side described above is also described.

Registration signal embedding means 01
In 02, the image data 2601 in the spatial domain is converted into the image data 2602 in the frequency domain, and the impulse signal 2603 is added in the frequency domain.

Image data in the frequency domain to which the impulse signal (registration signal) 2603 has been added is
The signal 260 in the spatial domain is obtained by the inverse frequency conversion.
Return to 1 'again. Image data 2 returned to the spatial domain again
601 'should have the effect of the addition of the impulse signal, but it is difficult for human eyes to perceive, and the image data 2601 and the image data 2601' substantially look the same. This is because the impulse signal 2603 applied in the frequency domain is distributed with a small amplitude over the entire image data by the inverse Fourier transform.

When an impulse signal such as 2603 in FIG. 26 is applied, it is equal to the addition of image data having a certain frequency component to the spatial domain. If the added impulse signal is greater than the human perceptible frequency and the amplitude is below the human perceptible limit, the added impulse signal is invisible to the human eye. Therefore, it can be said that the embedding of the registration signal itself is a kind of digital watermarking process.

In this embodiment, the image data 260
1, after the registration signal 2603 is embedded and additional information Inf to be actually embedded is embedded,
The signal 2601 ′ in the spatial domain is restored.

The registration signal embedded as shown in FIG. 26 is again subjected to Fourier transform at the time of extraction.
As a result, in the spatial domain, the registration signal 2603 once spread over the entire image data is converted to the frequency domain and appears again as an impulse signal.

The image in which the digital watermark information is embedded is JPEG
When an attack such as irreversible compression such as compression is performed, the impulse is likely to have a small amplitude. On the other hand, when a geometric attack such as scaling is received, the position of this impulse moves. In any case, the impulse signal can be extracted by performing the appropriate impulse extraction processing as described above, and a change from the original image data can be estimated. By correcting this change, a state can be created in which the additional information Inf embedded in the present embodiment can be reliably extracted.

By the above processing, the above-described impulse signal is output from the impulse extracting means 0703 of FIG. 7 and input to the scaling factor calculating means 0704. The scaling ratio calculation unit 0704 calculates what kind of scaling has been performed using the coordinates of the input impulse signal.

[0372] In the digital watermark extracting apparatus of the present embodiment is assumed to know whether embedded impulse signal in advance which frequency component. In this case, it is possible to calculate the scaling ratio based on the ratio between the frequency embedded in advance and the frequency at which the impulse is detected.
For example, the frequency in which the impulse signal is
a, if the frequency of the detected impulse signal is b, a /
It can be seen that the scaling is b times larger. This is a well-known property of the Fourier transform. By the above processing, the scaling ratio is output from the scaling ratio calculation means 0704.

However, the present invention is not limited to this, and information on the position (frequency) in which the registration signal is embedded may be received from the digital watermark embedding device as necessary. For example, a form in which the position information is received as an encrypted signal and the above-described scaling ratio is calculated is also included in the scope of the present invention. By doing so, only the person who knows the registration signal can correctly extract the additional information Inf. In this case, the registration signal can be used as a key for extracting the additional information Inf.

The scaling rate output from the scaling rate calculation means 0704 is input to the scaling means 0705. The scaling means 0705 stores image data
wI ₁ ′ is also input, and the image data wI ₁ ′ is subjected to scaling processing according to the input scaling ratio. Various types of scaling processing, such as bilinear interpolation and bicubic interpolation, can be applied. Then, the scaling unit 0705 outputs the image data wI ₂ ′ that has been subjected to the scaling process.

In this embodiment, the scaling ratio calculating means 0704 outputs the scaling ratio to the pattern array calculating means 0204 in FIG. 1 for another purpose.

One is, as already described, that the image data wI ₁ ′
Although for scaling the image data wI ₂ 'by scaling processing and the other one, the pattern array computing means 0204 in FIG. 2, determining the pattern arrangement used for embedding on the basis of the output scaling factor To do that.

[3-7 Additional Information Extraction Processing] Next, FIG.
From the blue component of the image data wI ′ in which the additional information Inf is embedded by the additional information
The operation of the additional information extracting means 0204 of FIG. 2 for extracting f will be described.

FIG. 20 shows a series of flow of the extraction processing of the additional information Inf.

As shown in FIG. 20, the scaling ratio is input to the pattern arrangement determining means 2001 from the registration processing 0202 in FIG. 2, and the pattern arrangement used for embedding is determined.

[3-7-1 Pattern Arrangement Judging Means] The scaled image data wI ₂ ′ is input into the additional information extracting means 0204. At this time, if the pattern arrangement is determined by the pattern arrangement determining means 0110 of FIG. 1 according to the resolution of the image and the output resolution of the printer, the pattern arrangement is used for embedding without knowing the resolution of the image and the output resolution of the printer. Unable to determine the pattern arrangement. If the pattern arrangement used for embedding is unknown, the additional information Inf cannot be extracted.

In this embodiment, the pattern arrangement judging means 2001 judges the pattern arrangement when the additional information Inf is embedded from the scaling ratio inputted from the registration means 0202.

It is assumed that the input resolution of the scanner when the printed matter pwI 'is scanned is fixed, the output resolution of the printer is known, and means for determining the pattern arrangement from the scaling ratio will be described.

The printed matter pwI ′ is converted into a 6 by the scanner 0201 in FIG.
Scan at an input resolution of 00 ppi
wI 'is obtained.

At this time, the output resolution of the printer is 1200
It is assumed that the input resolution of the scanner is 600 dpi.

[0385] The image data wI 'is stored in the color component extracting means 0202.
To extract color components and output image data wI ₁ ′. This image data wI ₁ ′ is
The scaling ratio and the scaled image wI ₂ ′ are output from the scaling ratio calculating unit 0704 inside the registration unit 0202 to the additional information extracting unit 0204.

The scaling ratio input to the additional information extracting means 0204 is input to the pattern arrangement determining means 2001.

As an example, consider an image wI ′ in which a digital watermark having a scaling ratio of 0.80 is embedded.

It can be seen from the scaling ratio and the input resolution of the scanner that the image was output from the printer at an image resolution of 600 ppi × 0.80 = 480 ppi.

Now, it is assumed that the resolution of the image is 500 ppi, and if the resolution is less than 500 ppi, the pattern arrangement is 4 in FIG.
901 (8 × 8 pattern array)
For 00 ppi or more, the pattern arrangement is 4903 in FIG.
When the correspondence for embedding in (12 × 12 pattern array) is defined in consideration of gradation conversion, an image wI ′ in which a digital watermark having a scaling ratio of 0.80 is embedded.
Shows that embedding was performed using the pattern array 4901 (8 × 8 pattern array) in FIG.

As described above, the pattern arrangement determining means 2
In step 001, the pattern arrangement used for embedding the additional information Inf is determined from the scaling ratio, and output to the embedding position determination unit 2002 in the subsequent stage.

It should be noted that an orthogonal pattern array 4902 corresponding to the pattern array 4901 of FIG. 49 and an orthogonal pattern array 4904 corresponding to the pattern array 4903 are provided.
Is extraction means 200 based on the second pattern arrangement of FIG.
5 used. The orthogonal pattern arrangement will be described later in detail.

[3-7-2 Embedding Position Determination Processing] Next, the embedding position determination means 2002 determines from which region in the image data wI ₂ ′ (blue component) the additional information Inf is extracted. This embedding position determining means 200
2 is the same as that of the above-described embedding position determination means 0103, and
2 are the same.

The additional information Inf is extracted from the determined area using the above-described correspondence table 2 and further using the pattern arrangement shown in FIG.

Here, the extraction of the additional information Inf is realized by convolving the pattern arrangement with the determined area.

When the pattern arrangement is variable according to the output resolution of the image, the pattern arrangement input from the pattern arrangement determining means 2001 in FIG. 20 is used for extracting the additional information Inf. . Hereinafter, FIG.
Only the case where the pattern arrangement input from the pattern arrangement determining means 2001 is 8 × 8 will be described, but the same operation is performed for other pattern arrangements.

[3-7-3 Reliability Distance Calculating Means] The reliability distance d is a calculation value required for extracting embedded information.

FIG. 6 shows a method of obtaining the reliability distance d corresponding to each bit information.

First, the convolution operation means 06 shown in FIG.
01 will be described with reference to FIGS. 21 and 22.

FIGS. 21 and 22 show an example of extracting 1-bit information constituting additional information Inf.

FIG. 21 shows image data (blue component) I ″ (x, embedded with certain 1-bit information constituting additional information Inf.
FIG. 22 shows an example in which the 1-bit information is extracted from the image data I ″ (x, y) in which the 1-bit information is not embedded. This is an example of trying to do so.

In FIG. 21, I ″ (x, y) is image data in which 1-bit information is embedded, and P (x, y) is an 8 × 8 pattern array used for convolution processing (for extracting additional information Inf). Pattern arrangement). Each element (0, ± c) constituting the 8 × 8 pattern array is input image data I ″ (x, y)
Are integrated with the pixel values arranged at the same position, and the sum of each integrated value is calculated. That is, P (x, y) for I '' (x, y)
Is folded. Here, I '' (x, y) is the image data I ′
This is an expression including an image when (x, y) is attacked.
If not attacked, I ″ (x, y) = I ′ (x, y). In the case of an image in which one-bit information is embedded in I ″ (x, y), it is highly likely that a non-zero value is obtained as a result of the convolution as shown in FIG. Especially I '' (x, y) = I '
(x, y) results of convolution when becomes 32c ^2.

In the present embodiment, the same pattern arrangement is used for embedding and the pattern arrangement used for extraction. However, this is not limited in the present invention. In general, the pattern array used for embedding is P (x, y), and the pattern array used for extraction is
In the case of P '(x, y), the relationship can be transformed into the relationship of P' (x, y) = aP (x, y). Here, a is an arbitrary real number, and in the present embodiment, a case where a = 1 will be described for simplicity.

On the other hand, in the example shown in FIG. 22, the same operation as that described above is performed on image data I ″ (x, y) in which 1-bit information is not embedded. As a result of the convolution operation from the original image (corresponding to the image data I), a value of zero is obtained as shown in FIG.

The method for extracting 1-bit information has been described above with reference to FIGS. 21 and 22. However, the above description is a case where the result of the convolution operation is 0 in the image data I in which the additional information Inf is embedded, which is a very ideal case. On the other hand, in the region corresponding to the 8 × 8 pattern arrangement of the actual image data I, the result of the convolution operation is hardly 0.

That is, 8 in the original image (image data I)
The area corresponding to the × 8 pattern arrangement is
Turn arrangement (cone mask is also used as placement information)
When the convolution operation is performed, unlike the ideal,
A value may be calculated. Conversely, additional information Inf is filled
8 × 8 pattern in the embedded image (image data wI)
Similarly, for the area corresponding to the
The result is "32c ^Two"0" instead of "
Sometimes. However, the video that constitutes the additional information Inf
Each piece of cut information is usually embedded multiple times in the original image data.
Is embedded. That is, the additional information Inf is embedded in the image multiple times.
It is rare.

Accordingly, the convolution operation means 0601 obtains the sum of a plurality of convolution operation results for each bit information constituting the additional information Inf. For example, additional information In
If f is 8 bits, 8 sums are obtained. The sum corresponding to each bit information is input to the average calculation means 0602, and each is divided by the number n of repetitions of the pattern arrangement corresponding to each bit information in all macroblocks, and is averaged. This average value is the reliability distance d. That is, the reliability distance d is a value that is determined by majority decision as to which of “32c ² ” and “0” in FIG. 21 is similar.

However, since the reliability distance d was defined as d = 1 / NΣ (ai-bi) in the description of the patchwork method, strictly speaking, the reliability distance d is P ′ (x , y) = 1 / c P (x, y) is the average value of the result of performing the convolution operation. However, even if the convolution operation is performed using P ′ (x, y) = aP (x, y), the average value of the convolution operation result is only a real number multiple of the reliability distance d, Essentially the same effect can be obtained. Therefore, in the present invention, P '(x, y) = aP
The average value of the convolution operation result using (x, y) is calculated as the reliability distance d
It is also possible to use it.

The obtained reliability distance d is stored in the storage medium of 0603.

The convolution operation means 0601 generates the additional information In
The above-mentioned reliability distance d is repeatedly generated for each bit constituting f and stored in the storage medium 0603 sequentially.

The operation value will be described in more detail. The reliability distance d calculated from the original image data I using the pattern arrangement shown in FIG. 9 (see also the cone mask as the arrangement information) is ideally 0. However, in the actual image data I, this value is very close to 0 but many non-zero values. FIG. 23 shows the frequency distribution of the reliability distance d generated for each bit information.

In FIG. 23, the horizontal axis is the value of the reliability distance d generated for each bit information, and the vertical axis is the number of convolutional bit information (the reliability distance) that generates the reliability distance d. d appearance frequency). It can be seen from the figure that the distribution is similar to the normal distribution. Further, in the original image data I, the reliability distance d is not always 0, but the average value is 0 (or a value very close to it).

On the other hand, instead of the original image data I, FIG.
When the image data (blue component) after embedding the bit information “1” is convolved with I ′ (x, y) as described above, the reliability distance d becomes the frequency distribution as shown in FIG. Becomes
That is, as shown in the figure, the distribution is shifted rightward while maintaining the distribution shape of FIG. As described above, in the image data after embedding one bit forming the additional information Inf, the reliability distance d is not necessarily c, but the average value is c.
(Or very close to it).

Note that FIG. 24 shows an example in which bit information “1” is embedded, but when bit information “0” is embedded, the frequency distribution shown in FIG. 23 is shifted to the left.

As described above, when embedding the additional information Inf (each bit information) using the patchwork method,
The statistical distribution as shown in FIGS. 23 and 24 tends to appear more accurately when the number of embedded bits (the number of times the pattern array is used) is increased as much as possible. That is, the accuracy with which it is possible to detect whether or not each bit information constituting the additional information Inf is embedded or whether the embedded bit information is “1” or “0” is increased.

[3-7-4 Offset Adjustment Processing] Next, the configuration of the offset adjustment means 2003 will be described.

[0416] The offset adjustment unit 2003, the image data wI ₂ 'after the implementation of the appropriate scaling input. Thereafter, the start bit is detected using the reliability distance calculation shown in FIG. Note that the offset adjusting means 200
No. 3 generates only five reliability distances corresponding to the five bits of the start bit Inf1. The start bit Inf1 is
As shown in FIG. 36, it is a part of the additional information Inf embedded in advance by the additional information embedding unit 0104, and is 5 bits in the present embodiment.

[0417] Note that these start bits Inf1 are conceptually the first five bits, but do not exist adjacently or densely in an image in which the additional information Inf is embedded, but rather are scattered. ing. This is because they are sequentially embedded in association with each coefficient value constituting the cone mask in the correspondence table 2.

FIG. 28 shows a flowchart of the processing of the offset adjusting means 2003. The following description will be given along the flow of the flowchart in FIG.

[0419] In the offset adjustment unit 2003, it is assumed the image data wI ₂ 'input, in step 2801, firstly the most upper left coordinates embedding start coordinates. At the same time, the maximum value MAX is set to 0. Then, in step 2802, the detection of the start bit is attempted using the reliability distance calculation means of FIG.

[0420] The first to fifth bit information obtained here is used as the correct start bit "1" in step 2803.
1111 "is determined. If this point is the correct embedding start coordinate, five consecutive positive reliability distances d are detected as a detection result, but if not, five positive reliability distances d are often not continuous. The above determinations are sequentially performed, and the position where the correct start bit Inf1 can be detected may be determined as the embedding start coordinate.

However, there may be cases where the correct start bit Inf1 is actually detected at a point other than the embedding start coordinates. This cause will be described with reference to FIG.

FIG. 27 shows the same pattern arrangement (2702, 2705) (cone mask) as used at the time of embedding additional information Inf in order to extract additional information Inf embedded by the patchwork method used in the present embodiment. This also shows how to search for the original macroblock position (2701, 2703, 2704) while performing convolution using (see also arrangement information). It is assumed that the search proceeds continuously from the left figure to the right figure.

In FIG. 27, for simplicity, attention is focused on one macroblock (the minimum unit from which the additional information Inf can be extracted), which is a part of the image data wI2 ′. One cell in this figure is 1
The concept of the size of a pattern array for embedding bit information is shown.

In the left of FIG. 27, when there is a relationship between 2701 and 2702, that is, when 2702 is located at the upper left of the actual macroblock 2701, the positions of the original image and the pattern arrangement for extracting the additional information Inf are: Only the shaded area overlaps.

In the center of the figure, a case is shown in which the search is further advanced, and the position being searched for and the actual macroblock position completely match. In this state, the pattern array to be convolved and the macroblock have the maximum area overlap.

[0426] On the right side of the figure, the position being searched is located lower right than the position of the macroblock in which the additional information Inf is actually embedded. In this state, in this state, the pattern array to be convolved and the macroblock overlap only in the shaded area.

[0427] In all cases in FIG. 27, it is possible to pattern arrangement and the macro block convolution target to extract the correct start bit Inf ₁ if overlap sufficiently. However, these three cases have different overlapping areas, and therefore have different reliability distances d.

The overlapping area can be replaced with the reliability distance d described above. That is, if the positional relationship between the pattern arrangement to be convolved and the macroblock completely match, the reliability distance d of each bit information becomes very close to the above-mentioned ± 32c ² .

Therefore, in the present embodiment, FIG.
As in, if it is determined not correct start bit Inf ₁ in step 2803, step 2807
Moves to the next search point in raster order. On the other hand, when it is determined that the correct start bit Inf _1, in step 2804, the sum of the reliability distance d corresponding to 5 bits you think that the start bit Inf ₁ is whether greater than the maximum value MAX judge. If it is smaller than the maximum value MAX, it moves to the next search point in raster order in step 2807. On the other hand, when the sum of the reliability distances d corresponding to the 5 bits considered to be the start bit Inf ₁ is larger than the maximum value MAX, the maximum value MAX is updated, and at the same time, the current search point is set as the embedding start point. Remember. Then, it is determined in step 2806 whether or not all search points have been searched.
Moves to the next search point in raster order. On the other hand, if all the processes have been completed, the embedding start point stored at that time is output, and the process ends.

[0430] By the above series of processes, the offset adjustment unit 2003 in the present embodiment detects the start bit Inf _1, in the correct start bit Inf ₁ is obtained coordinates, seems to start bit Inf ₁ The information of the coordinate having the largest sum of the reliability distances d corresponding to 5 bits is determined to be the embedding start point of the additional information Inf, and is output to the subsequent stage as the embedding start coordinate.

[3-7-5 Usage Information Extraction Means] The usage information extraction means 2004 is
Start coordinates embedded from 03, and additional input information Inf image data embedded, by also using the operation described in FIG. 6, where only the respective bit information constituting the usage information Inf ₂ is a reliability distance d Calculated, and the reliability distance d1 for the bit information is calculated by the statistical test means 2006 in the subsequent stage.
Output to

It should be noted that obtaining the reliability distance d1 corresponding to each bit information constituting the usage information Inf ₂ is substantially equivalent to:
Equivalent to extract each bit of use information Inf ₂ embedded. This will be described later.

[0433] Here, based on the embedding start coordinates it is determined by the search, only calculating each reliability distance d, not extracted for 5 bits of the start bit Inf _1.

[3-8 Statistical Test Processing] Statistical test means 200
In 6, the reliability of the reliability distance d1 obtained by the use information extraction means 2004 in FIG. 20 is determined. This determination is made by using a second pattern arrangement different from the first pattern arrangement used for extracting the additional information Inf (use information Inf ₂ ), and using the reliability distance.
This is performed by generating d2 and generating a reliability index D with reference to the appearance frequency distribution of the reliability distance d2.

Here, the reliability distance d1 is determined by the use information extracting means 2
To extract the usage information Inf ₂ at 004, a reliability distance obtained using the first pattern array (referred to as also the arrangement information cone mask), the reliability distance d2 first
Is a reliability distance obtained by using a second pattern arrangement, which will be described later, which is different from the pattern arrangement. The first pattern arrangement is the pattern arrangement of FIG. 9 used when embedding the additional information Inf (start bit Inf ₁ , usage information Inf ₂ ).

The details of the second pattern arrangement, the reliability index D, etc. will be described later.

[3-8-1 Extraction Process Using Second Pattern Array] ≪ Central limit theorem ≫ The subsets A and B are A = {a ₁ ,
a ₂ , ..., a _N }, a set composed of N elements represented by B = {b ₁ , b ₂ , ..., b _N }, and a subset as shown in FIG. A
And the pixel values of the elements of subset B.

The reliability distance d (Σ (a _i -b _i ) / N) is a sufficiently large value of N, and if there is no correlation between the pixel values a _i and b _i , the expectation of the reliability distance d The value becomes 0. Further, the distribution of the reliability distance d takes an independent normal distribution according to the central limit theorem.

Now, the central limit theorem will be briefly described.

[0440] When extracting the arbitrary sampling of the average value m _c, the size n _c from the population standard deviation sigma _c (may not be normally distributed), the distribution of the sample mean value S _c is n _c is greater Is a theorem indicating that the distribution approaches a normal distribution N (m _c , (σ _c / √n _c ) ^ 2).

Generally, the standard deviation σ _{c of the} population is often unknown, but the number of samples n _c is sufficiently large and the number of populations N _c
There practically almost no problem even if used in place of the standard deviation s _c a sigma _c specimens when further sufficiently larger than the number of samples n _c.

Description will be made returning to the present embodiment. First appearance frequency distribution of the reliability distance d1 obtained by the use information extraction unit 2004 differs greatly depending on whether or not correctly extract the usage information Inf _2.

For example, if there is an error in the detection of the start bit Inf ₁ (for example, when the offset adjustment fails), bit information is actually embedded at the position where the usage information Inf ₂ should be embedded. 25, the appearance frequency distribution of the reliability distance d1 is like a normal distribution 2501 in FIG.

[0444] On the other hand, if you have correctly extracted, the reliability distance d1 corresponding to the bit information "1" constituting a utilization information Inf ₂ are accumulated in the position of the normal distribution 2502, constituting a utilization information Inf ₂ Each reliability distance d1 corresponding to the corresponding bit information “0” is accumulated at the position of the normal distribution 2503. Therefore, in this case, two “mountains” appear. The ratio of the size of the two "peaks" is substantially equal to the ratio of the bit information "1" and "0" constituting a utilization information Inf _2.

However, this is because the reliability distance d1 obtained by performing convolution processing with the first pattern array on the original image in which no additional information is embedded is equal to the normal distribution 2501.
It is based on the assumption that

[0446] Therefore, in reality, it is impossible to determine whether or not extraction has been correctly performed unless the state of the original image is known.

Therefore, in the present embodiment, a normal distribution of the reliability distance d2 is generated by using a so-called second pattern arrangement capable of sufficiently discriminating the state of the original image even when the additional information is embedded. by considering distribution as 2501, it determines whether or not the use information Inf ₂ are correctly extracted.

For example, if the appearance frequency distribution of the reliability distance d1 exists outside the diagonally shaded portion (the component from the center to 95%) constituting the normal distribution 2501 created with the reliability distance d2, it becomes a target. Image has a statistical bias,
It can be considered that usage information Inf ₂ is embedded,
The probability of the usage information Inf ₂ can be statistically determined. This detailed method will be described later.

Next, using image data in which the additional information Inf (use information Inf ₂ ) is embedded, a distribution similar to the appearance frequency distribution of the reliability distance d1 before the additional information Inf is embedded (see FIG. 25). A method of generating such a normal distribution 2501) will be described.

In this embodiment, the reliability distance d2 forming a distribution similar to the normal distribution 2501 is obtained by using the extraction means 2005 based on the second pattern arrangement.

The extracting means 200 based on the second pattern arrangement
Reference numeral 5 denotes a means for obtaining a reliability distance d2 using the second pattern array “perpendicular to” the first pattern array used in the usage information extraction unit 2004. The operation itself is almost the same as the means 2004.

[0452] For the sake of comparison, the usage information extraction device 2
The pattern arrangement of FIG. 9 used in 004 and a mask (cone mask) for referring to the position where this pattern arrangement is arranged are called a “first pattern arrangement” and a “first position reference mask”, respectively. , A pattern array that is “perpendicular” to the first pattern array, and a mask for referring to the position where this pattern array is arranged are called “second pattern array” and “second position reference mask”, respectively. .

[0453] Extracting means 200 using the second pattern arrangement
5, first, the embedding start coordinates are input from the offset matching unit 2003, and the reliability distance d2 is also calculated using the reliability distance calculation of FIG. 6 described above.

At this time, the pattern array used in the reliability distance calculation of FIG. 6 is the pattern array 09 of FIG. 9 used for embedding.
Instead of 01, a pattern array 3301 or 3302 that is “perpendicular” to this pattern array 0901 is used.

The reason for this is that the pattern arrangement 330 shown in FIG.
The reliability distance d2 calculated using 1 and 3302 includes:
Pattern array 0 in FIG. 9 used for embedding additional information Inf
This is because the effect operated at 901 is not reflected at all.

As shown in FIG. 34, the pattern array 0 shown in FIG.
901 and the above-mentioned pattern arrangement 3 which is “perpendicular” to this
The result of the convolution processing of 301 and 0 is zero. This is the same for the pattern arrangement 3302. That is, the convolution result of the first and second pattern arrays is 0. Therefore, even if the density of the original image is changed using the first pattern array, the reliability distance d obtained by performing the convolution process using the second pattern array has no effect.

Therefore, the appearance frequency distribution of the reliability distance d2 obtained by performing the convolution process using the second pattern array on the image in which the additional information Inf is embedded is:
This is almost the same as the normal distribution 2501 in FIG. Therefore, the above-mentioned appearance frequency distribution is regarded as the normal distribution 2501.

The obtained normal distribution 2501 is shown in FIG.
It becomes a criterion necessary for the statistical test processing of 2207.

The extraction processing 200 using the second pattern arrangement
5 is a “pattern array“ perpendicular to the first pattern ”, such as 3301 and 3302 in FIG.
A normal distribution of the reliability distance d2 is generated using the second position reference mask indicated by 3502 in FIG.

Note that the conditions of the above “pattern arrangement orthogonal to the first pattern” are as follows: (1) As shown in FIG. 33, the same size as 0901 in FIG. Like the arrays 3301 and 3302, the pattern array 090 of FIG. 9 used when embedding the additional information Inf.
That is, the result of the convolution process with 1 becomes 0.

Also, the convolution processing shown in FIG.
1 and the convolution processing shown in FIG.

In the present embodiment, the case where the result of convolution becomes 0 is likened to the case where the inner product of vectors is 0 when the inner product is orthogonal. Call. Therefore, 3301 and 330 in FIG.
2 is “pattern array“ perpendicular ”to pattern array 0901 in FIG. 9”.

The reason why a pattern array “perpendicular” to the pattern array used at the time of embedding the additional information Inf is used in the calculation of the reliability distance d2 is that there is no statistical bias in the distribution of the reliability distance d2. This is for generating the center appearance frequency distribution.

Also, "the first pattern is" perpendicular "
The “pattern array” has (3) the number of non-zero elements equal to the number of non-zero elements of the pattern array used in the usage information extraction processing 2004, and it is necessary that the number of positive and negative elements be equal to each other. is there. This is to extract the reliability distance d1 and the reliability distance d2 under the same calculation conditions.

Next, in the present embodiment, the “second position reference mask” is the same as the 350 used when embedding the additional information Inf.
A reference mask shown in 3502 in FIG. 35 having a pattern different from 1 and having a different size from 3501 is used.

As described above, if the first and second pattern arrangements are different, the appearance frequency distribution of the reliability distance d2 becomes almost a normal distribution 2501.

However, when the detection position of the start bit is not perfect or the like, there is a possibility that a statistical bias may be detected even though the convolution is performed using the second pattern arrangement. In the present embodiment, taking this possibility into consideration, the first and second position reference masks are made different in size to cancel out periodic elements. Alternatively, the convolution in the same area is not performed by differentiating the arrangement method of each pattern arrangement in the mask.

In this case, the “second position reference mask” is not limited to a cone mask as long as the respective constituent coefficients are randomly distributed.

If "Second embedding position reference mask"
However, when the setting is made different from the “first embedding position reference mask”, the “second embedding position reference mask” is created by the embedding position determining means 2008 in FIG.

Generally, in consideration of the above-described cutout resistance, the first position reference mask (cone mask) cannot take such a large size as to the entire image data to be embedded with the additional information Inf. Unthinkable. Therefore, it is preferable to use a relatively large “second position reference mask”. In the present embodiment, the size of the second mask used for calculating the reliability distance d1 on the side of the additional information Inf is set to be larger than the first mask referred to when embedding the additional information Inf. And

However, the present invention is not limited to this, and a certain effect can be obtained even if the mask sizes are equal to each other.
Therefore, the “second position reference mask” may be created by the embedding position determination unit 2002 in FIG.

[0472] As a minimum condition of each other's mask, it is necessary that the number of repetitions of each bit constituting the additional information Inf applied to each other's mask is equal within an image area of the same size.

If a sufficient result is not obtained by the extraction processing using the second pattern array, the reliability is again determined using another second pattern array or a second position reference mask satisfying the above-mentioned conditions. By calculating the distance d2, there is a possibility that the ideal appearance frequency distribution 2501 in FIG. 25 can be generated.

Next, extraction means 2 based on the second pattern arrangement
005 shows the specific operation.

In the present embodiment, the first position reference mask is a 32 × 32 cone mask, and the second position reference mask is a 64 × 64 cone mask. Assume that the sequences are completely different.

First, in the extraction means 2005 based on the second pattern arrangement, the extraction position is determined according to the following correspondence table 3.

[0477]

[Table 3]

In the second position reference mask, there are 16 coefficients having the same value in the mask. On the other hand, for the 32 × 32 first position reference mask, when the mask is referred to in the above-described correspondence table 2, the number of repetitions of the same coefficient in 32 × 32 is four. That is, in image data of the same size, the first position reference mask and the second position reference mask have the same number of coefficients having the same value.

In this embodiment, the second pattern arrangement is assigned to the positional relationship according to the rules of the above-mentioned correspondence table 3, and convolution processing is sequentially performed to calculate 69 reliability distances d2 corresponding to each bit information. I do.

[3-8-2 Reliability Index D] The reliability distance d2 generated by the extraction means 2005 based on the second pattern arrangement appears in almost the same distribution as the normal distribution 2501, but in the normal distribution. Is generally given by the following equation (25.1)
It is known that 95% of samples (reliability distance d2) appear in the range of.

M-1.96σ <d2 <m + 1.96σ Equation (25.1) Here, σ is the standard deviation of the reliability distance d2, and m is the average.

The range in the above case is called "95% confidence interval".

[0483] m-1.96σ, m + 1.96σ are calculated using the reliability distance d2 obtained by the extraction means 2005 using the second pattern arrangement.

If the bit information is “1”, the appearance frequency distribution of the reliability distance d 1 input from the usage information extracting means 2004 to the statistical test means 2006 is the normal distribution 25 shown in FIG.
02, and if the bit information is “0”, the normal distribution 25
03, the reliability distance d1 corresponding to the usage information Inf ₂
Has a very high probability of being outside the 95% confidence interval (shaded portion in FIG. 25) obtained by the extraction means 2005 based on the second pattern arrangement.

By the way, the offset adjusting means 2003
At the time of processing, the usage information In is added to the image to be processed.
If the f ₂ does not exist, become like the appearance frequency distribution is also normal distribution 2501 of the reliability distance d1.

[0486] the additional information Inf ₂ Despite being embedded in the image, all 64 corresponds to the usage information Inf ₂ of the reliability distance d1 is not included in the confidence interval of formula (25.1) random Is as very small as (1-0.95) to the 64th power.

Therefore, if the normal distribution 2501 is obtained based on the reliability distance d2, it is determined whether or not the range occupying most of the normal distribution includes the appearance frequency distribution obtained based on the reliability distance d1. By considering the additional information Inf
(Usage information Inf ₂ ) can be almost certainly determined whether or not it is embedded.

The statistical test means 2006 judges the reliability of the embedded additional information Inf (use information Inf ₂ ) using the above-described properties.

[0489] In the present embodiment, the reliability of the use information Inf being embedded is treated as the reliability index D.

[0490] This reliability index D is used by the usage information extraction means 2
Equation (2) for all reliability distances d1 generated in 004
It is defined by the ratio of the number of reliability distances d1 existing outside the range of 5.1).

If the reliability index D is larger than the threshold Th, the statistical test means 2006 determines that the overall appearance frequency distribution of the reliability distance d1 is artificially biased to a position like 2502 or 2503 in FIG. Is determined to be an image in which the usage information Inf2 is securely embedded.

Therefore, it is considered that the reliability distance d1 used in the determination here is reliable information, and the transfer of the reliability distance d1 to the comparison means 2007 at the subsequent stage is permitted.

As shown in the reliability display step 3210 in FIG. 32, the reliability index D may be a reliability index D of the usage information Inf ₂ or a message based on the index D may be displayed on a monitor or the like. .

For example, if the reliability index D is not larger than the threshold Th, a message to the effect that “Usage information Inf ₂ has not been accurately extracted” is displayed, and an image is displayed from the statistical test step 3207 in FIG. Is returned to step 3202.

[3-9 Comparison Processing] The comparison means 20 in FIG.
07 is the usage information extracting means 2004 and the statistical testing means 20
06, and the value of the reliability distance d1 outputted through the step 06 is input.
Since the reliability distance d1 input here is highly reliable information, here, it is simply determined whether each bit information corresponding to the reliability distance d1 is “1” or “0”. Is good.

[0496] Specifically, if it is positive reliability distance d1 is given bit information constituting the usage information Inf _2, it determines that the bit information is "1", if the reliability distance d1 is negative Determines that this bit information is "0".

The use information Inf ₂ obtained by the above determination is
It is output as user's reference information or final data for use as a control signal.

[3-10 Consideration for High-Speed Processing] As described above, it is possible to extract additional information. On the other hand, in the present embodiment, it is not always necessary to perform all the extraction processing of the additional information Inf. In the embodiment described above, the input pwI ′ to the digital watermark extraction device shown in FIG.
It is assumed that the electronic watermark information is embedded in the. On the other hand, electronic watermark information may not be embedded in the input pwI 'to the electronic watermark extraction device. Further, some applications require that the processing of the digital watermark extraction information be completed as quickly as possible. As described above, in the digital watermark extracting device, when it is determined that the digital watermark information is not embedded in the input pwI ′ to the digital watermark extracting device, the digital watermark extracting process is terminated at a high speed. It is desirable from the point of view. This will be described with reference to the flowchart shown in FIG.

FIG. 55 is a modification of the flowchart of the digital watermark extracting process shown in FIG. In the flowchart of FIG.
FIG. 55 is a flowchart in which “04” has been added.

Registration signal detection determination processing 55
In step 04, a scale matching process 550 which is the pre-process is performed.
3 is used to determine whether the registration signal r has been detected. If no registration signal has been detected, the information extraction processing ends at this point. A registration signal r must be detected from an image in which digital watermark information is embedded. The fact that the registration signal r is not detected means that the digital watermark information is not embedded. Therefore, there is no need to continue the digital watermark extraction processing thereafter, and the digital watermark extraction processing can be ended at this point.

[0501] This is the end of the description of the series of processing from embedding to extraction of additional information.

(Modification) In the above embodiment, it is possible to use error-correction-encoded information as additional information Inf (use information Inf ₂ ).
Further, the reliability of the extracted use information Inf ₂ is improved.

The present invention is not limited to the case where the optimum pattern for the resolution of the input image data or the output (print) resolution of the printer is selected from the two pattern arrangements shown in FIG. That is, it has two or more pattern arrangements having m × n elements (m and n are integers) and different sizes from each other. The case where it is selectively used is included in the scope of the present invention.

[0504] Even if the present invention is applied as a part of a system composed of a plurality of devices (for example, a host computer, an interface device, a reader, a printer, etc.), one device (for example, a copying machine, a facsimile machine)
May be applied to a part of the system.

The present invention is not limited to only the apparatus and method for realizing the above-described embodiment, but includes a computer (CPU or MPU) in the system or apparatus.
In addition, the present invention is also applied to a case where the program of the software for realizing the above-described embodiment is supplied, and the computer of the system or the apparatus operates the various devices according to the program code to realize the above-described embodiment. Included in the category.

In this case, the program code of the software implements the functions of the above-described embodiment, and the program code itself and means for supplying the program code to the computer, specifically, A storage medium storing the above program code is included in the scope of the present invention.

As a storage medium for storing such a program code, for example, a floppy disk, hard disk, optical disk, magneto-optical disk, CD-ROM, magnetic tape, nonvolatile memory card, ROM and the like can be used.

In addition to the case where the computer controls various devices in accordance with only the supplied program codes to realize the functions of the above-described embodiment, the program codes operate on the computer. Such a program code is included in the scope of the present invention even when the above-described embodiment is realized in cooperation with an OS (Operating System) or other application software.

Further, after the supplied program code is stored in a memory provided in a function expansion board of a computer or a function expansion unit connected to the computer, the function expansion board or function is stored based on the instruction of the program code. The case where the CPU or the like provided in the storage unit performs part or all of the actual processing, and the above-described embodiment is realized by the processing is also included in the scope of the present invention.

[0510] In the above embodiment, the case where digital watermark information is embedded using a cone mask has been described, but the present invention is not limited to this. In particular, the case where digital watermark information is embedded using a blue noise mask is also included in the category of the present invention.

[0511] Also, a configuration including at least one of the above-mentioned various features is included in the scope of the present invention.

(Second Embodiment) Hereinafter, a second embodiment of the present invention will be described with reference to the drawings.

FIG. 53 shows an image processing system applicable to each embodiment of the present invention.

In the figure, the host computer 530
Reference numeral 1 denotes, for example, a generally-used personal computer, which can input an image read by a scanner 5314 and edit / store the image. Furthermore, the image obtained here can be printed from the printer 5315. Various manual instructions and the like from the user can be obtained by using the mouse 5
312, which is performed by input from the keyboard 5313.

[0515] Inside the host computer 5301,
Each block described below is connected by the bus 5316, and various data can be transferred.

[0516] In the figure, reference numeral 5303 denotes a CPU capable of controlling the operation of each internal block or executing a program stored therein.

[0517] Reference numeral 5304 denotes a ROM for storing a specific image which is not permitted to be printed, or for storing a necessary image processing program or the like in advance.

[0518] Reference numeral 5305 denotes a RAM for temporarily storing a program and image data to be processed for the CPU to perform processing.

[0519] Reference numeral 5306 denotes a hard disk (H) capable of storing in advance a program or image data to be transferred to a RAM or the like, or storing processed image data.
D).

[0520] Reference numeral 5307 denotes the current item or film
D is a scanner interface (I / F) that can be connected to a scanner that reads and generates image data and that can input image data obtained by the scanner.

[0521] Reference numeral 5308 denotes C which is one of the external storage media.
It is a CD drive that can read or write data stored in D (CD-R).

[0522] Reference numeral 5309 denotes an FD drive capable of reading from the FD and writing to the FD, similarly to the 5308. 5310 is also a DVD drive capable of reading from a DVD and writing to a DVD, similarly to 5308. If a program for image editing or a printer driver is stored in a CD, FD, DVD, or the like, these programs are installed on the HD 5306 and transferred to the RAM 5305 as necessary. .

[0523] Reference numeral 5311 denotes an interface (I / F) connected to the mouse 5312 or the keyboard 5313 to receive input instructions therefrom.

[0524] Next, in the above system, the flow of operation for embedding a digital watermark in image data is shown in FIG.
Explanation will be made using 0.

[Overall Configuration of Embedding Method] FIG. 50 shows a digital watermark embedding process according to the present embodiment. A computer-executable program describing the procedure shown in the flowchart of FIG.
4 or in advance, HD5306, C
This embodiment is realized by reading a program stored in D5308, FD5309, DVD5310, or the like into the RAM 5305, and then executing the program by the CPU 5303.

As shown in FIG. 50, the digital watermark embedding processing is performed by the image input unit 5001 and the block division unit 500.
2, a first information embedding unit 5003, a second information embedding unit 5004, and an image output unit 5005.

First, the image input unit 5001 will be described. Image data is input by the image input unit 5001. This is multi-valued image data in which a plurality of predetermined bits are assigned to one pixel. In the present embodiment,
It is possible to cope with whether the input image data is grayscale image data or color image data. Grayscale image data is composed of one type of element per pixel, and color image data is
It is assumed that one pixel is composed of three types of elements. In the present embodiment, the three types of elements are a red component (R), a green component (G), and a blue component (B). However, the present invention is applicable to other combinations of color components.

[0528] In this embodiment, the case where color image data is input will be described. In particular, the case of a bill among the color image data will be described. Although the present embodiment will be described with reference to banknotes, the present invention is not limited to this, and the case where color image data is securities or a specific printed matter having copyright is also included in the scope of the present invention. It is.

When color image data is input,
One or more elements among the elements constituting the color image data are selected. In the present embodiment, processing is performed on the blue component. This is because the blue component is the least sensitive to human vision among the red, blue, and green components. Therefore, embedding the digital watermark information in the blue component has the effect of making it harder for human eyes to perceive image quality degradation due to the digital watermark information than embedding the digital watermark information in other color components.

Next, the block dividing section 5002 will be described. The block dividing unit 5002 divides the image data input by the preceding image input unit 5001 into a plurality of non-overlapping areas. This area is, for example, a rectangular area of 128 × 128 pixels. In order to execute a digital watermark extraction process described later at high speed, it is desirable that the size of this area be as small as possible. This is because the smaller the size of the region, the faster the digital watermark extraction process can be executed.

Next, the first information embedding unit will be described. The first information is embedded in the block by a first information embedding unit. The first information is 1-bit information indicating a banknote in the present embodiment.

[0532] As the first information, the registration signal r as described in the first embodiment can be applied. In the first embodiment, the registration signal r is used for correcting geometric distortion. In the present embodiment, in addition to the purpose of correcting the geometric distortion, a registration signal is used as 1-bit information indicating a banknote. That is, 1-bit information (in the present embodiment, information indicating a banknote) is embedded depending on the presence or absence of a registration signal.

Note that another digital watermark may be used as the first information instead of the registration signal. Further, the first information is not limited to 1-bit information indicating whether or not the bill is a bill, but may be, for example, 2-bit information that can identify bills / securities / copyright images / others.

Next, the second information embedding unit will be described. The second information is embedded in the block by a second information embedding unit. The second information is detailed information relating to a bill in the present embodiment. This information embeds relatively more information than the first information. FIG. 51 shows a configuration example of the second information. In the example shown in FIG. 51, the second information is composed of 8 bits of billing country information and 8 bits of bill information. As the second information, the additional information Inf as described in the first embodiment can be applied.

[0535] The first information and the second information have been described above. Here, the properties of the first information and the second information in the present embodiment are summarized as follows. 1) The information amount of the first information is smaller than the information amount of the second information. 2) The embedding strength of the first information is higher than the embedding strength of the second information.

The gist of the present invention is that the first information
Is to determine whether or not to extract the information. Therefore, it is desirable that the first information can be extracted more reliably than the second information. Further, generally, it takes a long processing time to extract a large amount of information. In order to extract the first information in a short processing time, it is desirable that the first information has a smaller amount of information than the second information.

[0537] Finally, the image output section 5005 outputs image data in which the first information and the second information are embedded.

[Image Copying Process] Next, in the present embodiment, a flow of operations from editing of an image obtained by reading a document to printing after editing the image will be briefly described with reference to FIG.

First, at 5401, the original is read by the scanner 5314, and color image data composed of 8-bit RGB color components is generated. Next, at 5402, the color image is input to the host computer 5301 via the I / F 5307, and temporarily stored in the HD 5306.

At 5403, it is determined whether an instruction to perform image editing has been issued or not. If an instruction to edit an image has been input from a mouse or the like, the flow advances to 5407 to execute an image editing program. On the other hand, if there is no instruction to edit the image, the flow advances to step 5404.

At 5407, it is assumed that the image editing program is executed by the CPU, and the program used is transferred to the ROM 5304 or the RAM 5305. Note that this program can
The data may be transferred from the D5306 to the RAM 5305. The program stored in the HD 5306 is C
D, FD, DVD, or the like may be used after being installed. In the image editing, the image represented by the color image data is enlarged, reduced, combined with another image, cut out, converted in color, and the like, and the obtained color image data is stored in the HD 5306 again.

At 5404, it is determined whether an instruction for printing has been issued or not. If the instruction for printing has been input from a mouse or the like, the flow advances to 5405 to operate the printer driver. On the other hand, if there is no print instruction, 5
The process returns to step 403 and waits until an image editing instruction and a printing instruction are input. Note that this standby state may be canceled in response to time or interruption of another process.

In step 5405, the printer driver is executed by the CPU, and the printer driver program used is stored in the ROM 5304 or the RAM 530.
5 is stored. Note that this program may be transferred from the HD 5306 to the RAM 5305 as necessary. The programs stored in the HD 5306 may be used after installing those stored in a CD, FD, DVD, or the like.

In the above printer driver, the HD5306
After detecting whether the color image data to be printed stored in the printer is improper printing, if the printing is not improper printing, color space conversion (RGB / YMCK conversion), halftone processing (binarization) And the like, and transfers the data to the printer 5315 via the I / F 5316. The scope of the present invention also includes that color image data is divided into printing data units (bands), input to the printer driver for each band, and transferred to the printer.

At 5406, the host computer 530
1 (I / F 5316), and sequentially performs printing for each color image data transferred. Next, the operation of the printer driver will be described in detail.

[Explanation of Inside of Printer Driver] FIG.
3 shows a flow of an operation of the printer driver according to the present embodiment. As shown in FIG. 52, the printer driver according to the present embodiment includes an image input unit 5201, a block division unit 5202, a block selection unit 5203, a first information extraction unit 5204, an information extraction determination unit 5205, and a re-extraction determination unit 52.
06, a second information extraction unit 5207, a control unit 5208, and an image processing unit 5209.

Image input unit 5201 and block division unit 5
In step 202, the same processing as that performed by the image input unit 5001 and the block division unit 5002 in the digital watermark embedding process is executed.

Next, the block selection section 5203 will be described. The block selecting unit 5203 selects at least one or more selected blocks from the blocks divided by the preceding block dividing unit 5202. In the present embodiment, a case where one selected block is selected will be described.

Next, the first information extracting section 5204 will be described. The first information extraction unit 5204 extracts first information (in this embodiment, 1-bit information indicating whether or not the input image is a bill) from the selected block selected by the preceding block selection unit.

In this embodiment, the registration signal r as described in the first embodiment is extracted as the first information. As described in FIG. 5, the registration signal r is an impulse signal in the frequency domain, and can be extracted by the method of detecting an impulse as described in the first embodiment.

Next, the information extraction determining section 5205 will be described. The information extraction determination unit 5205 determines whether the first information extraction unit 5204 at the previous stage has extracted the first information. When the first information is extracted, it is determined that the first information is embedded (that is, in the present embodiment, the input image data is determined to be a bill), and the second information is determined. The information extraction unit 5207 is executed.

On the other hand, if the first information is not extracted, it is determined that the first information is not embedded (that is, in this embodiment, it is determined that the input image data is not a bill). ), And the re-extraction determination unit 5206 is executed.

In this embodiment, the determination is made based on whether or not the registration signal r has been extracted. If an impulse is extracted according to a predetermined threshold 0804,
It is determined that the first information has been extracted. 0804 on the other hand
If no impulse has been extracted, it is determined that the first information has not been extracted.

Next, the re-extraction judging unit 5206 will be described. The re-extraction determining unit 5206 determines whether to execute the first information extraction again. In the present embodiment,
First information is extracted from only one selected block selected by the block selection unit 5203.

[0555] Originally, in the case of a banknote, the first information should be able to be extracted from the selected block. However, when the image data input to the printer driver has been subjected to various attacks, the above-mentioned information is required. The first information that should have been embedded in the selected block may have been lost. Therefore, even when the first information is not extracted from the selected block, the first information is extracted from the blocks other than the selected block.
Try to extract the information of Re-extraction determining unit 5206
Then, a process of determining whether or not to re-extract the first information is executed.

In the present embodiment, the number of times the first information extraction has been performed is counted, and if this number is less than the predetermined number, the block selection unit is used to extract the first information from another block. Step 5203 is executed again. On the other hand, if the number is equal to or more than the predetermined number, the image processing unit 5209
Is executed. By the above processing, the first predetermined number of times
The image processing unit 5209 is executed only when the first information is not extracted even if the information extraction is performed. By increasing the predetermined number of times, it is possible to more reliably extract the first information. On the other hand, increasing the predetermined number of times increases the processing time in the printer driver, resulting in a longer time required for printout. For example, the predetermined number is set to about “5”.

Next, the second information extracting section will be described. In the second information extracting unit, a second information extracting unit
Is extracted. Here, the method of extracting the second information will be described later.

Next, the control unit 5208 will be described.
The control unit 5208 performs various controls based on the second information extracted in the previous stage. Here, the control means, for example, the second
Various kinds of processing can be executed, such as displaying the content of the information on the output device, transmitting the information of “blackout” to the image processing unit, and changing the medium to be printed out to the content of solid black. .

Next, the image processing unit 5209 will be described. The image processing unit 5209 converts the image data into data that can be output by a printer. The image processing unit 5209 includes a color conversion process, a gradation conversion process, and the like.

By the above processing, the first image is added to the input image.
Is embedded (that is, when it is determined that the banknote is a banknote), a warning message is output on the screen and the input image is not printed out, or an image of solid black is printed out. You. On the other hand, when the first information is not embedded in the input image (that is, when it is not determined that the input image is a bill), the input image is printed out in a normal manner. Here, since the first information is 1-bit information, the first information can be extracted at high speed. Therefore, most non-banknote image data of the image input to the digital watermark extraction processing can execute the printout processing in substantially the same time as in the related art.

[0561]

As described above, according to the present invention, the first
By ending the second information extraction for an image in which the information is not embedded, the extraction processing can be completed at high speed.

Further, according to the present invention, when detecting unauthorized printing, it is quickly detected whether or not the printing is unauthorized, and if the printing is non-illegal, normal print-out processing is executed. Can be executed without substantially changing the conventional printout processing time.

[Brief description of the drawings]

FIG. 1 is an overall configuration block diagram of a digital watermark embedding device according to a first embodiment.

FIG. 2 is an overall configuration block diagram of a digital watermark extraction device according to the first embodiment.

FIG. 3 is a diagram illustrating an example of image data generated on an extraction side in a printing process.

FIG. 4 is a block diagram of a registration signal embedding unit.

FIG. 5 is a diagram illustrating a registration signal.

FIG. 6 is a diagram showing a reliability distance calculating means.

FIG. 7 is a block diagram of a scale adjusting unit.

FIG. 8 is a diagram illustrating extraction of a registration signal.

FIG. 9 is a diagram showing a pattern array used when embedding and extracting additional information.

FIG. 10 is a block diagram of additional information embedding means.

FIG. 11 is a block diagram of an embedding position determination unit.

FIG. 12 is a diagram showing an appearance frequency distribution of a cone mask and a blue noise mask.

FIG. 13 is a diagram showing spatial frequency characteristics of human vision.

FIG. 14 is a diagram showing spatial frequency characteristics of a blue noise mask and a cone mask.

FIG. 15 is a simple explanatory diagram of a position reference mask.

FIG. 16 is a conceptual diagram showing an embedding position in a position reference mask.

FIG. 17 is a diagram showing how each pattern array is developed on the mask of FIG. 16;

FIG. 18 is a diagram illustrating an operation of repeatedly embedding a minimum embedding unit of additional information Inf in an entire image.

FIG. 19 is a diagram illustrating an operation for embedding additional information Inf.

FIG. 20 is a diagram illustrating additional information extracting means.

FIG. 21 is a diagram illustrating a state of extracting additional information Inf.

FIG. 22 is a diagram illustrating a state in which an attempt is made to extract information even when additional information Inf does not exist.

FIG. 23 is a diagram showing an ideal appearance frequency distribution when a reliability distance d is extracted from an original image.

FIG. 24 is a diagram illustrating a case where a reliability distance d is extracted from an image in which a digital watermark is embedded.

FIG. 25 shows reliability distances d1 and d2 in the first embodiment.
It is a figure explaining the example of appearance frequency distribution of.

FIG. 26 is a diagram illustrating the principle of embedding and extracting a registration signal.

FIG. 27 is a diagram showing the concept of embedded head position search in the offset matching means.

FIG. 28 is a flowchart illustrating a registration process.

FIG. 29 is a block diagram of a registration signal embedding unit in a spatial region.

FIG. 30 is a diagram illustrating two sets in the patchwork method.

FIG. 31 is a flowchart illustrating the entire digital watermark embedding process.

FIG. 32 is a flowchart illustrating the entire digital watermark extraction processing.

FIG. 33 is a diagram showing an example of a pattern array orthogonal to the pattern of FIG. 9;

FIG. 34 is a view for explaining “orthogonal” and “pattern arrangement”;

FIG. 35 is a diagram showing first and second position reference masks.

FIG. 36 is a diagram showing a configuration of additional information Inf.

FIG. 37 is a diagram illustrating an example of each coefficient in a blue noise mask.

FIG. 38 is a diagram illustrating an example of each coefficient in a cone mask.

FIG. 39 is a diagram showing the chromaticity space number characteristics of human vision.

FIG. 40 is a diagram illustrating a minimum coding unit in the JPEG system.

FIG. 41 is a diagram illustrating sampling of luminance and color difference components in the JPEG system.

FIG. 42 shows positive and negative operation portions (patches) of a pattern array
FIG.

FIG. 43 is a diagram for explaining the correspondence between the gray scale expressed by the area gray scale and the gray scale expressed by the density gray scale.

FIG. 44 is a diagram illustrating the principle of propagation of gradation information before and after gradation conversion processing.

FIG. 45 is a diagram illustrating a difference in halftone processing of a printer depending on the resolution of an image.

FIG. 46 is a diagram for explaining changes in ink dots due to patch embedding.

FIG. 47 is a diagram for explaining an increase in dots due to a patch size and an embedding depth.

FIG. 48 is a diagram illustrating a difference between a positive patch and a negative patch in a pattern arrangement unit due to a difference in image resolution.

FIG. 49 is a diagram showing a pattern array for embedding additional information Inf corresponding to each resolution of an image.

FIG. 50 is a diagram illustrating a digital watermark embedding process according to the second embodiment.

FIG. 51 is a diagram showing second information in the second embodiment.

FIG. 52 is a diagram illustrating processing inside a printer driver according to the second embodiment.

FIG. 53 is a diagram illustrating an image processing system applicable to the second embodiment;

FIG. 54 is a diagram showing an operation when copying a document using the system of FIG. 53;

FIG. 55 is a diagram illustrating a modification of the digital watermark extraction process according to the first embodiment.

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Megumi Iwamura 3-30-2 Shimomaruko, Ota-ku, Tokyo Inside Canon Inc. (72) Inventor Yoshihiro Ishida 3-30-2 Shimomaruko, Ota-ku, Tokyo 5B057 AA11 BA19 CA01 CA12 CA16 CB01 CB12 CB16 CC03 CE08 CH18 DB06 DC25 5C076 AA14 BA02 BA06 5C077 LL14 LL18 MP08 NP07 PP21 PP23 PP49 PP55 PP65 PP66 PP68 PQ08 5J104 AA14 NA01 NA08 NA10 NA08 DD07 EE03 GG14 HH21 HH23 JJ35 KK37 KK42 LL03

Claims (17)

[Claims] A first information extracting step of extracting first information from an image, and a determining step of determining whether to extract second information from the image according to the first information extraction result. An image processing method characterized by the following. 2. The image processing method according to claim 1, wherein the first information and the second information are embedded in the image as an invisible or invisible digital watermark. 3. The method according to claim 1, further comprising a dividing step of dividing the image into at least one or more blocks, and a selecting step of selecting one or more of the divided blocks. Image processing method. 4. The image processing method according to claim 1, wherein the first information is information indicating that the image includes a specific image. 5. The image processing method according to claim 1, wherein the first information is a registration signal used for correcting a geometric distortion of an image. 6. The image processing method according to claim 1, wherein said second information is additional information. 7. The image processing method according to claim 1, wherein the first information and the second information are added to a component of the image in which human eyes are insensitive. 8. The image processing method according to claim 1, wherein the first information is information for identifying bills, securities, copyright images, and other information. 9. The image processing method according to claim 4, wherein the specific image is a banknote, and the second information is information indicating at least one of an issue country and an amount of the banknote. 10. The image processing method according to claim 4, further comprising a determining step of determining that the image includes the specific image, and controlling the image processing based on the image when the image includes the specific image. 11. The image processing method according to claim 1, wherein the image processing method is executed by a printer driver. 12. The image processing method according to claim 1, wherein the first information has smaller information than the second information. 13. The image processing method according to claim 1, wherein an embedding strength of the first information in the image is higher than that of the second information. 14. The time required for the first information extraction is a second time.
2. The image processing method according to claim 1, wherein the time is shorter than the time required for extracting the information. 15. The method according to claim 1, wherein the first information has a larger number per unit area than the second information.
Item. 16. An image processing apparatus for performing the image processing method according to claim 1. Description: 17. A storage medium storing a code for executing the image processing method according to claim 1. Description: