JP4630765B2 - Image processing method and image processing apparatus - Google Patents

Description

  The present invention, for example, performs image processing for creating composite image information by embedding and synthesizing additional additional sub information (security information or the like) in the invisible state with respect to main image information (such as a human face image) in the visible state. The present invention relates to a method and an image processing apparatus.

Recently, with the digitization of information and the spread of the Internet, techniques such as digital watermarks and digital signatures have come to be regarded as important in order to prevent forgery and alteration of images. In particular, a digital watermark technique for embedding additional sub information in the main image information in an invisible state has been proposed as a measure against illegal copying, forgery, and falsification of a photo in which an ID card or copyright information is embedded.
For example, a digital watermark method for embedding the sub-information by superimposing the sub-information on the main image information using color difference information is disclosed (for example, see Patent Document 1).
JP 2001-268346 A

The digital watermark technology used to prevent forgery and alteration of images is a technology that embeds embedded information such as sub information (digital watermark information) in the invisible state in the main image information, but it is restored to retrieve the sub information later. It is necessary to perform processing.
Various methods have been conventionally used for restoration processing when sub-information is extracted, and among them, methods such as frequency filtering using a frequency domain are often used.
However, in the frequency filtering method, if the frequency component of the main image information and the frequency component of the embedded sub information interfere with each other, there arises a problem that it is difficult to extract using the frequency filtering method.

  Therefore, an object of the present invention is to provide an image processing method and an image processing apparatus that can stably extract sub information by a restoration process without being affected by the frequency distribution of main image information.

The image processing method of the present invention is a key information peak component extraction in an image processing method for creating composite image information by embedding the sub information in an invisible state by superimposing the sub information on the main image information in the visible state. The key information used when restoring the sub-information later is converted into a two-dimensional image by the means, and then converted from the two-dimensional space region to the frequency region by performing Fourier transform, and the frequency region of the first peak component After performing the transformation from the spatial domain to the frequency domain by performing a Fourier transform on the main image information before embedding the sub information by the key information peak component extraction step for extracting the sub information and the main image frequency processing means, Processing for removing or reducing the frequency region of the first peak component obtained by the key information peak component extraction step, or before embedding the sub information A main image frequency processing step for performing filtering processing to remove or reduce a frequency region of the first peak component obtained by the key information peak component extraction step for information, and an embedding processing means, in the main image frequency processing step. An embedding process step for embedding the sub-information in the invisible state by superimposing the sub-information on the processed main image information to create composite image information.

The image processing apparatus according to the present invention is an image processing apparatus that creates composite image information by embedding the sub information in an invisible state by superimposing the sub information on the main image information in the visible state. Key information peak for extracting the frequency region of the first peak component by converting the key information used for restoring information into a two-dimensional image and then performing a Fourier transform to convert the two-dimensional space region to the frequency region. A first component obtained by the key information peak component extraction unit after the transformation from the spatial domain to the frequency domain is performed by performing Fourier transform on the main image information before embedding the sub information and the component extraction unit. Processing for removing or reducing the frequency region of the peak component, or the first peak obtained by the key information peak component extracting means for the main image information before embedding the sub information. Main image frequency processing means for performing a filtering process to remove or reduce the frequency region of the noise component, and the sub-information is invisible by superimposing the sub-information on the main image information processed by the main image frequency processing means. Embedding processing means for embedding in a state and creating composite image information.

  According to the present invention, by removing frequency components affecting the digital watermark from the original image before embedding the digital watermark, the sub-information is stably extracted by the restoration process without being affected by the frequency distribution of the main image information. It is possible to provide an image processing method and an image processing apparatus that can be used.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
First, the first embodiment will be described.
FIG. 1A schematically shows a configuration of an embedding processing system of the image processing apparatus according to the first embodiment. The embedding processing system of this image processing apparatus is applied to, for example, processing of a face image for personal authentication of an ID card, and sub-information (security information, etc.) is invisible to main image information (face image) visible to the naked eye Key information for performing so-called digital watermark embedding processing for embedding composite image information, converting key information used when restoring sub-information later into the frequency domain and extracting the frequency domain of the first peak component Key information peak component extraction unit 104 as a peak component extraction unit, a main component that removes or reduces the frequency region of the first peak component obtained by the key information peak component extraction unit 104 from the main image information before embedding the sub information. Sub-information is superimposed on the main image information processed by the main image frequency processing unit 105 and the main image frequency processing unit 105 as image frequency processing means. Embedded in the invisible state, the digital watermark embedding processing unit 106 as an embedding processing unit for creating composite image information, and the composite image information 107 created by the digital watermark embedding processing unit 106 on the recording surface of the recording medium A recording processing unit 108 serving as recording processing means for recording (printing) in a visible state is formed on the recording processing unit 108.

Next, the flow of processing in such a configuration will be described.
As input information, there are main image information 101, sub-information (digital watermark information) 102, and key information 103. The main image information 101 is, for example, a face image for personal authentication, the sub information 102 is information for improving the security of the main image information 101 or information used for authenticity determination, and information used for copyright management. This is key information used when the sub information 102 embedded as a watermark is later retrieved and restored.

  First, the key information peak component extraction unit 104 performs peak component extraction processing with the key information 103 as an input. Here, after converting the key information 103 used for restoration into a two-dimensional image, for example, by performing Fourier transform or the like, the transformation from the two-dimensional space region to the frequency region is performed, and the first peak component in the frequency region is obtained. Extract. Here, the first peak component indicates a region having the highest amplitude signal level in the frequency region.

  Next, the main image frequency processing unit 105 performs main image frequency processing using the first peak component extracted by the key information peak component extraction unit 104 and the main image information 101 as inputs. Here, for example, after performing transformation from the spatial domain to the frequency domain by performing Fourier transform or the like on the main image information 101, a frequency component corresponding to the first peak component of the key information 103 is obtained from the main image information 101. Reduce or eliminate.

  Next, the digital watermark embedding processing unit 106 receives the main image information 101, the sub information 102, and the key information 103, which have been processed by the main image frequency processing unit 105 as to the first peak component of the key information 103, as input. Perform embedding. Here, the composite image information 107 is created by embedding the sub information 102 in the invisible state in the processed main image information 101.

  Next, the recording processing unit 108 records the synthesized image information 107 created by the digital watermark embedding processing unit 106 on a recording medium as a visible image, thereby creating recorded synthesized image information (recorded material) 109. The The recorded material in this case indicates an ID card such as an employee ID card.

  Hereinafter, the sub information embedding process in FIG. 1A described above is referred to as a digital watermark embedding process.

FIG. 1B schematically shows the configuration of the restoration processing system of the image processing apparatus according to the first embodiment. The restoration processing system of this image processing apparatus performs so-called digital watermark restoration processing for restoring the sub information 102 from the recorded material 109 created by the digital watermark embedding processing of FIG. The recorded image input unit 110 that captures the converted composite image information and converts it into digital image information, the restoration processing unit 111 that restores the sub-information (digital watermark information) 102 from the captured image information, and the restored The result display processing unit 114 displays the result of the sub information 102.
The restoration processing unit 111 is a frequency detection processing unit 112 that detects a spatial frequency component of the key information 103 from the captured image information, and a reconstruction processing unit 113 that reconstructs the sub information 102 from the detected spatial frequency component. It is constituted by.

Next, the flow of processing in such a configuration will be described.
First, the recorded image input unit 110 optically reads the composite image information recorded on the recorded material 109 and inputs it as digital image information. Next, the frequency detection processing unit 112 detects the spatial frequency component of the key information 103 from the image information captured by the recorded image input unit 110. Next, the reconstruction processing unit 113 reconstructs the sub information 102 from the spatial frequency components detected by the frequency detection processing unit 112. Next, the result display processing unit 114 displays the sub information reconstructed by the reconstruction processing unit 113.

  Hereinafter, the sub information restoration processing of FIG. 1B described above is referred to as digital watermark restoration processing.

  FIG. 2 shows a specific example of the created ID card 201 (corresponding to the recorded material 109). A face image 202 for personal authentication of the owner is recorded on the ID card 201. This face image 202 is created and recorded by the process described with reference to FIG. In addition, personal management information 203 such as an identification number (so-called ID number), name, date of birth, and expiration date is recorded. By using the personal management information 203 as the sub information 102 in the digital watermark embedding process in FIG. 1A, the authentication face image 202 of the ID card 201 and the personal management information 203 are associated with each other. It becomes difficult to falsify or counterfeit a part of the network, and security can be improved.

  FIG. 3 shows an example of digital watermark embedding processing using superimposition processing. In the present embodiment, for example, a digital watermark processing method using color difference and superimposition processing described in Japanese Patent Application No. 2003-424329 can be applied. For details, please refer to the description in the above document, and only a simple principle will be described here.

First, the color difference modulation process will be described.
Embedded information is created by subjecting the key information 103 in FIG. 1A to color difference modulation processing according to the following equations (A-1) to (A-6).
When KEY (x, y) = white pixel → EMD (x, y) _−R =
+ ΔCD- _R (A-1)
EMD (x, y) _−G =
-△ CD _-G (A-2)
EMD (x, y) _−B =
-ΔCD _-B (A-3)
When KEY (x, y) = black pixel → EMD (x, y) _−R =
-△ CD _-R (A-4)
EMD (x, y) _−G =
+ ΔCD _-G (A-5)
EMD (x, y) _−B =
+ ΔCD _-B (A-6)
KEY (x, y): Key information
EMD (x, y): embedded information (color difference modulation processing result)
ΔCD: Color difference amount As a result of the color difference modulation processing, the portion corresponding to the white pixel in the key information is red component rich (R-rich), and the portion corresponding to the black pixel is cyan component rich (C-rich). become. Since red and cyan are in a relationship of physical complementary colors, adding them together results in an achromatic color. Therefore, by setting this pixel pitch to a high resolution (about 300 dpi or more) that exceeds the perceived range of the human naked eye, the embedded information that is the result of the color difference modulation processing cannot be distinguished from red and cyan by the naked eye. ) By utilizing this property, the key information pattern can be apparently replaced with achromatic information.

  In the above equation, color difference modulation is performed so that the key information is rich in the cyan component when the pixel is white, and the red component is rich when the key information is black, but since this relationship is relative, the cyan component rich and red Even if the component rich is reversed, there is no problem in principle.

Next, the superimposition process will be described.
The superimposition process defines the main image information, the embedding information for superimposition, and the composite image information at the coordinates (x, y) of the image as follows:
Main image information: SRC- _C (x, y): equivalent to FIG. 3A ... (B-1)
Embedded information: EMD- _C (x, y) ... corresponds to FIG. 3 (b) ... (B-2)
Composite image information: DES- _C (x, y) ... corresponding to FIG. 3 (c) (B-3)
x and y are coordinate values of the image (the coordinate sizes are all the same)
C = {R (red), G (green), B (blue)} plane
Each value is an integer value between 0 and 255 for 24-bit color operations.
It is expressed by the following formula.

DES- _R (x, y) = SRC- _R (x, y) + EMD- _R (x, y) (C-1)
DES- _G (x, y) = SRC- _G (x, y) + EMD- _G (x, y) (C-2)
DES- _B (x, y) = SRC- _B (x, y) + EMD- _B (x, y) (C-3)
In this method,
(1) Utilizing human visual characteristics
・ The gradation discrimination ability decreases as the frequency of the image increases.
-Color difference information is more difficult to distinguish than luminance information
(2) Example of complementary colors: Red + cyan = achromatic (white) (in case of additive color mixture)
(3) Applying complementary color relationship and color difference information to the high frequency carrier pattern image (color difference modulation processing)
Reason)
By using the sub information, it is possible to embed the sub information in the main image information in an invisible state without causing image quality deterioration.

In the example of (2) above, red and cyan (= green + blue) are complementary colors in the case of additive color mixing, and even if red and cyan are adjacent to each other, they are not recognized by human eyes. It looks difficult and achromatic.
As described in (3) above, by using the high-frequency carrier pattern image, red-rich pixels and cyan-rich pixels are repeatedly arranged, so that the human eye cannot identify these small color difference differences. The human visual characteristic that the color difference amount is determined to be plus or minus “0” is used.

  FIG. 4 shows an example in which the main image information (human face image) used in the present embodiment is converted from the spatial domain to the frequency domain. FIG. 4A shows a human face image 401 such as an authentication photograph, and FIG. 4B schematically shows an amplitude signal level in the frequency domain. In FIG. 4B, reference numeral 402 denotes a frequency distribution on the xy plane, and 403 denotes a frequency distribution on the xz plane. In general, a natural image such as a human face image has a high signal level in the center, which is a low frequency region, and the signal level decreases as it goes to the periphery, which is a high frequency region.

  FIG. 5 shows an example in which key information used in the present embodiment is converted from the spatial domain to the frequency domain. The key information in FIG. 5 uses a rectangular lattice pattern. However, when a bit string of “1” or “0” is used, for example, “1” corresponds to a black pixel and “0” corresponds to a white pixel. This can be handled by converting to a binary image.

5A shows the key information 501, FIG. 5B is an enlarged view of a part of FIG. 5A, and FIG. 5C schematically shows the signal level of the amplitude in the frequency domain. It is shown. In FIG. 5C, reference numeral 503 denotes a frequency distribution on the xy plane, and 504 denotes a frequency distribution on the xz plane. As shown in the enlarged view of FIG. 5B, the key information 502 in this example is a rectangular lattice pattern having a half cycle of horizontal KEY (P) x and vertical KEY (P) y. In such a case, the frequency distribution is
KEY (P) x: KEY (P) y = f (P) y: f (P) x (D-1)
As shown in FIG. 5 (c), there are four first peaks P1 having the relationship of FIG. 5B, and high-frequency components having an odd-numbered period of the fundamental period in the x and y directions are the second peak P2 and the third peak. Appears as P3 (not shown).

FIG. 6 schematically shows a result obtained by converting the superimposition processing represented by the above formulas (C-1) to (C-3) into a representation in the frequency domain.
6A is a frequency domain representation of main image information, FIG. 6B is a frequency domain representation of key information, and FIG. 6C is a frequency domain representation of composite image information. To be precise, in the superimposition processing, FIG. 6B shows not the frequency domain of the key information but the frequency domain expression of the embedded information, but in the color difference modulation, attention is paid to specifying one color plane of R, G, and B. In this case, since the frequency of the key information is not changed at all and is converted into embedded information as it is, it may be regarded as [frequency domain expression of key information = frequency expression of embedded information].

As a result of the superimposition process, as shown in FIG. 6C, when viewed on the xy plane, the frequency distribution of the main image information and the frequency component of the key information overlap, causing frequency interference. (See part E).
Therefore, as described with reference to FIG. 1B, it is difficult to extract only the frequency component of the key information in the frequency detection process in the restoration process, and the main image information component is also detected together.

Therefore, the above problem is solved by creating a frequency reduction function as shown in FIG. 7 and performing frequency filtering on the main image information.
The frequency reduction function ± fx1 of FIG. 7 is obtained from f (p) x of FIG.
fx1 ≦ f (p) x (E-1)
It becomes.

Further, although not shown, ± fy1 is obtained from f (p) y in FIG.
fy1 ≦ f (p) y (E-2)
It becomes.

± fx2 and ± fy2 are determined from the number of pixels of the image, and the number of pixels in the x direction Ix and the number of pixels in the y direction Iy of the main image information are
fx2 = 1 / (Ix / 2) (E-3)
fy2 = 1 / (Iy / 2) (E-4)
It becomes.

The coefficients A1 and A2 of the frequency reduction function in FIG. 7 are obtained experimentally. In this embodiment,
A1 = 1.0 (E-5)
A2 = 0.5 (E-6)
It was.

When the frequency reduction function Fr (x, y) as described above is multiplied in the state converted into the frequency region of the main image information, the changed main image information SRC2 (x, y) is expressed as follows.
SRC2 (x, y) = SRC (X, y) · Fr (x, y) (F-1)
FIG. 8A schematically shows the frequency distribution of the main image information before the change, and FIG. 8B schematically shows the frequency distribution of the main image information after the change. According to the coefficients A1 and A2 of the frequency reduction function, the spread of the frequency is narrowed in both the x and y directions while maintaining the entire peak with little change.

  FIG. 9 schematically shows the frequency distribution as a result of the superimposition processing using the main image information after the change. As shown in the figure, there is almost no frequency overlap between the main image information and the key information (embedded information) in the xy plane, and therefore no frequency interference occurs (see section F). Therefore, when the restoration process is performed, only the frequency of the key information can be detected with high accuracy.

  As described above, according to the first embodiment, by removing the frequency component that affects the digital watermark from the original image before embedding the digital watermark, frequency interference is less likely to occur during the sub information restoration process. Only the frequency of the key information can be detected with high accuracy, and the electronic watermark information (sub information) can be easily extracted.

Next, a second embodiment will be described.
FIG. 10 schematically shows a configuration of an embedding processing system of the image processing apparatus according to the second embodiment. In the embedding processing system of the image processing apparatus according to the second embodiment, the key information 103 is a plurality (N pieces) of key information 103 _{1 to} 103 _N , and accordingly, the key information peak component extraction unit 104 is also a plurality ( N) key information peak component extraction units 104 _{1 to} 104 _N , and _one key information peak component is synthesized by combining the frequency regions of the first peak components obtained by the key information peak component extraction units 104 _{1 to} 104 _N. Unlike the first embodiment, the key information peak component group synthesizing unit 115 as a key information peak component group synthesizing unit for creating a frequency region is added, and the rest is the same as the first embodiment. Therefore, explanation is omitted.

Hereinafter, the flow of processing will be described.
First, the key information peak component extraction units 104 _{1 to} 104 _N perform key information peak component extraction processing with a plurality of pieces of key information 103 _{1 to} 103 _N as inputs. For example, the key information 103 _{1 to} 103 _N used for restoration is converted into a two-dimensional image, and then subjected to Fourier transform or the like to perform conversion from the two-dimensional space region to the frequency region. One peak component is extracted. Here, the first peak component indicates a region having the highest amplitude signal level in the frequency region.

Next, the key information peak component group synthesizing unit 115 synthesizes the first peak components of the key information 103 _{1 to} 103 _N obtained by the plurality of key information peak component extracting units 104 _{1 to} 104 _N into one frequency distribution. .

Next, the main image frequency processing unit 105 performs main image frequency processing with the frequency distribution and the main image information 101 collected by the key information peak component group synthesizing unit 110 as inputs. Here, for example, the frequency distribution obtained by synthesizing the first peak components of the plurality of key information 103 _{1 to} 103 _N after performing transformation from the spatial domain to the frequency domain by performing Fourier transform or the like on the main image information 101. Are reduced or removed from the main image information 101.

Next, the digital watermark embedding processing unit 106 processes the main image information 101, the sub-information 102, and the plurality of pieces of key information 103 _{1 to} 103 _N corresponding to the first peak component by the main image frequency processing unit 105. The digital watermark embedding process is performed with the key information 103 _{1 to} 103 _N as an input. Here, the composite image information 107 is created by embedding the sub information 102 in the invisible state in the processed main image information 101.

  Next, the recording processing unit 108 records the synthesized image information 107 created by the digital watermark embedding processing unit 106 on a recording medium as a visible image, thereby creating recorded synthesized image information (recorded material) 109. The The recorded material in this case indicates an ID card such as an employee ID card.

The key information peak component group synthesizing unit 115 synthesizes key image peak components according to the following mathematical formula.
fc (x, y) = Σ (fp1 (x, y) + fp2 (x, y) +.
+ Fpi (x, y) +... + FpN (x, y)) (G-1)
fc (x, y): result of the key information peak component group synthesis unit
fpi (x, y): Result of the i-th key information peak component extraction unit The digital watermark embedding processing unit 106 can apply, for example, a digital watermark embedding processing method described in Japanese Patent Application No. 2003-424329. For details, refer to the description of the above document, and only a simple principle will be described below.

FIG. 11 shows a flow of processing for creating the composite image information by embedding the sub-information in the main image information, and will be described below with reference to this flowchart.
As input information, there are main image information 101 ′ processed by the main image frequency processing unit 105, sub information 102, and N pieces of key information 103 _{1 to} 103 _N. The key information 103 _{1 to} 103 _N is a binary image representing N (N: integer) key information of different types.

First, in N color difference amount correction processing steps 116 _{1 to} 116 _N , color difference amount correction processing is performed by using the main image information 101 ′ and a preset color difference amount ΔCd as inputs.
Next, in the N color difference modulation processing steps 117 _{1 to} 117 _N , the color difference modulation processing is performed with each result of the color difference amount correction processing steps 111 _{1 to} 111 _N and the key information 103 _{1 to} 103 _N being input. N color difference modulation processing results are obtained.

Next, the selection and combining process step 118, the respective results and sub information 102 of the color difference modulation processing step ₁₁₇ 1 _{~117 N} as input, respectively, selection and combining process is performed, one selection and combining results.
Next, in the superimposition processing step 119, the main image information 101 ′ and the result of the selection / composition processing step 118 are input, respectively, and the superimposition processing of the selection / composition processing result on the main image information 101 ′ is performed. Is created.

  As described above, according to the second embodiment, even when there are a plurality of pieces of key information, sub-information is removed by removing frequency components affecting the digital watermark from the original image before embedding the digital watermark. Since the frequency interference is less likely to occur during the restoration process, only the frequency of the key information can be detected with high accuracy, and the electronic watermark information (sub-information) can be easily extracted.

1 shows a configuration of an image processing apparatus according to a first embodiment of the present invention, in which (a) a diagram schematically shows a configuration of an embedding processing system, and (b) a diagram shows a configuration of a restoration processing system. The block diagram shown schematically. The top view which shows an example of the produced ID card typically. The schematic diagram for demonstrating the superimposition process of a digital watermark. The schematic diagram which shows the frequency distribution of main image information. The schematic diagram which shows the frequency distribution of key information. The schematic diagram which shows the frequency distribution of synthetic | combination image information. The schematic diagram which shows a frequency reduction function. The schematic diagram which shows the example of the frequency distribution process of main image information. The schematic diagram for demonstrating an effect. The block diagram which shows schematically the structure of the embedding processing system of the image processing apparatus which concerns on the 2nd Embodiment of this invention. The flowchart for demonstrating the process of a digital watermark embedding process part.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 101 ... Main image information, 102 ... Sub information (digital watermark information), 103 ... Key information, 104 ... Key information peak component extraction part (key information peak component extraction means), 105 ... Main image frequency processing part (main image frequency processing) Means), 106 ... digital watermark embedding processing section (embedding processing means), 107 ... composite image information, 108 ... recording processing section (recording processing means), 109 ... recorded matter, 201 ... ID card, 202 ... authentication face image, 203 ... personal management information.

Claims (2)

In an image processing method for creating composite image information by embedding the sub information in an invisible state by superimposing the sub information on the main image information in the visible state,
The key information peak component extracting means converts the key information used when restoring the sub information later into a two-dimensional image, and then performs a Fourier transform to convert the two-dimensional space region to the frequency region. Key information peak component extraction step for extracting the frequency region of the peak component;
Obtained by the key information peak component extraction step after performing transformation from the spatial domain to the frequency domain by performing Fourier transform on the main image information before embedding the sub information by the main image frequency processing means. Processing for removing or reducing the frequency region of the first peak component, or removing or reducing the frequency region of the first peak component obtained by the key information peak component extraction step with respect to the main image information before embedding the sub information. A main image frequency processing step for performing filtering processing;
An embedding processing step for embedding the sub-information in an invisible state by superimposing the sub-information on the main image information processed in the main image frequency processing step by an embedding processing unit, and creating composite image information;
An image processing method comprising: In an image processing apparatus that creates composite image information by embedding the sub information in an invisible state by superimposing the sub information on the main image information in the visible state,
The key information used when restoring the sub information later is converted into a two-dimensional image, and then the Fourier transform is performed to convert the two-dimensional space region to the frequency region, thereby extracting the frequency region of the first peak component. Key information peak component extraction means;
The frequency domain of the first peak component obtained by the key information peak component extraction means after performing the transformation from the spatial domain to the frequency domain by performing Fourier transform on the main image information before embedding the sub information. A main image that is subjected to a process for removing or reducing the main image information, or a filter process for removing or reducing the frequency region of the first peak component obtained by the key information peak component extracting means before embedding the sub information. Frequency processing means;
An embedding processing means for embedding the sub information in an invisible state by superimposing the sub information on the main image information processed by the main image frequency processing means, and creating composite image information;
An image processing apparatus comprising:

Images